0-multi-step-tracing.ipynb

0-multi-step-tracing.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/caleb-kaiser/7f1bd0da708bbd7eb0915ed44a4dd48c/0-multi-step-tracing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/comet-ml/opik/main/apps/opik-documentation/documentation/static/img/opik-logo.svg\" width=\"250\"/>"
      ],
      "metadata": {
        "id": "9xROyQYP1DM9"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ScvGXUo3I80"
      },
      "source": [
        "# Tracking a Multi-step LLM Chain\n",
        "\n",
        "In this exercise, you'll track a multi-step LLM chain with Opik. You can use OpenAI or open source models via LiteLLM.\n",
        "\n",
        "If you have multiple steps in your LLM pipeline, you can use the `track` decorator to log the traces for each step. If OpenAI is called within one of these steps, the LLM call with be associated with that corresponding step:"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Imports & Configuration"
      ],
      "metadata": {
        "id": "2YJRuver_SmK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%pip install opik openai --quiet"
      ],
      "metadata": {
        "id": "hZ6cxMoh3cpS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from opik import track\n",
        "import opik\n",
        "from opik.integrations.openai import track_openai\n",
        "from openai import OpenAI\n",
        "import getpass\n",
        "import os\n",
        "\n",
        "os.environ[\"OPIK_PROJECT_NAME\"] = \"Multi-step-Chain-Demo\""
      ],
      "metadata": {
        "id": "Vhy3HZn63ce4"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# opik configs\n",
        "if \"OPIK_API_KEY\" not in os.environ:\n",
        "    os.environ[\"OPIK_API_KEY\"] = getpass.getpass(\"Enter your Opik API key: \")\n",
        "\n",
        "opik.configure()"
      ],
      "metadata": {
        "id": "ThX2YArw3mda"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# openai configs\n",
        "if \"OPENAI_API_KEY\" not in os.environ:\n",
        "    os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")\n",
        "client = OpenAI()\n",
        "openai_client = track_openai(client)"
      ],
      "metadata": {
        "id": "T52NO_R73qb3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Define First Step"
      ],
      "metadata": {
        "id": "rCOFr4Wd4Frj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "@track\n",
        "def generate_meal(ingredient):\n",
        "    prompt = f\"Generate one example of a meal that can be made with {ingredient}.\"\n",
        "    res = openai_client.chat.completions.create(\n",
        "        model=\"gpt-3.5-turbo\",\n",
        "        messages=[\n",
        "            {\"role\": \"user\", \"content\": prompt}\n",
        "        ]\n",
        "    )\n",
        "    return res.choices[0].message.content"
      ],
      "metadata": {
        "id": "ZJToIZM6pR5v"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Define Second Step"
      ],
      "metadata": {
        "id": "M-1R6q7W4JnZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "@track\n",
        "def generate_recipe(meal):\n",
        "    prompt = f\"Generate a step-by-step recipe for {meal}\"\n",
        "    res = openai_client.chat.completions.create(\n",
        "        model=\"gpt-3.5-turbo\",\n",
        "        messages=[\n",
        "            {\"role\": \"user\", \"content\": prompt}\n",
        "        ]\n",
        "    )\n",
        "    return res.choices[0].message.content"
      ],
      "metadata": {
        "id": "Z_iBzyQgpvEo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Call Chain"
      ],
      "metadata": {
        "id": "H_Wc5RDhCaJs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "@track\n",
        "def generate_recipe_from_ingredient(ingredient):\n",
        "    meal = generate_meal(ingredient)\n",
        "    story = generate_recipe(meal)\n",
        "    return story\n",
        "\n",
        "generate_recipe_from_ingredient(\"garlic\")"
      ],
      "metadata": {
        "id": "K6WmeCQ4p6js"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Try with your own example!"
      ],
      "metadata": {
        "id": "4fRPxqyFq83h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "generate_recipe_from_ingredient(input(\"Enter an ingredient: \"))"
      ],
      "metadata": {
        "id": "SYgndLdprBQ3"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "comet-eval",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.15"
    },
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "2YJRuver_SmK",
        "rCOFr4Wd4Frj",
        "M-1R6q7W4JnZ",
        "H_Wc5RDhCaJs",
        "4fRPxqyFq83h"
      ],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}