generate_llm_response/generate_llm_response.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d431b273",
   "metadata": {},
   "source": [
    "# Generating LLM Responses to Customer Service Requests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "926caab2-f041-4489-92eb-1e159a621499",
   "metadata": {},
   "source": [
    "This notebook demonstrates how to instruct a LLM to simulate the role of a customer service agent and generate responses to customer service queries.\n",
    "\n",
    "See how you can evaluate the quality of these LLM generated responses in this tutorial: [Detecting Issues in LLM Outputs](https://help.cleanlab.ai/tutorials/llm/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baa6dbea-2e4f-479f-9ae0-8caac6ec361c",
   "metadata": {},
   "source": [
    "## Import Dependencies"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1988f46-f5b9-4a3e-8818-957ce52d9145",
   "metadata": {},
   "source": [
    "Make sure you have wget installed to run this tutorial. You can use pip to install all other packages required for this tutorial as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "59c1b469-9e5d-48c7-9205-d44d19ea5dc2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install pandas scipy openai transformers torch accelerate bitsandbytes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7f48f382",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline\n",
    "import openai\n",
    "\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4f749d0-892a-4f3d-93b1-f0714024de22",
   "metadata": {},
   "source": [
    "## Fetch and View Customer Service Requests\n",
    "\n",
    "To fetch the data for this tutorial, make sure you have wget installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13335149-2a99-4d9a-8ed0-c3630411857e",
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -nc https://cleanlab-public.s3.amazonaws.com/Datasets/llm-customer-service-prompts.csv -P data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63f9942d-8fd3-468b-ab29-21064e67b61d",
   "metadata": {},
   "source": [
    "For this notebook we have a csv file that contains various customer service queries, let's view a selection of these requests:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e0a1984a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['help trying to update the delivery address',\n",
       "       'want help to get my invoices from {{Person Name}}',\n",
       "       'find information about the termination of my {{Account Type}} account',\n",
       "       'could you help me to correct my shipping address?',\n",
       "       'i cannot make payments help to report an error'], dtype=object)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv(\"data/llm-customer-service-prompts.csv\")\n",
    "\n",
    "requests = data[\"request\"].values\n",
    "requests[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df1a0f85",
   "metadata": {},
   "source": [
    "## Obtaining Responses from Llama 2 \n",
    "\n",
    "First, we demonstrate how to generate responses to our customer service requests using Llama 2, we can load the model from Hugging Face and create a pipeline for text generation.\n",
    "\n",
    "Note that this portion of the notebook requires GPU, if that is unavailable, scroll to the next section to see how to obtain responses using OpenAI's API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f1a9f647",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_id = 'meta-llama/Llama-2-7b-chat-hf' \n",
    "# model_id = \"mistralai/Mistral-7B-v0.1\"  # alternatively, can use mistral by changing the model_id\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
    "model = AutoModelForCausalLM.from_pretrained(\n",
    "    model_id,\n",
    "    load_in_8bit=True,\n",
    "    device_map=\"auto\", \n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "10c2268c",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = pipeline(\n",
    "    \"text-generation\",\n",
    "    model=model, \n",
    "    tokenizer=tokenizer, \n",
    "    max_length=400,\n",
    "    temperature=0.8, # can adjust temp \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfbd45d4-2fc4-4747-9bec-4675ff664370",
   "metadata": {},
   "source": [
    "Next, we create a prompt template which we will use to format the inputs to the LLM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e3f60620",
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_llama_prompt(text):\n",
    "    return f\"\"\"You are a customer service agent, provide a response with next steps to the following question: \n",
    "    {text}\n",
    "    \n",
    "    Answer:\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "563d7855-c7a2-427c-9e41-2c336dd8abdd",
   "metadata": {},
   "source": [
    "Finally, we pass the requests to the LLM using the formatted prompt and save its outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cec55b1a",
   "metadata": {},
   "outputs": [],
   "source": [
    "text_gen = (format_llama_prompt(t) for t in requests)\n",
    "llama2_outputs = np.zeros(len(requests)).astype(object)\n",
    "\n",
    "for i, out in tqdm(enumerate(pipe(text_gen))):\n",
    "    llama2_outputs[i] = out[0][\"generated_text\"].split(\"Answer:\")[1].strip()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb55ef62-54c0-43ad-8170-8cc131d8c9ff",
   "metadata": {},
   "source": [
    "Let's view a sample response from Llama 2:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "845b4a9d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thank you for reaching out to us. I apologize for any inconvenience you're experiencing with your delivery address. To update your delivery address, please follow these steps:\n",
      "\n",
      "1. Log in to your account on our website or mobile app.\n",
      "2. Click on the \"Account\" or \"Settings\" tab.\n",
      "3. Scroll down to the \"Delivery\" section and click on \"Edit.\"\n",
      "4. Enter your new delivery address and confirm.\n",
      "5. If you have any trouble updating your address, please contact our customer service team for further assistance.\n",
      "\n",
      "Remember, you can also track your packages and view your order history in the \"Account\" or \"Settings\" tab. If you have any other questions or concerns, feel free to reach out to us.\n"
     ]
    }
   ],
   "source": [
    "print(llama2_outputs[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b45cb194",
   "metadata": {},
   "source": [
    "## Obtaining Responses from OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "888372fc-be3c-49dc-8317-d45fd9f1962c",
   "metadata": {},
   "source": [
    "Here, we demonstrate how to generate responses to our customer service requests using the OpenAI API. We wil be using the `gpt-3.5-turbo` model in this notebook.\n",
    "\n",
    "Firstly, we need to enter our OpenAI API key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "574d21bf-a220-42d1-b44e-8994302844ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "openai.api_key = \"<insert API key>\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "181de07d-c80b-4800-9b9a-33047a6acddf",
   "metadata": {},
   "source": [
    "Next, we define a function to format our prompts into the appropriate format to pass to the OpenAI API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "61983036-46a9-4c68-b94a-dbcefa54337f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_openai_prompt(text):\n",
    "    return [\n",
    "        {\"role\": \"system\", \"content\": \"You are a customer service agent, provide a response with next steps to the following question in a few sentences.\"},\n",
    "        {\"role\": \"user\", \"content\": text}\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dca40407-3919-4682-8ef9-f685fb234a77",
   "metadata": {},
   "source": [
    "Then, we pass the requests to the LLM using the formatted prompt and save its outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ad2ad74-b0b0-423c-a4b9-21c486440e55",
   "metadata": {},
   "outputs": [],
   "source": [
    "openai_outputs = np.zeros(len(requests)).astype(object)\n",
    "\n",
    "for i, text in tqdm(enumerate(requests)):\n",
    "    output = openai.ChatCompletion.create(\n",
    "        model=\"gpt-3.5-turbo\",\n",
    "        messages=format_openai_prompt(text),\n",
    "        temperature=0.8,  # can adjust temp \n",
    "    ) \n",
    "    \n",
    "    openai_outputs[i] = output.choices[0].message.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28868b63-7395-432a-a47f-11c0428b0491",
   "metadata": {},
   "source": [
    "Finally, let's view a sample response from GPT-3.5:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "8daa8dfb-8e36-41de-848c-6f99fdd06d0d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I apologize for any inconvenience you may be facing with updating your delivery address. In order to assist you further, please provide me with your order number and the new delivery address. Once I have this information, I will be able to initiate the process of updating the delivery address for you. Thank you for your patience.\n"
     ]
    }
   ],
   "source": [
    "print(openai_outputs[0])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}