04-Adding_Memory.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "01a8b5c0-87cb-4302-8e3c-dc809d0039fb",
   "metadata": {},
   "source": [
    "# Understanding Memory in LLMs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2f73380-6395-4e9f-9c83-3f47a5d7e292",
   "metadata": {},
   "source": [
    "In the previous Notebook 03, we successfully explored how OpenAI models can enhance the results from Azure Cognitive Search. [Bing Chat](http://chat.bing.com/) is a search engine with a GPT-4 model that utilizes the content of search results to provide context and deliver accurate responses to queries.\n",
    "\n",
    "However, we have yet to discover how to engage in a conversation with the LLM. With Bing Chat, this is possible, as the LLM can understand and reference the previous responses.\n",
    "\n",
    "There is a common misconception that GPT models have memory. This is not true. While they possess knowledge, they do not retain information from previous questions asked to them.\n",
    "\n",
    "The aim of this Notebook is to demonstrate how we can \"provide memory\" to the LLM by utilizing prompts and context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "733c782e-204c-47d0-8dae-c9df7091ab23",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import random\n",
    "from langchain.chat_models import AzureChatOpenAI\n",
    "from langchain.chains import LLMChain\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.chains import ConversationChain\n",
    "from langchain.chains.conversational_retrieval.prompts import CONDENSE_QUESTION_PROMPT\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "from openai.error import OpenAIError\n",
    "from langchain.docstore.document import Document\n",
    "from langchain.memory import CosmosDBChatMessageHistory\n",
    "\n",
    "from IPython.display import Markdown, HTML, display  \n",
    "\n",
    "def printmd(string):\n",
    "    display(Markdown(string))\n",
    "\n",
    "#custom libraries that we will use later in the app\n",
    "from common.utils import (\n",
    "    get_search_results,\n",
    "    order_search_results,\n",
    "    model_tokens_limit,\n",
    "    num_tokens_from_docs,\n",
    "    embed_docs,\n",
    "    search_docs,\n",
    "    get_answer,\n",
    ")\n",
    "\n",
    "from common.prompts import COMBINE_QUESTION_PROMPT, COMBINE_PROMPT, COMBINE_CHAT_PROMPT\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")\n",
    "\n",
    "import logging\n",
    "\n",
    "# Get the root logger\n",
    "logger = logging.getLogger()\n",
    "# Set the logging level to a higher level to ignore INFO messages\n",
    "logger.setLevel(logging.WARNING)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6bc63c55-a57d-49a7-b6c7-0f18bca8199e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_BASE\"] = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]\n",
    "os.environ[\"OPENAI_API_TYPE\"] = \"azure\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dc72b22-11c2-4df0-91b8-033d01829663",
   "metadata": {},
   "source": [
    "### Let's start with the basics\n",
    "Let's use a very simple example to see if the GPT model of Azure OpenAI have memory. We again will be using langchain to simplify our code "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3eef5dc9-8b80-4085-980c-865fa41fa1f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"Tell me some use cases for reinforcement learning?\"\n",
    "FOLLOW_UP_QUESTION = \"Can you summarize your last response?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a00181d5-bd76-4ce4-a256-75ac5b58c60f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define model\n",
    "MODEL = \"gpt-35-turbo\"\n",
    "# Create an OpenAI instance\n",
    "llm = AzureChatOpenAI(deployment_name=MODEL, temperature=0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9502d0f1-fddf-40d1-95d2-a1461dcc498a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We create a very simple prompt template, just the question as is:\n",
    "prompt = PromptTemplate(\n",
    "    input_variables=[\"question\"],\n",
    "    template=\"{question}\",\n",
    ")\n",
    "\n",
    "chain = LLMChain(llm=llm, prompt=prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c5c9903e-15c7-4e05-87a1-58e5a7917ba2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "1. Game playing: Reinforcement learning has been used extensively in game playing, such as in AlphaGo, which defeated the world champion in the game of Go.\n",
       "\n",
       "2. Robotics: Reinforcement learning can be used to train robots to perform complex tasks, such as grasping objects or navigating through a maze.\n",
       "\n",
       "3. Autonomous driving: Reinforcement learning can be used to train self-driving cars to make decisions on the road, such as when to brake or accelerate.\n",
       "\n",
       "4. Recommendation systems: Reinforcement learning can be used to personalize recommendations for users based on their preferences and behavior.\n",
       "\n",
       "5. Healthcare: Reinforcement learning can be used to optimize treatment plans for patients with chronic conditions, such as diabetes or hypertension.\n",
       "\n",
       "6. Finance: Reinforcement learning can be used to optimize trading strategies for financial markets.\n",
       "\n",
       "7. Manufacturing: Reinforcement learning can be used to optimize production processes in manufacturing, such as reducing waste or improving efficiency.\n",
       "\n",
       "8. Energy management: Reinforcement learning can be used to optimize energy consumption in buildings or power grids."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let's see what the GPT model responds\n",
    "response = chain.run(QUESTION)\n",
    "printmd(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "99acaf3c-ce68-4b87-b24a-6065b15ff9a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"I'm sorry, as an AI language model, I do not have access to the previous conversation. Please provide me with more context or information about the previous response, and I will try my best to summarize it for you.\""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Now let's ask a follow up question\n",
    "chain.run(FOLLOW_UP_QUESTION)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3e1c143-c95f-4566-a8b4-af8c42f08dd2",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "As you can see, it doesn't remember what it just responded. This proof that the LLM does NOT have memory and that we need to give the memory as a a conversation history as part of the prompt, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0946ce71-6285-432e-b011-9c2dc1ba7b8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "hist_prompt = PromptTemplate(\n",
    "    input_variables=[\"history\", \"question\"],\n",
    "    template=\"\"\"\n",
    "                {history}\n",
    "                Human: {question}\n",
    "                AI:\n",
    "            \"\"\"\n",
    "    )\n",
    "chain = LLMChain(llm=llm, prompt=hist_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6d088e51-e5eb-4143-b87d-b2be429eb864",
   "metadata": {},
   "outputs": [],
   "source": [
    "Conversation_history = \"\"\"\n",
    "Human: {question}\n",
    "AI: {response}\n",
    "\"\"\".format(question=QUESTION, response=response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d99e34ad-5539-44dd-b080-3ad05efd2f01",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Reinforcement learning can be used in various fields such as game playing, robotics, autonomous driving, recommendation systems, healthcare, finance, manufacturing, and energy management to optimize processes, personalize recommendations, and improve efficiency.'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.run({\"history\":Conversation_history, \"question\": FOLLOW_UP_QUESTION})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "045e5af6-55d6-4353-b3f6-3275c95db00a",
   "metadata": {},
   "source": [
    "**Bingo!**, so we now know how to create a chatbot using LLMs, we just need to keep the state/history of the conversation and pass it as context every time"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eafd1694-0077-4aa8-bd01-e9f763ce36a3",
   "metadata": {},
   "source": [
    "## Now that we understand the concept of memory via adding history as a context, let's go back to our GPT Smart Search engine"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8b27c45-7fbb-40da-a2e3-61e66a8e49b0",
   "metadata": {},
   "source": [
    "In order to not duplicate code, we have put many of the code used in Notebook 3 into functions. These functions are in the `common/utils.py` and `common/prompts.py` files This way we can use these functios in the app that we will build later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ef9f459b-e8b8-40b9-a94d-80c079968594",
   "metadata": {},
   "outputs": [],
   "source": [
    "index1_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index1_name, index2_name]\n",
    "\n",
    "agg_search_results = get_search_results(QUESTION, indexes)\n",
    "ordered_results = order_search_results(agg_search_results, reranker_threshold=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9b2a3595-c3b7-4376-b9c5-0db7a42b3ee4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Custom token limit for gpt-35-turbo : 3000\n",
      "Combined docs tokens count: 111872\n"
     ]
    }
   ],
   "source": [
    "docs = []\n",
    "for key,value in ordered_results.items():\n",
    "    for page in value[\"chunks\"]:\n",
    "        docs.append(Document(page_content=page, metadata={\"source\": value[\"location\"]}))\n",
    "\n",
    "# Calculate number of tokens of our docs\n",
    "tokens_limit = model_tokens_limit(MODEL)\n",
    "\n",
    "if(len(docs)>0):\n",
    "    num_tokens = num_tokens_from_docs(docs)\n",
    "    print(\"Custom token limit for\", MODEL, \":\", tokens_limit)\n",
    "    print(\"Combined docs tokens count:\",num_tokens)\n",
    "        \n",
    "else:\n",
    "    print(\"NO RESULTS FROM AZURE SEARCH\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c26d7540-feb8-4581-849e-003f4bf2a601",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Token count after similarity search: 4152\n",
      "Chain Type selected: map_reduce\n",
      "CPU times: user 710 ms, sys: 41.2 ms, total: 751 ms\n",
      "Wall time: 5.61 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "if num_tokens > tokens_limit:\n",
    "    index = embed_docs(docs)\n",
    "    top_docs = search_docs(index,QUESTION)\n",
    "    \n",
    "    # Now we need to recalculate the tokens count of the top results from similarity vector search\n",
    "    # in order to select the chain type: stuff or map_reduce\n",
    "    \n",
    "    num_tokens = num_tokens_from_docs(top_docs)   \n",
    "    print(\"Token count after similarity search:\", num_tokens)\n",
    "    chain_type = \"map_reduce\" if num_tokens > tokens_limit else \"stuff\"\n",
    "    \n",
    "else:\n",
    "    # if total tokens is less than our limit, we don't need to vectorize and do similarity search\n",
    "    top_docs = docs\n",
    "    chain_type = \"stuff\"\n",
    "    \n",
    "print(\"Chain Type selected:\", chain_type)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "3ce6efa9-2b8f-4810-904d-5986b4ae0372",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised ServiceUnavailableError: The server is overloaded or not ready yet..\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Some use cases for reinforcement learning include robotics, industrial manufacturing, combinatorial search problems such as game playing (e.g. backgammon), filling containers with non-identical products, controlling machinery to maintain weight constraints, and controlling robots for tasks such as juggling, box-pushing, and disk-collecting. However, there is no comprehensive list of use cases for reinforcement learning.\\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the answer\n",
    "response = get_answer(llm=llm, docs=top_docs, query=QUESTION, language=\"English\", chain_type=chain_type)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27501f1b-7db0-4ee3-9cb1-e609254ffa3d",
   "metadata": {},
   "source": [
    "And if we ask the follow up question:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5cf5b323-3b9c-479b-8502-acfc4f7915dd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The first three contents state that there is no previous response to summarize. The last content describes the standard reinforcement-learning model and how the agent should find a policy that maximizes some long-run measure of reinforcement. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = get_answer(llm=llm, docs=top_docs,  query=FOLLOW_UP_QUESTION, language=\"English\", chain_type=chain_type)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "035fa6e6-226c-400f-a504-30255385f43b",
   "metadata": {},
   "source": [
    "Until now we just have the same as the prior Notebook 03: results from Azure Search enhanced by OpenAI model, with no memory\n",
    "\n",
    "**Now let's add memory to it:**\n",
    "\n",
    "Reference: https://python.langchain.com/docs/modules/memory/how_to/adding_memory_chain_multiple_inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d98b876e-d264-48ae-b5ed-9801d6a9152b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Reinforcement learning has a wide range of practical applications, including robotics, industrial manufacturing, combinatorial search problems like computer game playing, and more. One example is Samuel's checkers playing system, which learned a value function represented by a linear function approximator and employed a training scheme similar to the updates used in value iteration, temporal differences, and Q-learning. Another example is Tesauro's application of the temporal difference algorithm to backgammon. Additionally, reinforcement learning techniques work effectively on a variety of small problems, with most interesting examples of robotic reinforcement learning employing problem decomposition to some extent. Some other use cases include filling containers with variable numbers of non-identical products while meeting various constraints, controlling a juggling robot, controlling mobile robots for box-pushing and disk-collecting tasks, and optimizing the production of containers to meet weight requirements. It's important to consider pre-programmed knowledge, different exploration strategies depending on the task at hand, and computational demands when implementing reinforcement learning algorithms. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\""
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# memory object, which is neccessary to track the inputs/outputs and hold a conversation.\n",
    "memory = ConversationBufferMemory(memory_key=\"chat_history\",input_key=\"question\")\n",
    "\n",
    "response = get_answer(llm=llm, docs=top_docs, query=QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                        memory=memory)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "bf28927b-d9ee-4412-bb07-13e055e832a7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised ServiceUnavailableError: The server is overloaded or not ready yet..\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"My previous response provided some examples of practical applications for reinforcement learning, including robotics, industrial manufacturing, and computer game playing. It's important to consider pre-programmed knowledge, different exploration strategies, and computational demands when implementing reinforcement learning algorithms. If you would like more information on the standard reinforcement-learning model, my sources provide a detailed explanation. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\""
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now we add a follow up question:\n",
    "response = get_answer(llm=llm, docs=top_docs, query=FOLLOW_UP_QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                      memory=memory)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3830b0b8-0ca2-4d0a-9747-f6273368002b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"You're welcome! Let me know if you have any other questions. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\""
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Another follow up query\n",
    "response = get_answer(llm=llm, docs=top_docs, query=\"Thank you\", language=\"English\", chain_type=chain_type,  \n",
    "                      memory=memory)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "111e732b-3c8c-4df3-8fcb-c3d01e7bec74",
   "metadata": {},
   "source": [
    "You might get a different answer on the above cell, and it is ok, this bot is not yet well configured to answer any question that is not related to its knowledge base, including salutations.\n",
    "\n",
    "Let's check our memory to see that it's keeping the conversation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1279692c-7eb0-4300-8a66-c7025f02c318",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Human: Tell me some use cases for reinforcement learning?\\nAI: Reinforcement learning has a wide range of practical applications, including robotics, industrial manufacturing, combinatorial search problems like computer game playing, and more. One example is Samuel's checkers playing system, which learned a value function represented by a linear function approximator and employed a training scheme similar to the updates used in value iteration, temporal differences, and Q-learning. Another example is Tesauro's application of the temporal difference algorithm to backgammon. Additionally, reinforcement learning techniques work effectively on a variety of small problems, with most interesting examples of robotic reinforcement learning employing problem decomposition to some extent. Some other use cases include filling containers with variable numbers of non-identical products while meeting various constraints, controlling a juggling robot, controlling mobile robots for box-pushing and disk-collecting tasks, and optimizing the production of containers to meet weight requirements. It's important to consider pre-programmed knowledge, different exploration strategies depending on the task at hand, and computational demands when implementing reinforcement learning algorithms. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\\nHuman: Can you summarize your last response?\\nAI: My previous response provided some examples of practical applications for reinforcement learning, including robotics, industrial manufacturing, and computer game playing. It's important to consider pre-programmed knowledge, different exploration strategies, and computational demands when implementing reinforcement learning algorithms. If you would like more information on the standard reinforcement-learning model, my sources provide a detailed explanation. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\\nHuman: Thank you\\nAI: You're welcome! Let me know if you have any other questions. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\""
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory.buffer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87405173",
   "metadata": {},
   "source": [
    "## Using CosmosDB as persistent memory\n",
    "\n",
    "In previous cell we have added local RAM memory to our chatbot. However, it is not persistent, it gets deleted once the app user's session is terminated. It is necessary then to use a Database for persistent storage of each of the bot user conversations, not only for Analytics and Auditing, but also if we wisg to provide recommendations. \n",
    "\n",
    "Here we will store the conversation history into CosmosDB for future auditing purpose.\n",
    "We will use a class in LangChain use CosmosDBChatMessageHistory, see [HERE](https://python.langchain.com/en/latest/_modules/langchain/memory/chat_message_histories/cosmos_db.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "c7131daa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create CosmosDB instance from langchain cosmos class.\n",
    "cosmos = CosmosDBChatMessageHistory(\n",
    "    cosmos_endpoint=os.environ['AZURE_COSMOSDB_ENDPOINT'],\n",
    "    cosmos_database=os.environ['AZURE_COSMOSDB_NAME'],\n",
    "    cosmos_container=os.environ['AZURE_COSMOSDB_CONTAINER_NAME'],\n",
    "    connection_string=os.environ['AZURE_COMOSDB_CONNECTION_STRING'],\n",
    "    session_id=\"Agent-Test-Session\" + str(random.randint(1, 1000)),\n",
    "    user_id=\"Agent-Test-User\" + str(random.randint(1, 1000))\n",
    "    )\n",
    "\n",
    "# prepare the cosmosdb instance\n",
    "cosmos.prepare_cosmos()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "d87cc7c6-5ef1-4492-b133-9f63a392e223",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create or Memory Object\n",
    "memory = ConversationBufferMemory(memory_key=\"chat_history\",input_key=\"question\",chat_memory=cosmos)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "27ceb47a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Reinforcement learning can be applied to a variety of practical use cases such as robotics, industrial manufacturing, and combinatorial search problems like computer game playing. Some specific examples of reinforcement learning applications include filling containers with variable numbers of non-identical products while meeting various constraints, controlling a juggling robot, controlling mobile robots for box-pushing and disk-collecting tasks, and optimizing a packaging task with easy constraints. Additionally, reinforcement learning is being implemented in various practical applications, and the insights and task constraints from these implementations will shape future algorithm development. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf'"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Testing using our Question\n",
    "response = get_answer(llm=llm, docs=top_docs, query=QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                        memory=memory)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "1a5ff826",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised ServiceUnavailableError: The server is overloaded or not ready yet..\n",
      "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 2.0 seconds as it raised ServiceUnavailableError: The server is overloaded or not ready yet..\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Sorry, my previous response was a summary of use cases for reinforcement learning. I mentioned that reinforcement learning can be applied to a variety of practical use cases such as robotics, industrial manufacturing, and combinatorial search problems like computer game playing. Some specific examples of reinforcement learning applications include filling containers with variable numbers of non-identical products while meeting various constraints, controlling a juggling robot, controlling mobile robots for box-pushing and disk-collecting tasks, and optimizing a packaging task with easy constraints. Additionally, reinforcement learning is being implemented in various practical applications, and the insights and task constraints from these implementations will shape future algorithm development.\\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now we add a follow up question:\n",
    "response = get_answer(llm=llm, docs=top_docs, query=FOLLOW_UP_QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                      memory=memory)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "be1620fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"You're welcome! Is there anything else you would like me to assist you with?\\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\""
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Another follow up query\n",
    "response = get_answer(llm=llm, docs=top_docs, query=\"Thank you\", language=\"English\", chain_type=chain_type,  \n",
    "                      memory=memory)\n",
    "response['output_text']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdc5ac98",
   "metadata": {},
   "source": [
    "Let's check our Azure CosmosDB to see the whole conversation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "e1d7688a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[HumanMessage(content='Tell me some use cases for reinforcement learning?', additional_kwargs={}, example=False),\n",
       " AIMessage(content='Reinforcement learning can be applied to a variety of practical use cases such as robotics, industrial manufacturing, and combinatorial search problems like computer game playing. Some specific examples of reinforcement learning applications include filling containers with variable numbers of non-identical products while meeting various constraints, controlling a juggling robot, controlling mobile robots for box-pushing and disk-collecting tasks, and optimizing a packaging task with easy constraints. Additionally, reinforcement learning is being implemented in various practical applications, and the insights and task constraints from these implementations will shape future algorithm development. \\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf', additional_kwargs={}, example=False),\n",
       " HumanMessage(content='Can you summarize your last response?', additional_kwargs={}, example=False),\n",
       " AIMessage(content='Sorry, my previous response was a summary of use cases for reinforcement learning. I mentioned that reinforcement learning can be applied to a variety of practical use cases such as robotics, industrial manufacturing, and combinatorial search problems like computer game playing. Some specific examples of reinforcement learning applications include filling containers with variable numbers of non-identical products while meeting various constraints, controlling a juggling robot, controlling mobile robots for box-pushing and disk-collecting tasks, and optimizing a packaging task with easy constraints. Additionally, reinforcement learning is being implemented in various practical applications, and the insights and task constraints from these implementations will shape future algorithm development.\\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf', additional_kwargs={}, example=False),\n",
       " HumanMessage(content='Thank you', additional_kwargs={}, example=False),\n",
       " AIMessage(content=\"You're welcome! Is there anything else you would like me to assist you with?\\nSOURCES: https://demodatasetsp.blob.core.windows.net/arxivcs/9605/9605103v1.pdf\", additional_kwargs={}, example=False)]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#load message from cosmosdb\n",
    "cosmos.load_messages()\n",
    "cosmos.messages"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5e30694-ae2a-47bb-a5c7-db51ecdbba1e",
   "metadata": {},
   "source": [
    "![CosmosDB Memory](./images/cosmos-chathistory.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6789cada-23a3-451a-a91a-0906ceb0bd14",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### Adding memory to our application allows the user to have a conversation, however this feature is not something that comes with the LLM, but instead, memory is something that we must provide to the LLM in form of context of the question.\n",
    "\n",
    "We added persitent memory using CosmosDB.\n",
    "\n",
    "We also can notice that the current chain that we are using is smart, but not that much. Although we have given memory to it, it searches for similar docs everytime, it struggles to respond to prompts like: Hello, Thank you, Bye, What's your name, What's the weather and any other task that is not search in the knowledge base.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c629ebf4-aced-45b7-a6a2-315810d37d48",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "We know now how to do a Smart Search Engine that can power a chatbot!! great!\n",
    "\n",
    "But, does this solve all the possible scenarios that a virtual assistant will require?  **What about if the answer to the Smart Search Engine is not related to text, but instead requires to look into tabular data?** The next notebook 05 explains and solves the tabular problem and the concept of Agents"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}