BHAGAVAD GITA SEARCH ENGINE : Bhagavad Gita AI Search using Azure

This application will help answering the questions from the Bhagavad Gita. Here I am showing a few of the questions where the reply is generated by GPT-4. It uses the technique called RAG - Retrieval Augmented Generation. Generation is performed by the Large Language Model and Retrieval Augmentation is done by external information. The external information here is the pdf document downloaded from web which summarizes the Bhagavad Gita. Below is the illustration of combination of LLM and External information.

We use Streamlit for creating the application.

RAG - Retrieval Augmented Generation

Its a technique which generates the response based on augmented prompt using our custom datasource.

Here is Application Architecture:

We put External datasources (pdfs or word files etc.) into Azure AI Search and create indexes. When user fires the query, orchestrator will pass it to Azure AI Search. The Azure AI Search gets back the knowledge from exterbal datasources and the knowledge along with the prompt again sent back to Azure OpenAI LLM by orchestrator. It gets the response back and pass it to the user.

Architecture Breakdown:

1. Data Ingestion:

File is broken into pages and uploaded to Azure Blob. The program will breaks each page into sections and each section will be loaded to Azure AI Search. Lastly indexing of each section is performed.

Here is the illustration:

2. Query

Example - Case 1:

Example - Case 2:

Execution:

1. You need to create .env file with all required configurations:

2. Run install.ipynb file cell by cell

3. Run upload_docs.ipynb file cell by cell

4. Run below command in terminal streamlit run app.py