06-best-practices

Module 6: Best practices

In this module, we'll cover the techniques that could improve your RAG pipeline.

6.1 Techniques to Improve RAG Pipeline

• Small-to-Big chunk retrieval
• Leveraging document metadata
• Hybrid search
• User query rewriting
• Document reranking

Links:

• Slides
• Five Techniques for Improving RAG Chatbots - Nikita Kozodoi [Video]
• Survey on RAG techniques [Article]

6.2 Hybrid search

• Hybrid search strategy
• Hybrid search in Elasticsearch

Links:

• Notebook
• Hybrid search [Elasticsearch Guide]
• Hybrid search [Tutorial]

6.3 Document Reranking

• Reranking concept and metrics
• Reciprocal Rank Fusion (RRF)
• Handmade raranking implementation

Links:

• Reciprocal Rank Fusion (RRF) method [Elasticsearch Guide]
• RRF method [Article]
• Elasticsearch subscription plans

We should pull and run a docker container with Elasticsearch 8.9.0 or higher in order to use reranking based on RRF algorithm:

docker run -it \
    --rm \
    --name elasticsearch \
    -m 4GB \
    -p 9200:9200 \
    -p 9300:9300 \
    -e "discovery.type=single-node" \
    -e "xpack.security.enabled=false" \
    docker.elastic.co/elasticsearch/elasticsearch:8.9.0

6.4 Hybrid search with LangChain

• LangChain: Introduction
• ElasticsearchRetriever
• Hybrid search implementation

pip install -qU langchain langchain-elasticsearch langchain-huggingface

Links:

• Notebook
• Chatbot Implementation [Tutorial]
• ElasticsearchRetriever

Homework

TBD

Notes

• First link goes here
• Did you take notes? Add them above this line (Send a PR with links to your notes)