Exploring robust RAG development with LlamaPacks, Lighthouz AI, and Llama Guard
Since the launch in late November 2023, LlamaPacks has curated over 50 packs to help jump-start your RAG pipeline development. Among these, many advanced retrieval packs emerged. In this article, let’s dive into seven advanced retrieval packs; see the diagram below.
We will perform two steps:
- Given a use case, we will generate the benchmarks using Lighthouz AutoBench and evaluate the packs with Lighthouz Eval Studio to determine which best suits our use case.
- Once the winning pack is identified, we will add Llama Guard to the RAG pipeline, tweak its custom taxonomy, re-evaluate it with Eval Studio, and observe how the evaluation score for categories such as prompt injection changes.
First, let’s look at these seven advanced retrieval LlamaPacks to see how they work under the hood.
Hybrid Fusion
This pack ensembles the vector retrievers and BM25 (Best Match 25) retrievers using fusion. BM25 estimates the relevance of documents to a given search query, helping rank documents in order of most likely relevance to the user’s needs.
Hybrid Fusion fuses results from the vector retriever and BM25 retriever out of the box; you can provide other retriever templates you want by customizing this pack.
documents = SimpleDirectoryReader(RAG_DIRECTORY).load_data()
node_parser = SimpleNodeParser.from_defaults()
nodes = node_parser.get_nodes_from_documents(documents)# download and install dependencies
HybridFusionRetrieverPack = download_llama_pack(
"HybridFusionRetrieverPack", "./hybrid_fusion_pack"
)
# create the pack
hybrid_fusion_pack = HybridFusionRetrieverPack(
nodes,
chunk_size=256,
vector_similarity_top_k=2,
bm25_similarity_top_k=2
)