AI RAG & Search Engineering

Acerca de

This skill provides a comprehensive framework for engineering high-performance Retrieval-Augmented Generation (RAG) and semantic search systems. It moves beyond basic implementations to focus on modern paradigms like contextual retrieval, page-level chunking, and hybrid fusion (BM25 + vector) with cross-encoder reranking. Whether you're debugging hallucinations or architecting a multi-million document search pipeline, this skill offers data-backed guidance on selecting index types, implementing adaptive retrieval systems, and validating quality using industry-standard metrics like nDCG, MRR, and groundedness.

Características Principales

• Integrates Anthropic’s 2024 Contextual Retrieval techniques to reduce retrieval failures.
• Standardizes evaluation using RAGAS, TruLens, and metrics such as nDCG and groundedness.
• Optimizes retrieval using hybrid fusion (BM25 + vector) and cross-encoder reranking.
• Provides decision trees for selecting vector indices like HNSW and IVF based on dataset scale.
• Implements advanced chunking strategies including page-level and semantic methods for higher accuracy.
• 22 GitHub stars

Casos de Uso

• Debugging and fixing hallucinations in existing retrieval-based AI systems.
• Optimizing search relevance and latency for large-scale production datasets.
• Architecting a scalable RAG pipeline for complex technical documentation.