What is the difference between dense and sparse retrieval?

Dense retrieval uses vector embeddings to find semantic similarity, while sparse retrieval uses traditional keyword matching (like BM25). This skill teaches how to combine both using hybrid search for optimal results.

How does this skill help reduce LLM hallucinations?

It provides specific prompt engineering templates and retrieval verification steps that force the LLM to answer only based on the provided context, significantly increasing factual accuracy.

What is RAG and why is it used?

Retrieval-Augmented Generation (RAG) is a technique that provides LLMs with specific, external data at query time, allowing the model to answer questions based on information not included in its original training data.

Which vector databases are covered in this skill?

This skill includes configuration and implementation patterns for Pinecone, Weaviate, Milvus, Chroma, Qdrant, and FAISS.

Can I use this for local development?

Yes, the skill includes patterns for local-first vector stores like Chroma and FAISS, as well as lightweight embedding models suitable for local environments.

RAG Implementation Guide

Name: RAG Implementation Guide
Author: Microck

byMicrock

•

データサイエンスとML

Builds robust Retrieval-Augmented Generation systems using vector databases and semantic search to ground LLM responses in proprietary data.

概要

This skill provides a comprehensive framework for implementing Retrieval-Augmented Generation (RAG) within AI applications to ensure grounded, factual, and domain-specific responses. It covers the entire lifecycle of a RAG pipeline, from document chunking and embedding generation to advanced retrieval strategies like hybrid search and reranking. By leveraging these patterns, developers can integrate LLMs with external knowledge bases using popular tools like LangChain and vector stores such as Pinecone, Chroma, and Weaviate, effectively reducing hallucinations and enabling source-cited AI interactions.

主な機能

Comprehensive document chunking strategies including recursive, token-based, and semantic splitting
Integration guides for high-performance embedding models like OpenAI text-embedding-ada-002 and BGE
Reranking methods and evaluation metrics to optimize retrieval quality and response accuracy
Implementation patterns for leading vector databases including Pinecone, Chroma, and Milvus
Advanced retrieval techniques such as hybrid search, multi-query retrieval, and contextual compression
81 GitHub stars

ユースケース

Building enterprise Q&A systems over proprietary documentation and internal knowledge bases
Developing chatbots that access real-time or domain-specific data to minimize model hallucinations
Creating AI research assistants that provide factual answers with verifiable source citations

概要

主な機能

Comprehensive document chunking strategies including recursive, token-based, and semantic splitting
Integration guides for high-performance embedding models like OpenAI text-embedding-ada-002 and BGE
Reranking methods and evaluation metrics to optimize retrieval quality and response accuracy
Implementation patterns for leading vector databases including Pinecone, Chroma, and Milvus
Advanced retrieval techniques such as hybrid search, multi-query retrieval, and contextual compression
81 GitHub stars

ユースケース

Building enterprise Q&A systems over proprietary documentation and internal knowledge bases
Developing chatbots that access real-time or domain-specific data to minimize model hallucinations
Creating AI research assistants that provide factual answers with verifiable source citations