What is hybrid multimodal retrieval?

Hybrid retrieval combines direct image embeddings with text-based captions and metadata search, ensuring higher accuracy by utilizing both visual and semantic data.

Which embedding models are supported by this skill?

The skill includes patterns for Voyage multimodal-3 (32K context), SigLIP 2, CLIP ViT-L/14, and ColPali for document-specific retrieval.

Can I use this for cross-modal search?

Yes, it specifically provides logic for projecting text queries into the same vector space as images, allowing for 'text query to image result' workflows.

How does it handle large PDF files in Claude Code?

It provides a batch-processing pattern that splits large PDFs into page-range segments to respect the 20-page and 20MB limits while maintaining context.

Is this compatible with vector databases like Milvus?

Absolutely. It includes boilerplate for setting up Milvus collections specifically optimized for multimodal vector types and document metadata.

Multimodal RAG Patterns

Name: Multimodal RAG Patterns
Author: yonatangross

byyonatangross

•

数据科学与机器学习

Implements advanced retrieval-augmented generation systems for images, text, and hybrid document content using CLIP and Voyage AI.

The Multimodal RAG skill equips Claude with specialized implementation patterns for building production-ready retrieval systems that go beyond text. It provides comprehensive guidance on integrating models like CLIP, SigLIP 2, and Voyage multimodal-3 to enable cross-modal search, visual question answering, and complex document processing. Users can leverage this skill to handle PDFs with embedded charts, build visual product searches, and implement hybrid retrieval pipelines that combine image embeddings with AI-generated captions for maximum accuracy in production environments.

主要功能

0173 GitHub stars

02Voyage multimodal-3 integration for long-context (32K token) retrieval.

03Optimized Claude Code PDF batching to handle 20-page processing limits.

04Cross-modal search implementation for text-to-image and image-to-image queries.

05Multimodal document chunking for PDFs containing text, charts, and images.

06Vector database schema design for high-performance multimodal indexing.

使用场景

01Creating AI documentation assistants that interpret charts and diagrams within technical PDFs.

02Developing image deduplication and similarity search pipelines for large datasets.

03Building e-commerce visual search engines for product discovery.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yonatangross/orchestkit multimodal-rag

For use in Claude.ai and ChatGPT

Download Skill