How does this handle cache corruption?

The implementation is designed to handle corruption gracefully; any JSON decoding errors or missing keys are treated as a cache miss, prompting the system to re-process the file.

Does this work with large files?

Yes, the pattern utilizes chunked hashing (reading in 64KB chunks) to ensure large files can be processed without loading the entire content into memory at once.

Can I use this with existing functions?

Absolutely. The pattern uses a service layer wrapper approach, allowing you to add caching to existing pure functions without modifying their original code logic.

What makes content-hash caching better than path-based caching?

Path-based caching breaks if you move or rename a file. Content-hash caching identifies the file by its actual data, meaning it survives renames and automatically invalidates if the content changes.

Content-Hash File Caching

Name: Content-Hash File Caching
Author: flatrick

byflatrick

0•

데이터 과학 및 ML

Optimizes expensive file processing pipelines by caching results based on content identity rather than file paths.

This skill implements a robust, content-addressable caching system designed for high-cost operations like PDF parsing, OCR, and image analysis. By using SHA-256 hashes of file contents as cache keys, it ensures cache persistence through file moves or renames and provides automatic invalidation when contents change. The pattern emphasizes clean architecture through service layer separation, keeping core processing functions pure and decoupled from caching logic, making it ideal for high-performance CLI tools and data pipelines.

주요 기능

01Path-independent caching using SHA-256 content hashes

02Service layer wrapper for Single Responsibility Principle (SRP)

03Automatic cache invalidation upon file modification

04O(1) cache lookups via content-addressed JSON storage

05Memory-efficient chunked hashing for large file support

060 GitHub stars

사용 사례

01Building high-performance CLI tools for batch document processing

02Optimizing machine learning data preparation and extraction pipelines

03Reducing latency and API costs in AI-driven image and text analysis

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add flatrick/mdt content-hash-cache-pattern

For use in Claude.ai and ChatGPT

주요 기능

01Path-independent caching using SHA-256 content hashes

02Service layer wrapper for Single Responsibility Principle (SRP)

03Automatic cache invalidation upon file modification

04O(1) cache lookups via content-addressed JSON storage

05Memory-efficient chunked hashing for large file support

060 GitHub stars

사용 사례

01Building high-performance CLI tools for batch document processing

02Optimizing machine learning data preparation and extraction pipelines

03Reducing latency and API costs in AI-driven image and text analysis

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add flatrick/mdt content-hash-cache-pattern

For use in Claude.ai and ChatGPT