What happens if a cache file is corrupted?

The pattern is designed to handle corruption gracefully by treating invalid or unreadable JSON files as a cache miss, triggering a re-process rather than a crash.

Does this pattern handle large files efficiently?

Yes, the implementation uses chunked reading (64KB chunks) for hashing, which prevents high memory consumption even when processing very large files.

Why use content hashes instead of file paths for caching?

Content hashes allow for cache hits even if files are renamed or moved. It also ensures that the cache is automatically invalidated the moment the file content changes, preventing stale data.

Can I add this to my existing code without refactoring?

Yes, the pattern utilizes a service-layer wrapper approach, allowing you to wrap your existing pure functions with caching logic without changing the original function code.

How does the storage mechanism work?

Results are stored as individual JSON files named by their hash. This allows for direct O(1) file system lookups without the complexity or bottleneck of a central index file.

Content-Hash File Caching

Name: Content-Hash File Caching
Author: flatrick

byflatrick

0•

데이터 과학 및 ML

Implements path-independent, auto-invalidating file processing caches using SHA-256 content hashes for maximum performance.

This skill provides a robust architectural pattern for caching expensive file processing results, such as PDF extraction, OCR, or image analysis. By using SHA-256 content hashes as cache keys instead of file paths, the system ensures cache persistence even if files are moved or renamed, while providing automatic invalidation the moment content changes. It promotes a clean service-layer separation, keeping your core processing functions pure while offering efficient, O(1) file-based storage that requires no central index file.

주요 기능

01SHA-256 content-based cache keys for path-independent result retrieval

02Automatic cache invalidation triggered by any change in file content

03Service layer separation to keep processing functions pure and testable

04O(1) lookup performance using individual hash-named JSON storage files

050 GitHub stars

06Memory-efficient chunked hashing logic for processing large files safely

사용 사례

01Image analysis and computer vision tasks where results are reused across runs

02High-cost document processing pipelines involving PDFs or large text files

03Building CLI tools that require robust and reliable --cache/--no-cache flags

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add flatrick/mdt content-hash-cache-pattern

For use in Claude.ai and ChatGPT

주요 기능

01SHA-256 content-based cache keys for path-independent result retrieval

02Automatic cache invalidation triggered by any change in file content

03Service layer separation to keep processing functions pure and testable

04O(1) lookup performance using individual hash-named JSON storage files

050 GitHub stars

06Memory-efficient chunked hashing logic for processing large files safely

사용 사례

01Image analysis and computer vision tasks where results are reused across runs

02High-cost document processing pipelines involving PDFs or large text files

03Building CLI tools that require robust and reliable --cache/--no-cache flags

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add flatrick/mdt content-hash-cache-pattern

For use in Claude.ai and ChatGPT