What happens if a cache file becomes corrupted?

The implementation is designed to handle corruption gracefully by treating invalid or unreadable JSON files as a standard cache miss, triggering a re-process and overwriting the corrupted entry.

Is this pattern suitable for real-time data?

No, this pattern is best suited for static files or assets where processing is expensive. For real-time feeds that change constantly, the overhead of hashing might outweigh the caching benefits.

Why use content hashes instead of file paths for caching?

File paths change when files are moved or renamed, which leads to unnecessary cache misses. Content hashes identify the file by its data, ensuring the cache remains valid regardless of where the file is stored.

How does this pattern handle very large files?

The pattern utilizes chunked reading (typically 64KB increments) to compute the SHA-256 hash. This allows the system to process large files without exhausting system memory.

Content-Hash Cache Pattern

Name: Content-Hash Cache Pattern
Author: affaan-m

byaffaan-m

•

112,917

•

开发者工具

Optimizes file processing pipelines by caching results using content-based SHA-256 hashes to eliminate redundant computations.

This skill provides a robust architecture for caching expensive file processing operations, such as PDF parsing or image analysis, by using SHA-256 content hashes as unique identifiers. Unlike traditional path-based caching which breaks when files are moved or renamed, this pattern ensures cache hits survive file system changes and automatically invalidates results the moment a file's content is modified. By implementing a service-layer wrapper, it maintains a clean separation of concerns, keeping your core processing logic pure while providing efficient O(1) lookup performance for previously processed assets.

主要功能

01SHA-256 content hashing for reliable, path-independent identification

02Service layer separation to maintain pure, testable processing functions

03Automatic cache invalidation triggered by any file content change

04O(1) file-based storage using hashed filenames for rapid lookups

05Graceful corruption handling that treats invalid data as a cache miss

06112,917 GitHub stars

使用场景

01Reducing API costs for cloud-based image analysis or OCR services

02Implementing persistent caching for CLI tools that handle large-scale file transformations

03Accelerating PDF text extraction pipelines where documents are processed repeatedly

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add affaan-m/everything-claude-code content-hash-cache-pattern

For use in Claude.ai and ChatGPT

主要功能

01SHA-256 content hashing for reliable, path-independent identification

02Service layer separation to maintain pure, testable processing functions

03Automatic cache invalidation triggered by any file content change

04O(1) file-based storage using hashed filenames for rapid lookups

05Graceful corruption handling that treats invalid data as a cache miss

06112,917 GitHub stars

使用场景

01Reducing API costs for cloud-based image analysis or OCR services

02Implementing persistent caching for CLI tools that handle large-scale file transformations

03Accelerating PDF text extraction pipelines where documents are processed repeatedly

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add affaan-m/everything-claude-code content-hash-cache-pattern

For use in Claude.ai and ChatGPT