How does this skill optimize GPU performance?

It implements prefetching and parallel data loading using tf.data.AUTOTUNE, ensuring the CPU prepares data batches ahead of time so the GPU never remains idle.

Can I use this with custom data formats?

Yes, the skill includes templates for loading data from custom Python generators, CSVs, TFRecords, and raw image files with specialized parsing logic.

Does it support real-time data augmentation?

Absolutely. It provides ready-to-use patterns for mapping augmentations like flipping, brightness adjustment, and rotation directly into the input pipeline.

Is it compatible with Keras and distributed training?

Yes, it is designed for seamless integration with Keras model.fit() and includes patterns for DistributeStrategy to handle multi-GPU workloads.

TensorFlow Data Pipelines

Name: TensorFlow Data Pipelines
Author: TheBushidoCollective

byTheBushidoCollective

•

データサイエンスとML

Architect high-performance data input pipelines using the TensorFlow tf.data API to maximize GPU and TPU training efficiency.

This Claude Code skill provides a comprehensive framework for building scalable and optimized data pipelines within the TensorFlow ecosystem. It empowers developers to implement best practices for dataset creation from multiple sources, complex transformations, and advanced performance tuning techniques like prefetching and caching. By automating the boilerplate of data loading and preprocessing, it ensures that training hardware remains fully utilized and model iteration cycles are significantly reduced through standardized, production-ready implementation patterns.

主な機能

01Optimized dataset creation from tensors, generators, and file formats like TFRecord and CSV

02Seamless integration with Keras model.fit() and custom distributed training loops

03High-performance buffering strategies utilizing automatic prefetching and multi-threaded mapping

04Memory management patterns including disk/memory caching and stratified class sampling

05Advanced transformation pipelines featuring parallelized normalization and real-time data augmentation

0697 GitHub stars

ユースケース

01Loading and parsing large-scale structured data from TFRecord shards for distributed training

02Implementing complex stratified sampling and dynamic batching for imbalanced sequence data

03Preprocessing massive image datasets with parallelized augmentation for computer vision models

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add thebushidocollective/han tensorflow-data-pipelines

For use in Claude.ai and ChatGPT

Download Skill