Why is the output directory structure flat instead of hierarchical?

A flat structure (e.g., Human-X/ and Record-X.Y/) is used to simplify data loading and avoid the complexities of deep directory nesting when managing thousands of entities.

What does the 'design-chef' subcommand do?

The design-chef subcommand invokes builder scripts to automatically generate the Python code for HumanFn and RecordFn modules, ensuring they follow the required framework patterns.

What is the primary role of the Record Processor in HaiPipe?

It acts as Layer 2 of the pipeline, converting standardized source tables (SourceSet) into temporally-aligned structured records (RecordSet) mapped to specific entities.

Does this skill handle raw file ingestion?

No, raw file ingestion is handled by Layer 1 (Source). This skill (Layer 2) requires an existing SourceSet as input to generate aligned records.

Can I customize the temporal alignment intervals?

Yes. While it defaults to 5-minute intervals for CGM data, the framework is domain-configurable to support daily encounters or other temporal requirements.

HaiPipe Record Processor

Name: HaiPipe Record Processor
Author: jluo41

byjluo41

0•

Data Science & ML

Transforms raw source datasets into temporally-aligned structured record sets for academic research and machine learning.

HaiPipe Record Processor serves as Layer 2 of the 6-layer HaiPipe research data pipeline, bridging the gap between raw source tables and feature extraction. It specializes in converting SourceSets into RecordSets, where disparate data points are mapped to specific human entities and aligned to precise temporal grids (such as 5-minute intervals for CGM data). By utilizing a modular 'cooking' metaphor, the skill automates the generation of Human and Record functions, providing a standardized, flat-file architecture that ensures consistency across complex longitudinal studies in healthcare, ecology, and wearable analytics.

Key Features

01Includes subcommands for inspecting, processing (cooking), and designing pipeline infra

02Supports domain-specific temporal grid alignment (e.g., 5-minute CGM snapping)

03Implements a standardized flat-file directory structure for optimized data access

04Temporally aligns multi-source data into structured, queryable RecordSets

050 GitHub stars

06Automates HumanFn and RecordFn code generation via interactive builder scripts

Use Cases

01Building structured longitudinal datasets from raw EHR and wearable device tables

02Synchronizing continuous sensor streams with discrete event logs for clinical research

03Preparing multi-modal time-series data for downstream feature extraction and ML modeling

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add jluo41/research-skills haipipe-data-2-record

For use in Claude.ai and ChatGPT

Download Skill