HAIPipe ML Data Preparation FAQs

Question 1

Is the pipeline restricted to specific data types?

Accepted Answer

No, the framework is domain-agnostic. It applies equally to CGM, EHR, wearables, NLP, and any tabular or sequence data where structured feature extraction is required.

Question 2

Does this skill support time-series data?

Accepted Answer

Yes, it includes specialized support for Two-Stage TEToken transforms, allowing you to convert time-series fields into continuous timesteps, discrete tokens, or sparse event dictionaries.

Question 3

What are 'Chefs' and 'Recipes' in this context?

Accepted Answer

It is a metaphor for the pipeline: Chefs are the Python transform functions (TfmFn/SplitFn), Recipes are the YAML configuration files, and the Kitchen is the underlying AIData_Pipeline code.

Question 4

What is an AIDataSet?

Accepted Answer

An AIDataSet is a standardized output package containing a HuggingFace DatasetDict (train/val/test splits), vocabulary JSON files (cf_to_cfvocab and feat_vocab), and a manifest for data lineage.

Question 5

How are data splits handled?

Accepted Answer

Splits are managed via SplitFn functions and YAML configurations (SplitArgs), which allow you to define rules for assigning cases to train, validation, and test sets based on metadata columns.

HAIPipe ML Data Preparation

Key Features

Use Cases

HAIPipe ML Data Preparation

Key Features

Use Cases