What is the primary purpose of the data-science-cli skill?

It provides a set of Unix-based tools and best practices for performing data acquisition, transformation, and validation directly from the command line.

Is this skill suitable for large datasets?

Absolutely. It leverages streaming Unix pipes and tools like GNU Parallel to process large files efficiently while maintaining a low memory footprint.

Can I use this skill for SQL queries?

Yes, it integrates DuckDB to allow for high-performance SQL-based querying of CSV and Parquet files without needing a dedicated database server.

Which data formats does this skill support?

It is highly proficient in handling CSV, JSON, and Parquet formats using specialized tools like jq, Miller (mlr), and DuckDB.

How does the color-tracking system work?

The skill uses a color-based system (Warm, Neutral, Cold) to categorize filesystem operations as Create (+1), Transform (0), or Filter/Delete (-1), facilitating easier lineage tracking.

Command-Line Data Science

Name: Command-Line Data Science
Author: plurigrid

byplurigrid

•

데이터 과학 및 ML

Provides a comprehensive suite of Unix command-line utilities for high-performance data transformation, validation, and lifecycle tracking.

This skill transforms the terminal into a robust data science environment by integrating specialized Unix utilities like jq, Miller, and csvkit. It introduces a unique color-coded filesystem tracking system to monitor data state changes while providing structured patterns for ETL pipelines and statistical analysis. By leveraging algebraic balancing and tools like DuckDB for SQL-based querying, users can perform complex data manipulation and auditing directly within the terminal with minimal overhead.

주요 기능

01High-performance SQL querying and Parquet conversion via DuckDB

02Statistical validation and schema auditing with datamash and csvstat

03Color-indexed filesystem change tracking for operational auditability

048 GitHub stars

05Stream-oriented ETL pipeline patterns for efficient data flow

06Advanced JSON and CSV processing using jq, Miller, and csvkit

사용 사례

01Managing data lineage through color-coded filesystem monitoring

02Performing rapid exploratory data analysis on large-scale log files

03Automating data cleaning and transformation in headless environments

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add plurigrid/asi data-science-cli

For use in Claude.ai and ChatGPT

Download Skill