How does it handle potentially destructive operations?

The skill emphasizes the use of temporary directories (mktemp) and mock data to ensure tests are isolated and do not interfere with production system data.

What commands can this skill test?

It can test any command-line interface, including system commands, local binary files, shell scripts, and package scripts like npm run.

Can it help with error message improvement?

Yes, a core part of the methodology is validating that error messages are clear, actionable, and correctly routed to stderr rather than stdout.

Why is exit code verification important?

Standardized exit codes (like 0 for success and non-zero for errors) allow other scripts and CI/CD pipelines to programmatically detect failures.

Does it support different output formats?

Yes, it includes specific patterns for validating JSON, CSV, and table-formatted outputs for both structural correctness and data accuracy.

CLI Testing Patterns

Name: CLI Testing Patterns
Author: tommymorgan

bytommymorgan

•

セキュリティとテスト

Systematizes exploratory testing for command-line tools by validating help text, option flags, error handling, and output accuracy.

概要

The CLI Testing Patterns skill provides a structured framework for auditing and testing command-line interfaces, scripts, and build tools. It guides AI agents through a rigorous methodology that includes command discovery, help text parsing, exhaustive flag validation, and exit code verification. By following these systematic patterns, developers can ensure their CLI tools are robust, follow Unix conventions, and provide clear feedback to users, making it an essential utility for anyone building binary tools or complex automation scripts.

主な機能

Safe test data management using temporary directories and mock input generation
Comprehensive testing of individual flags, arguments, and complex combinations
1 GitHub stars
Correctness checks for various output formats including JSON, CSV, and table layouts
Systematic discovery and parsing of CLI help text and available options
Automated validation of exit codes and error message clarity across stderr and stdout

ユースケース

Regression testing CLI outputs and help documentation after refactoring command-line logic
Exploratory testing of legacy shell scripts to ensure they handle invalid input correctly
Verifying a new CLI tool follows standard Unix conventions and exit codes

概要

主な機能

Safe test data management using temporary directories and mock input generation
Comprehensive testing of individual flags, arguments, and complex combinations
1 GitHub stars
Correctness checks for various output formats including JSON, CSV, and table layouts
Systematic discovery and parsing of CLI help text and available options
Automated validation of exit codes and error message clarity across stderr and stdout

ユースケース

Regression testing CLI outputs and help documentation after refactoring command-line logic
Exploratory testing of legacy shell scripts to ensure they handle invalid input correctly
Verifying a new CLI tool follows standard Unix conventions and exit codes