What is the recommended sample size for statistical assertions?

The skill provides a calibrated scale: N=400 for standard moderate precision (+/-5%) and N=10,000 for critical validations requiring high precision (+/-1%).

When should I use Bootstrap versus Clopper-Pearson patterns?

Use Clopper-Pearson for simple binary outcomes (success/failure) and use Bootstrap for more complex statistics like medians, percentiles, or custom metrics.

Can I test time-series simulations with this skill?

Yes, it includes patterns for 'Tolerance Bands,' which are specifically designed to test simulated paths and trends over time.

How does this skill prevent flaky tests in CI?

It implements a strategy of using fixed, documented seeds for CI environments, ensuring that random processes produce reproducible results every time the test suite runs.

Statistical Testing & Simulation Validation

Name: Statistical Testing & Simulation Validation
Author: nikhillinit

bynikhillinit

0•

セキュリティとテスト

Ensures the reliability of probabilistic code and Monte Carlo simulations through rigorous statistical assertions and stable testing patterns.

This skill provides a comprehensive framework for testing non-deterministic code, such as simulations, statistical estimates, and random sampling, without the common pitfall of flaky CI tests. It guides developers through choosing the right testing approach—ranging from deterministic mocking for control flows to seeded statistical tests for distributional properties. By implementing these patterns, teams can maintain high confidence in their probabilistic algorithms while ensuring that CI/CD pipelines remain stable, reproducible, and mathematically sound.

主な機能

01Standardizes sample size (N) selection based on desired precision for proportions and means.

020 GitHub stars

03Eliminates flaky CI tests using seeded statistical validation and deterministic mocking.

04Provides a decision tree for choosing between mock, seeded, and random-seed testing approaches.

05Includes implementation patterns for Clopper-Pearson, Bootstrap, and Tolerance Band confidence intervals.

06Integrates property-based testing to validate invariants across diverse simulation outcomes.

ユースケース

01Testing financial models or portfolio simulations for expected returns and risk profiles.

02Validating machine learning model predictions and distributional properties in a pipeline.

03Verifying the robustness of stochastic algorithms against edge cases using fast-check patterns.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add nikhillinit/updog_restore statistical-testing

For use in Claude.ai and ChatGPT

Download Skill