Can I trigger a DQ report manually in a Kubernetes cluster?

Yes, you can use 'kubectl create job' to manually trigger a run from the 'ssmd-dq-daily' CronJob or use a sed-based override to re-run for a specific historical date.

How is the data quality score calculated?

The score is calculated as the average weight of 13 specific checks, where a 'pass' is 1.0, a 'warn' is 0.7, and a 'fail' is 0.0, then multiplied by 100.

How do I run a data quality check for a specific feed?

You can run a check locally using the command 'uv run data/dq.py' followed by the specific --date, --feed, and --stream parameters for the target data source.

What do the different DQ grades (Green, Yellow, Red) mean?

Green (>= 98) indicates a healthy pipeline, Yellow (>= 85) suggests minor issues for investigation, and Red (< 85) signals significant failures requiring prompt attention.

Which data feeds are supported by the SSMD DQ runner?

The system currently supports Kalshi (crypto stream), Kraken Futures (futures stream), and Polymarket (markets stream) data feeds.

SSMD Data Quality Monitor

Name: SSMD Data Quality Monitor
Author: aaronwald

byaaronwald

•

分析与监控

Executes and monitors data quality checks for SSMD pipelines, providing automated scoring, local verification, and Kubernetes-based reporting.

The ssmd-dq-run skill provides comprehensive procedures for managing the data quality lifecycle within the SSMD ecosystem. It enables Claude to execute local DQ checks using the DQRunner engine, interpret grading metrics (Green/Yellow/Red), and manage Kubernetes CronJobs for production deployments. By utilizing DuckDB and GCS, this skill facilitates deep-dive data verification for feeds including Kalshi, Kraken Futures, and Polymarket, making it essential for engineers verifying pipeline health after deployments or performing large-scale data backfills.

主要功能

01Automated data quality scoring with categorical grading (Green/Yellow/Red)

02Programmatic DuckDB access for ad-hoc queries on GCS-hosted Parquet data

03Local execution support for Kalshi, Kraken, and Polymarket feeds via UV

041 GitHub stars

05Automated HTML email reporting with SMTP integration for pipeline alerts

06Kubernetes CronJob management for manual triggers and date-specific re-runs

使用场景

01Automating daily monitoring reports and alerting for production data feeds

02Troubleshooting data inconsistencies across different exchange streams

03Verifying pipeline health and data integrity after deployments or backfills

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add aaronwald/dlawskillz ssmd-dq-run

For use in Claude.ai and ChatGPT

Download Skill