What information is included in the analysis report?

The report includes a list of identified flaky tests, a consistency score, detected failure patterns (like timing or memory issues), and specific recommendations for remediation.

How does this skill help with CI/CD pipelines?

It automates the comparison of CI logs against baseline runs to highlight environment-specific failures that are difficult to replicate on a local machine.

What is a flaky test?

A flaky test is a software test that yields both passing and failing results inconsistently without any changes to the code, often due to timing, environment, or randomness.

Does it work with any test runner?

While it is optimized for Mojo and ProjectOdyssey test outputs, its core logic focuses on diffing log patterns which can be adapted for various test frameworks.

Test Diff Analyzer

Name: Test Diff Analyzer
Author: mvillmow

bymvillmow

•

セキュリティとテスト

Analyzes test result differences across multiple runs to identify flaky tests and diagnose consistency issues in Mojo projects.

Test Diff Analyzer is a specialized Claude Code skill designed to eliminate the frustration of intermittent test failures. By programmatically comparing test logs across local and CI environments, it helps developers pinpoint 'flaky' tests that pass or fail inconsistently. The skill identifies specific failure patterns—such as timing issues, race conditions, or resource constraints—and provides structured reports featuring consistency scores and root cause hypotheses, making it an essential tool for maintaining stable and reliable Mojo-based software pipelines.

主な機能

01Automated comparison of multiple test run logs to detect inconsistencies

02Structured remediation recommendations for stabilizing intermittent failures

03Identification of flaky tests with specific pass/fail rate calculations

04Consistency scoring to measure overall test suite reliability

0512 GitHub stars

06Pattern recognition for timing-dependent and resource-constrained failures

ユースケース

01Debugging tests that pass in local development but fail intermittently in CI pipelines

02Identifying race conditions and non-deterministic behavior in parallel test execution

03Analyzing the impact of infrastructure or code changes on test stability

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add mvillmow/projectodyssey test-diff-analyzer

For use in Claude.ai and ChatGPT

Download Skill