Test Effectiveness Auditor FAQs

Question 1

Why does the skill insist on reading production code first?

Accepted Answer

To prevent miscategorization. You cannot accurately judge if a test is meaningful without understanding the underlying logic it claims to exercise and verifying that it isn't just testing a mock.

Question 2

How does this differ from standard coverage tools?

Accepted Answer

Standard coverage tools only show which lines were executed. This skill analyzes the quality of the assertions and ensures tests are actually capable of catching bugs rather than just 'gaming' the metrics.

Question 3

What do the RED, YELLOW, and GREEN categories mean?

Accepted Answer

RED tests are harmful or useless (tautologies or mock-only tests). YELLOW tests are 'happy path' only or have weak assertions. GREEN tests are high-quality tests that verify actual production behavior with precise assertions.

Question 4

What is the 'bd' integration mentioned in the skill?

Accepted Answer

It integrates with the 'beads' task tracking system to automatically convert identified test weaknesses into actionable, prioritized tasks that are further refined using SRE principles.

Question 5

Which languages are supported for mutation testing?

Accepted Answer

The skill provides patterns for Pitest (Java), Stryker (JS/TS), and Mutmut (Python), but the general analysis logic can be applied to any programming language.

Test Effectiveness Auditor

Key Features

Use Cases

Test Effectiveness Auditor

Key Features

Use Cases