How does this prevent LLMs from 'gaming' tests?

Traditional tests can be 'gamed' if an LLM changes a threshold (like changing a timeout from 50ms to 100ms) to make a test pass. Spec tests require an intent statement; if the code violates the intent, Claude marks it as a failure regardless of the raw output.

Where should I store my spec tests?

Spec tests should be stored in the 'specs/tests/' directory. Each file should use Markdown format and include frontmatter to define its target source files.

What is an intent-based spec test?

An intent-based spec test is a natural language specification that explains why a test exists. Claude acts as a judge to determine if the code satisfies the underlying business need, rather than just checking if a technical value matches.

Do I need a separate API key for the test runner?

No, the included test runner uses the 'claude -p' command, which utilizes your existing Claude subscription and environment settings.

Intent-Based Spec Tests

Name: Intent-Based Spec Tests
Author: ianphil

byianphil

セキュリティとテスト

Implements natural language specification testing using Claude as a judge to ensure code aligns with business intent and user requirements.

概要

This skill enables a robust, intent-based testing framework that uses Claude as an evaluator to verify that your code fulfills specific business logic rather than just passing arbitrary technical assertions. By mandating intent statements for every test case, it creates a 'cheat-proof' TDD environment where Claude prevents code from 'gaming' the tests, ensuring that UX requirements, performance targets, and functional goals are strictly met. It is ideal for developers using AI-driven workflows who need to bridge the gap between natural language requirements and technical implementation.

主な機能

LLM-as-judge evaluation for natural language specifications
Multi-target support to validate logic across multiple files simultaneously
Markdown-based test format for high readability and documentation
Full TDD workflow support: Plan, Spec, Implement, and Verify
Mandatory intent statements to capture the 'why' behind every test
0 GitHub stars

ユースケース

Guiding AI code generation to ensure it meets requirements without cutting corners
Enforcing non-negotiable UX requirements like latency and responsiveness
Validating complex business logic that is difficult to express in traditional unit tests

概要

主な機能

LLM-as-judge evaluation for natural language specifications
Multi-target support to validate logic across multiple files simultaneously
Markdown-based test format for high readability and documentation
Full TDD workflow support: Plan, Spec, Implement, and Verify
Mandatory intent statements to capture the 'why' behind every test
0 GitHub stars

ユースケース

Guiding AI code generation to ensure it meets requirements without cutting corners
Enforcing non-negotiable UX requirements like latency and responsiveness
Validating complex business logic that is difficult to express in traditional unit tests