Can I use this with Mathlib-dependent code?

No, bisection requires self-contained test files without Mathlib imports. You should use the lean-mwe skill or a minimizer to create a standalone file first.

How does the bisection script determine a behavior change?

The script creates a 'signature' based on the program's exit code, standard output, and standard error. If any part of this signature changes between versions, it marks a behavior shift.

Can I bisect between specific dates?

Yes, you can provide a range of nightly releases (e.g., nightly-2024-01-01..nightly-2024-06-01) to narrow down the search window.

What is Lean Toolchain Bisect used for?

It is used to identify the exact Lean 4 commit or nightly release that introduced a change in code behavior, such as a bug, regression, or even a performance improvement.

What does the --ignore-messages flag do?

It tells the bisection script to only consider the process exit code when determining if a test passed or failed, ignoring differences in stdout and stderr.

Lean Toolchain Bisect

Name: Lean Toolchain Bisect
Author: leanprover

byleanprover

•

Developer Tools

Pinpoints specific Lean 4 toolchain versions or commits where code behavior changed or regressions occurred using automated bisection.

Lean Toolchain Bisect is a specialized utility designed for Lean 4 developers to automate the process of identifying regressions within the compiler or language toolchain. By analyzing self-contained test files, it executes a binary search through nightly releases or specific commits to find the exact point where a 'signature'—composed of exit codes and output—deviated. This skill is essential for debugging unexpected behavior changes in the Lean ecosystem, helping developers report bugs or fix toolchain-specific issues without manual trial and error.

Key Features

0125 GitHub stars

02Signature-based comparison using exit codes, stdout, and stderr

03Support for custom bisection ranges and execution timeouts

04Automated bisection of Lean 4 nightly releases and specific commits

05Integration with Lean guard_msgs for precise message-based testing

06Intelligent cache management to optimize build artifact performance

Use Cases

01Debugging unexpected compiler errors by finding the toolchain delta

02Identifying which Lean 4 nightly release caused a theorem to stop proving

03Locating the specific commit that introduced a performance regression

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add leanprover/skills lean-bisect

For use in Claude.ai and ChatGPT

Download Skill