How do I debug a failing test?

You can use debug flags like --skip-delete and --pause-on-failure to keep resources alive and inspect their state after a test fails.

What is Chainsaw testing in the context of Ark?

Chainsaw is an E2E testing tool used within the Ark framework to validate the behavior and orchestration of agentic resources like models and agents.

What cloud providers are supported for real LLM tests?

The Ark framework is provider-agnostic, though it includes built-in support for Azure OpenAI for E2E validation.

How is an Ark test directory structured?

A standard test includes a chainsaw-test.yaml definition, a manifests folder for resource YAMLs, and a README for documentation.

Can I run tests without spending LLM credits?

Yes, this skill supports mock-llm configurations, allowing you to run comprehensive test suites locally without calling external AI providers.

Ark Chainsaw Testing

Name: Ark Chainsaw Testing
Author: mckinsey

bymckinsey

•

305

•

보안 및 테스팅

Executes and authors end-to-end tests for agentic resources using the Chainsaw framework and mock LLM configurations.

The Ark Chainsaw Testing skill enables developers to validate complex agentic application patterns by running automated end-to-end tests within the McKinsey Ark ecosystem. It facilitates the creation of robust test suites for models, agents, and queries, supporting both mock LLM environments for cost-effective local development and real LLM providers like Azure OpenAI for production-grade validation. With specialized debugging modes that allow resource persistence on failure, this skill streamlines the process of identifying and fixing orchestration issues in AI-driven workflows.

주요 기능

01Mock LLM support for fast and cost-effective local test execution.

02Seamless integration with Azure OpenAI for real-world provider validation.

03End-to-end testing for agentic resources including models, agents, and queries.

04Advanced debug mode with resource persistence and failure pausing.

05Standardized test structure using YAML manifests and Chainsaw definitions.

06305 GitHub stars

사용 사례

01Validating agentic workflow orchestration before production deployment.

02Automating regression testing for complex AI-driven application logic.

03Debugging failed resource deployments in isolated, reproducible environments.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add mckinsey/agents-at-scale-ark chainsaw

For use in Claude.ai and ChatGPT

Download Skill