How do I start an experiment after setup?

Once setup is complete, you can use the /ar:run command to begin iterating or /ar:loop for fully autonomous optimization.

What are built-in evaluators?

Built-in evaluators are pre-configured metrics for common tasks like checking build speed, memory usage, test pass rates, and LLM-based content scoring.

Where are the experiment configurations stored?

You can choose to store configurations either locally within a project's .autoresearch/ directory or globally in the user's home directory.

Can I use custom evaluation scripts?

Yes, the setup allows you to specify any evaluation command, such as pytest or a custom Python script, to measure your experiment's success.

What is the /ar:setup command?

It is a command within the gemini plugin for Claude Code that initializes a new research experiment by gathering configuration data like metrics and targets.

AutoResearch Setup

Name: AutoResearch Setup
Author: alirezarezvani

byalirezarezvani

•

9,958

•

Analíticas y Monitorización

Initializes and configures automated research experiments to optimize code performance, content quality, and LLM prompts.

The AutoResearch Setup skill provides an interactive or command-line interface for defining experimental parameters within Claude Code. It streamlines the creation of optimization workflows by collecting essential data like target files, evaluation commands, and performance metrics. Whether you are benchmarking API speed, reducing bundle sizes, or testing LLM judge quality, this skill automates the configuration process, establishes baseline metrics, and prepares your environment for autonomous iteration and improvement.

Características Principales

01Interactive step-by-step setup wizard for defining experimental parameters

02Built-in evaluators for performance benchmarking, test pass rates, and LLM judging

039,958 GitHub stars

04Automatic baseline calculation and experiment environment preparation

05Support for diverse domains including engineering, marketing, and prompt engineering

06Flexible storage options for project-specific or global user configurations

Casos de Uso

01Optimizing API response times or function execution speed using benchmarking evaluators

02Improving the quality of generated marketing copy or system prompts through LLM-as-a-judge scoring

03Reducing build sizes or peak memory usage in CI/CD pipelines via automated tracking

Características Principales

01Interactive step-by-step setup wizard for defining experimental parameters

02Built-in evaluators for performance benchmarking, test pass rates, and LLM judging

039,958 GitHub stars

04Automatic baseline calculation and experiment environment preparation

05Support for diverse domains including engineering, marketing, and prompt engineering

06Flexible storage options for project-specific or global user configurations

Casos de Uso

01Optimizing API response times or function execution speed using benchmarking evaluators

02Improving the quality of generated marketing copy or system prompts through LLM-as-a-judge scoring

03Reducing build sizes or peak memory usage in CI/CD pipelines via automated tracking