How does this skill automate the testing process?

The skill reads your app's tool definitions from your code and automatically generates a variety of prompt categories, saving you the time of manually brainstorming test cases.

Why are negative prompts important for my app?

Negative prompts help identify 'false positives' where the AI might incorrectly trigger a tool for an unrelated query, allowing you to refine your tool descriptions for better precision.

Where are the generated prompts stored?

Prompts are saved to a file located at `.chatgpt-app/golden-prompts.json` within your project directory for easy access by other testing tools.

What are Golden Prompts in the context of ChatGPT Apps?

Golden Prompts are a curated set of test phrases used to verify that an AI model triggers specific tools only when intended, covering direct requests, indirect goals, and non-triggering phrases.

ChatGPT App Golden Prompts

Name: ChatGPT App Golden Prompts
Author: hollaugo

byhollaugo

•

Security & Testing

Generates comprehensive test datasets to validate tool triggering and intent recognition for custom ChatGPT applications.

This skill streamlines the development of custom ChatGPT Apps by automatically generating 'Golden Prompts'—a robust set of test phrases designed to verify that the LLM correctly invokes specific tools. By analyzing your application's state or server code, it produces direct commands, indirect natural language requests, and negative prompts to ensure the model handles user intent accurately while avoiding false positives. This structured approach helps developers maintain high reliability and consistent performance in how their custom tools respond to varied user inputs.

Key Features

01Creates 5+ indirect prompts per tool to validate natural language intent

0220 GitHub stars

03Automated tool analysis from server or application state code

04Generates 5+ direct prompts per tool for explicit action testing

05Exports structured test data to a standardized JSON format for CI/CD integration

06Produces negative prompts to minimize incorrect tool triggers

Use Cases

01Validating tool calling reliability before deploying a custom ChatGPT App

02Creating a benchmark dataset for regression testing during app updates

03Improving natural language understanding (NLU) for niche domain-specific tools

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add hollaugo/tutorials golden-prompts

For use in Claude.ai and ChatGPT

Download Skill