Which vLLM flags are required for gpt-oss tool calling?

The vLLM server must be started with --tool-call-parser openai and --enable-auto-tool-choice to correctly handle tool calling for gpt-oss models.

Does this skill help with client-side llama-stack issues?

Yes, by helping you identify when errors reported by clients like llama-stack are actually originating from the vLLM server, preventing wasted time searching the client codebase for fixes.

Can I use this for both 20b and 120b gpt-oss models?

Yes, the skill covers diagnostic patterns and configuration requirements applicable to both the 20b and 120b variants of the GPT-OSS model family.

How do I fix Unexpected token errors with GPT-OSS models?

The most common fix involves updating your model's generation_config.json files from the latest HuggingFace repository and ensuring your vLLM server is running a recent version.

What causes openai_harmony.HarmonyError in vLLM?

This error originates from the vLLM server's serving layer, often indicating a mismatch between expected Harmony format control tokens and the tokens actually generated by the model.

GPT-OSS & vLLM Troubleshooter

Name: GPT-OSS & vLLM Troubleshooter
Author: bbrowning

bybbrowning

데이터 과학 및 ML

Diagnoses and resolves openai_harmony.HarmonyError and tool calling failures when using GPT-OSS models with vLLM.

소개

This skill provides specialized guidance for developers working with GPT-OSS models and vLLM serving infrastructure. It streamlines the debugging process for complex 'openai_harmony.HarmonyError' messages by distinguishing between server-side vLLM issues and client-side application errors. Users can map specific token mismatch patterns to known GitHub issues, access critical vLLM server configuration flags for tool calling, and follow a systematic workflow for updating model configuration files and verifying server health to ensure reliable function calling performance.

주요 기능

Identifies the root source of openai_harmony.HarmonyError within vLLM layers
Maps token mismatch errors to specific vLLM and GPT-OSS GitHub issues
0 GitHub stars
Provides verified vLLM server startup flags for tool-call parsing
Guides updates for model generation_config.json files from HuggingFace
Offers a structured diagnostic workflow for empty tool_calls arrays

사용 사례

Configuring vLLM servers to support tool calling with GPT-OSS-20b/120b
Fixing 'Unexpected token' errors during vLLM model inference
Resolving compatibility issues between vLLM and frameworks like llama-stack

소개

주요 기능

Identifies the root source of openai_harmony.HarmonyError within vLLM layers
Maps token mismatch errors to specific vLLM and GPT-OSS GitHub issues
0 GitHub stars
Provides verified vLLM server startup flags for tool-call parsing
Guides updates for model generation_config.json files from HuggingFace
Offers a structured diagnostic workflow for empty tool_calls arrays

사용 사례

Configuring vLLM servers to support tool calling with GPT-OSS-20b/120b
Fixing 'Unexpected token' errors during vLLM model inference
Resolving compatibility issues between vLLM and frameworks like llama-stack