Which LLM providers are supported for tracing?

It provides implementation patterns for major providers including Anthropic and OpenAI, as well as frameworks like LangChain.

Does this skill log my private prompt data?

No, the skill specifically enforces an anti-pattern against logging full prompts and responses to protect PII and reduce storage overhead.

Does it support streaming LLM responses?

Yes, it includes specialized considerations for tracking chunks and Time-to-First-Token (TTFT) to measure the responsiveness of streaming calls.

Can it calculate costs for different models automatically?

Yes, it includes logic for calculating USD costs based on input/output tokens for models like Claude 3 Opus, Sonnet, Haiku, and GPT-4o.

How does this help with debugging AI agents?

By tracking retry counts, error types, and specific failure reasons, it allows developers to identify exactly where and why an agentic workflow is failing.

LLM Call Tracing & Observability

Name: LLM Call Tracing & Observability
Author: nexus-labs-automation

bynexus-labs-automation

分析与监控

Instruments LLM API calls to capture high-fidelity observability data including latency, token usage, and costs.

关于

This skill provides standardized patterns for instrumenting Large Language Model (LLM) calls within Claude Code, enabling developers to gain deep insights into model performance and operational expenses. It enforces best practices for capturing essential metadata like provider-specific spans, token counts, and error types while ensuring sensitive data remains private by avoiding full prompt logging. Whether using direct SDKs or frameworks like LangChain, this skill helps developers optimize AI agents for cost-efficiency, speed, and production reliability.

主要功能

Comprehensive token and cost tracking for major providers like Anthropic and OpenAI
Integrated error handling and retry tracking for robust agent behavior analysis
High-precision latency monitoring including Time-to-First-Token (TTFT) for streaming
Standardized span attribute schema for consistent observability across diverse models
Privacy-first logging patterns that exclude PII by focusing on metadata over content
0 GitHub stars

使用场景

Monitoring and optimizing API costs across multiple LLM providers and models
Auditing agent reliability by tracking failure rates and specific error types in production
Debugging performance bottlenecks and high-latency model calls in AI agent workflows

关于

主要功能

Comprehensive token and cost tracking for major providers like Anthropic and OpenAI
Integrated error handling and retry tracking for robust agent behavior analysis
High-precision latency monitoring including Time-to-First-Token (TTFT) for streaming
Standardized span attribute schema for consistent observability across diverse models
Privacy-first logging patterns that exclude PII by focusing on metadata over content
0 GitHub stars

使用场景

Monitoring and optimizing API costs across multiple LLM providers and models
Auditing agent reliability by tracking failure rates and specific error types in production
Debugging performance bottlenecks and high-latency model calls in AI agent workflows