VLLM Benchmark
Benchmarks vLLM endpoints interactively through MCP, enabling performance evaluation of large language models.
概要
This tool facilitates interactive benchmarking of vLLM (Very Large Language Model) endpoints using MCP (Meta Control Plane). It's designed to provide a proof-of-concept for how MCP can be used to benchmark vLLM, offering insights into the performance of different language models. The tool allows users to specify the endpoint, model, and number of benchmark iterations to run, then compares the results. Note that the tool is still under development and may produce some random outputs, potentially identifying invalid JSON.
主な機能
- 1 GitHub stars
Sponsored