VLLM Benchmark
CreatedEliovp-BV
Benchmarks vLLM endpoints interactively through MCP, enabling performance evaluation of large language models.
About
This tool facilitates interactive benchmarking of vLLM (Very Large Language Model) endpoints using MCP (Meta Control Plane). It's designed to provide a proof-of-concept for how MCP can be used to benchmark vLLM, offering insights into the performance of different language models. The tool allows users to specify the endpoint, model, and number of benchmark iterations to run, then compares the results. Note that the tool is still under development and may produce some random outputs, potentially identifying invalid JSON.
Key Features
- 1 GitHub stars
