llama.cpp
1 listing
Tools
Rigtune
Open-source tool that benchmarks llama.cpp, vLLM and SGLang settings on your GPU and traffic, then serves the winner behind an OpenAI-compatible API.
1 listing
Open-source tool that benchmarks llama.cpp, vLLM and SGLang settings on your GPU and traffic, then serves the winner behind an OpenAI-compatible API.