Aphrodite Engine
High-throughput open-source LLM inference engine, OpenAI-compatible.
About
Aphrodite Engine is a large-scale open-source LLM inference engine in the vLLM lineage, focused on high-throughput serving with features like distributed inference, speculative decoding, quantized KV cache, multi-LoRA support, and multimodal models. It exposes an OpenAI-compatible API server on a single command.
It ships as a Python package (pip install aphrodite-engine) and Docker images, runs on NVIDIA and AMD GPUs as well as CPUs, and targets both self-hosters and scale-out deployments. The project is copyleft-licensed under AGPL-3.0, which matters for commercial redistribution.
Community forks continue active development under names like Sonar, so check which fork is current before deploying.
What it can do
- OpenAI-compatible inference server
- Speculative decoding and paged attention
- Multi-LoRA and multimodal serving
- Distributed multi-GPU inference
Setup
pip install -U aphrodite-engine, then run aphrodite run <model> to start the OpenAI-compatible server.
Prerequisites
Python 3.10 to 3.13; CUDA 12+ GPU recommended.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 122; Trending score 74/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.