Fly.io

Fly.io runs applications as lightweight Firecracker microVMs across a global edge network, with Docker-based deploys via fly.toml, a Machines REST API for programmatic start/stop/scale, Anycast networking, and edge SQLite replication through LiteFS. Deployment ergonomics are consistently praised: package an inference server in a Dockerfile, declare the machine size and region, run fly deploy, and traffic is served from the region closest to the user.

For AI workloads the platform found real traction with self-hosted inference: practitioners report serving Ollama with 7B-class models at 200-400ms first-token latency in-region, with per-second billing and scale-to-zero making steady-state costs predictable. The important lifecycle caveat is that Fly.io discontinued GPU Machines entirely: A10, L40S, and A100 instances were deprecated July 31, 2026 and became unavailable from August 2026. GPU-hungry workloads need a different host.

The free tier for new accounts is also gone: organizations created after October 2024 get only a short trial (2 VM hours or 7 days) and must add a card. Shared-cpu microVMs start around $2/month running 24/7, with volumes at $0.15/GB/month and regionally tiered egress.

Install flyctl, run fly launch in a project directory, configure fly.toml, deploy with fly deploy.