nanochat
Karpathy's minimal open-source project: train your own ChatGPT-class LLM for ~$100 in a few hours.
About
nanochat is Andrej Karpathy's open-source project (capstone of the Eureka Labs LLM101n course) that implements a full ChatGPT-style stack — tokenization, pretraining, finetuning, evaluation, inference, and a web UI — in a single clean, dependency-lite ~8,000-line Python codebase. A single speedrun.sh script runs the entire pipeline on one 8xH100 node in about four hours for roughly $100 in cloud compute.
The reference d32 model (1.9B parameters, 38B tokens) outperforms GPT-2 (2019) but is candidly described as kindergarten-level next to modern LLMs. Its value is educational: everything is yours, readable, tweakable, and trainable end-to-end, with detailed benchmark reports (ARC, GSM8K, MMLU, HumanEval). Released under the MIT License, it became one of 2025's most-discussed LLM projects.
What it can do
- Train a 1.9B-parameter LLM end-to-end with one script in ~4 hours
- Full pipeline: tokenization, pretraining, SFT, eval, inference, web chat UI
- ~8,000-line hackable codebase with detailed benchmark reports
- Runs on a single GPU with reduced batch size for learners
Setup
Clone the repo and run speedrun.sh; requires rented GPU compute, not a signup.
Prerequisites
Access to a GPU node (reference: 8xH100, ~$100 per speedrun), Python with uv, basic ML familiarity.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 73; Trending score 80/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.