Segment Anything

Released April 5, 2023 alongside the SA-1B dataset (1.1 billion masks across 11 million licensed images — the largest segmentation dataset ever published), the Segment Anything Model turned segmentation into a prompting problem: give it a point, box, or rough mask, and it returns a valid segmentation mask for any object, zero-shot, even for objects never seen in training.

SAM's architecture pairs a heavyweight Vision Transformer image encoder (computed once per image) with a lightweight prompt encoder and mask decoder, so it responds to prompts cheaply. Its zero-shot results were often competitive with fully supervised models. The weights are Apache 2.0; the SA-1B dataset carries a custom research license. SAM 2 (2024) later extended it to video.