MiniGPT-4

MiniGPT-4 is the vision-language model from KAUST's Vision-CAIR lab that demonstrated GPT-4-like multimodal abilities with remarkable efficiency: it aligns a frozen visual encoder with a frozen Vicuna LLM using only a single projection layer, trained on about 5 million image-text pairs plus a small curated conversational set.

Published as arXiv 2304.10592 in April 2023, it was among the first open efforts to show detailed image description, website generation from hand-drawn drafts, and creative image-grounded writing emerging from simple visual alignment. It directly inspired the LLaVA line and the broader open vision-language boom. Check the project's repository for its license terms before commercial use.