BioGPT

BioGPT is a domain-specific generative transformer from Microsoft Research, pretrained from scratch on roughly 15 million PubMed abstracts rather than adapted from a general model. The GPT-2-style decoder comes in a 347-million-parameter base version and a 1.5-billion-parameter BioGPT-Large, and because it learned biomedical language natively it outperformed general models on relation extraction, PubMedQA, and document classification.

Released in October 2022 under the MIT license on GitHub and Hugging Face, it set records of 44.98% F1 on BC5CDR, 38.42% on KD-DTI, 40.76% on DDI, and 78.2% PubMedQA accuracy (81.0% for Large). It has not been updated since the chat-model wave, so treat it as a lightweight, self-hostable specialist for narrow biomedical text tasks rather than a general assistant.