Nucleotide Transformer

The Nucleotide Transformer is a collection of genomic foundation models from InstaDeep, NVIDIA, and the Technical University of Munich, introduced via bioRxiv in January 2023. The models are pre-trained on DNA sequences from over 3,200 diverse human genomes plus 850 genomes from other species.

Across 19 benchmarks the models matched or beat task-specific methods on 15, covering molecular phenotype prediction and other genomics tasks. Nine model sizes were open-sourced, from 500M parameters up. The release uses a CC BY-NC-SA 4.0 license, so commercial use requires permission.