Nous Hermes 2 Mixtral 8x7B DPO

Nous Hermes 2 Mixtral 8x7B DPO is Nous Research's early 2024 flagship, fine tuning Mixtral 8x7B on over a million entries of primarily GPT-4 generated data from the OpenHermes 2.5 collection, followed by direct preference optimization. At release it was reported as the first community model to beat Mistral's official Mixtral instruct fine tune on benchmarks.

Both SFT only and SFT plus DPO versions were released under Apache 2.0, alongside QLoRA adapters and GGUF quantizations, making it a staple of the 2024 open MoE ecosystem. It sits in a lineage that continued through Hermes 2 Pro and the Hermes 3 and 4 families.