The OpenOrca dataset has been used to fine-tune the Mistral 7B model, resulting in a high-performing, fully open model that can run on moderate consumer GPUs. The model, codenamed “MistralOrca”, ranks #2 for all models smaller than 30B, outperforming all but one 13B model. It was trained on a curated subset of GPT-4 augmented data and is available for testing on fast GPUs.
Read more…