OpenLLaMA is an open-source reproduction of Meta AI’s LLaMA large language model, offering 3B, 7B, and 13B models trained on 1T tokens. The project provides PyTorch and JAX weights, evaluation results, and comparisons with the original LLaMA models. OpenLLaMA is trained on the RedPajama dataset and exhibits comparable performance to the original LLaMA and GPT-J across various tasks. The models can be loaded with Hugging Face Transformers and evaluated with LM-Eval-Harness.
Read more…