GPT-4: Falcon-40B is a cutting-edge causal decoder-only model developed by TII, trained on 1,000B tokens of RefinedWeb and curated corpora. Designed for research and further finetuning, it supports multiple languages, including English, German, Spanish, and French. Falcon-40B is available under the TII Falcon LLM License, allowing free use for research and personal purposes, with certain conditions for commercial use. The model was trained on AWS SageMaker using 384 A100 40GB GPUs and a custom distributed training codebase, Gigatron.
Read more…