The Viking models are at 100% of training on Viking 7B, 85% on 13B and 65% on 33B, and have shown best-in-class performance in low-resource languages without compromising performance in English and programming languages. The models are trained with a dataset of 2 trillion tokens, including Danish, English, Finnish, Icelandic, Norwegian, Swedish and a variety of programming languages. The Viking model family is freely available under the Apache 2.0 License for both commercial and research use.
Key takeaways:
- Silo AI, in collaboration with University of Turku’s research group TurkuNLP and HPLT, is releasing the first multilingual large language model (LLM) for all Nordic languages, named Viking 7B.
- Viking 7B is a best-in-class open source model that is sensitive to local values and cultures, and is designed to perform well in low-resource languages without compromising English performance.
- The model is trained on the EuroHPC supercomputer LUMI, which is Europe’s most powerful supercomputer and the 5th most powerful in the world, utilizing up to 4096 AMD MI-250X GPUs.
- The Viking model family covers English, Finnish, Swedish, Norwegian, Danish, Icelandic and a variety of programming languages, and is freely available under the Apache 2.0 License for both commercial and research use.