Good news for small languages and LLMs. Paper on open Poro 34B model shows how training on Finnish, English and programming languages creates a very strong Finnish model, that excels in translation and is competitive in generating English and code: arxiv.org/abs/2404.01856

We will be presenting the HPLT datasets HOW-TO and insights at LREC COLING 2024 in Torino. Paper already in Arxiv.org: arxiv.org/pdf/2403.14009….

Thanks a lot Bram for making HPLT data available on HF!

HPLT partners have explored the effects of multililingual vs monolingual instruction tuning, under a constrained budget and a self-made machine-translated Alpaca-based dataset. Spoiler: go multilingual! Work will be presented at #eacl2024
Findings: arxiv.org/abs/2309.08958

Attending the HPLT & NLPL Winter School in Skeikampen, Norway was a blast and highly recommended if you are interested in Large Language Models. Bonus: we had our very own take on building a snowman!

HPLT scientific highlights directly from one of the authors :
'LTG-BERT: an efficient LM architecture developed within the HPLT project, won the BabyLM challenge. Look forward to our future release of 75 LTG-BERTs for 75 different languages.'
Thanks David Samuel and congrats!

The HPLT winter school hosted a panel discussion on Large 🦣 vs. Small 🤏 language models with Jan Hajic from Matematicko-fyzikální fakulta Univerzity Karlovy among the panelists.

It's snowing large language models this week in Norway!
1st, the 5th NLPL and HPLT Winter School on LLMs is ongoing now in Skeikampen
And 2nd, the LTG has released three fully open generative language models for Norwegian, based on Mistral and BLOOM architectures
#NLProc

Right now at Skeikampen, Norway: the HPLT&NLPL Winter School on Large Language Models: Creation, Customization, Evaluation, and Use wiki.nlpl.eu/Community/trai… is starting! Institute of Formal and Applied Linguistics Stephan Oepen LTG Oslo @unioslo #NLProc #LLM European Commission @digitaleu hplt-project.org

HPLT project presented at the online Language Data Space Workshop today language-data-space.ec.europa.eu/events/legisla… Institute of Formal and Applied Linguistics Charles University DG Connect CLARIN ERIC LINDAT/CLARIAH-CZ . Jan Hajic was a panelist @ the LDS Legal Issues panel at the workshop led by Thomas Margoni of CiTiP KU Leuven.

HPLT LLMs are currently training on the LUMI supercomputer, this blog is worth a reading if you want to know more about it!

The 2024 HPLT & NLPL Winter School focus is...

📜Large Language Models: Creation, Customisation, Evaluation, and Use.

🔥Take a look at our fabulous programme! wiki.nlpl.eu/Community/trai…

👏 Big thanks to our presenters afra alishahi, Desmond Elliott, Niklas Muennighoff and Aurélie N.!

HPLT scientific news👨‍🏫!
🎙️Collaboration between HPLT, Hugging Face, and Harvard on 'Scaling Data-Constrained Language Models' awarded runner-up outstanding paper at NeurIPS'23. Well done TurkuNLP!

Paper: arxiv.org/pdf/2305.16264…

🚨Happenning now! HPLT birds of a feather session at #EMNLP2023 . Want to know more about international academic collaboration on training very large open models? Come discuss high-performance language and translation models with us! ✌️

We just published version 1.2 of HPLT datasets. What's new?
- we fixed a bug in monolingual dedup, please redownload! 🛠️
- we filtered out very ugly monolingual documents🤮
- we anonymised the bilingual datasets🕵️‍♀️
hplt-project.org/datasets/v1.2

HPLT is at EMNLP'2023! Join us for the HPLT BoF meeting that will take place Friday 08, 16:00 in the Aquarius room. See these web pages for more BoF details: 2023.emnlp.org/program/bof/ virtual2023.emnlp.org/socials.html#t…

Took some photos of you guys, Tom Kocmi, Nikolay Bogoychev :)

HPLT News and Tools!!! If you are interested in filtering your datasets for quality and using them to train MT and LLMs, you are interested in this thread 👇

Helsinki-NLP is busy at the FCAI AI day blogs.helsinki.fi/language-techn…
Lots of interesting things including the HPLT project, the shroom2024 shared task and more. See us at the presentations and posters!