Europe’s largest private #AI lab #helyes @silo_AI, #tribetampere Turun yliopisto - University of Turku’s research group TurkuNLP + HPLT released the 1st multilingual large language model (LLM) for all Nordic languages + English + programming languages. By Tech.eu tech.eu/2024/05/15/sil… #NordicMade

LREC COLING 2024 has arrived! We will presenting our work on how we built the HPLT datasets! 📅 Friday 24th of May ⏰ 9.20h-9.40h 📍Room Londra ⌛️Session D3-S1-R3 - Multilinguality, Machine Translation, and Translation Aids II

Our project, HPLT, in a nutshell. Thanks LT-Innovate for spreading our voice: linkedin.com/pulse/high-per…

Still recovering from the excitement of #lreccoling2024, where we presented the HPLT resources! We introduce: - monoHPLT: monolingual collection covering 75 languages - biHPLT: parallel data for 18 language pairs - multiHPLT: synthetic data obtained through pivoting

HPLT at EAMT2024! Barry Haddow and Pinzhen "Patrick" Chen from EdinburghNLP got many questions about when and how much new HPLT data. Quick reply: autumn and a lot.

📢 Job offer: Work with us! 🤓 Institute of Formal and Applied Linguistics Matematicko-fyzikální fakulta Univerzity Karlovy is looking for 🖥️ a Front-End and ⌨️ a Back-End Java developer to work on 🇪🇺 European Open Science Cloud. More details are at ufal.mff.cuni.cz/jobs. The application deadline is 🗓️ Aug 28.

The MT Marathon continues on its third day! We already had great talks by Ondrej Bojar, @prajdabre1, Vilém Zouhar, and Elizabeth Salesky 👏 and a poster session with 10 posters 🖼️. Today, we continue with more talks, and of course, the week-long hackathon continues with interesting projects.

Today Joseph Attieh and I presented KD4MT in the MT Marathon in Prague organized by Institute of Formal and Applied Linguistics, one of my favourite events of the year! Helsinki-NLP HPLT

🚀 INTRODUCING THE LATEST HPLT MONOLINGUAL DATASETS! TL;DR: 🔍 4.5 PB of web crawls 📄 21 billion documents 💝 careful extraction, dedup, annotation and cleaning 💥 193 languages! Explore and download the new HPLT Monolingual Datasets NOW! hplt-project.org/datasets/v2.0 #HPLT

in collaboration with Indraneil Paul Peiqin Lin CIS, LMU Munich Jörg Tiedemann and others I cannot @, and funded by HPLT UTTER

Just finished: very interesting talk about Open Source and LLMs by Percy Liang from Stanford University at EMNLP 2025 : what we can and cannot do with closed vs. (various levels of) open(ess) in LLMs. Very relevant for HPLT & all future LLM projects. CLARIN ERIC Matematicko-fyzikální fakulta Univerzity Karlovy Univerzita Karlova

We are speaking about HPLT datasets and HPTL Analytics as a way to inspect them for quantitative analysis at #LI2024 today. We are introducing samples as a new feature! If you 😍 or 🤮 our dataset, this is now the time to tell us! If you want us to take a look to yours, DM us.

Join us on a new edition of the Winter School! "Pretraining Data Quality 🧐 and Multilingual Evaluation of LLMs👀" 🪂Feb. 3–5, 2025, Norway More info and registration: wiki.nlpl.eu/Community/trai… Jointly organised by HPLT and the Nordic Language Processing Laboratory (NLPL)

🥳 Amazing performance of the #HPLT v2 dataset! HuggingFace multilingual evaluation + HPLT English internal evaluation show that HPLT v2 is one of the best datasets to train LLMs. Downloads and more at either HPLT ➡️ hplt-project.org/hplt-v2-datase… or HF ➡️huggingface.co/datasets/HPLT/…

We are happy to announce the second release of HPLT bilingual datasets: - 50 English-centric language pairs = 380M parallel sentences (HPLT) 🤩 - 1,275 non-English-centric language pairs = 16.7B parallel sentences (MultiHPLT) 😮 Available at the HPLT dataset catalogue and OPUS.

New paper on the HPLT v2 dataset making-of: - pipeline documentation and code - extensive analysis of the quality and characteristics - evaluation of the performance of language models and machine translation systems trained on it 🤓Happy reading! arxiv.org/pdf/2503.10267

HPLT v2 datasets now enriched with register labels from Turun yliopisto - University of Turku. As Amanda Myntti and Veronika Laippala's show: "Appropriate metadata increases the value of a dataset". - Blog post: hplt-project.org/register-labels - Datasets + register labels (to be merged): hplt-project.org/datasets/v2.0

HPLT stopped by MTSummit2025 last week. We exchanged info with participants at a crowded poster session about HPLT v2 datasets while v3 is still in the oven. Next stop, ACL 2025!

Great use of HPLT v2 datasets! Eager to hear more about #HPLT? Join us at ACL 2025: - BoF "Multilingualism: from data crawling to evaluation" on July 29, 16:00 - Poster "An Expanded Massive Multilingual Dataset for High-Performance Language Technologies" on July 30, 11:00

It's happening now. Our HPLT v2 dataset language coverage is awesome, provides competitive and stable results and complements other data beautifully. We are at ACL 2025, come and say hi! #hplt #datasets