AitorSoroa
@aitor57
ID: 297848855
13-05-2011 06:53:21
623 Tweet
130 Followers
183 Following
Oso garrantzitsua da gizartean eragina eduki dezaketen teknologiak modu ireki batean garatzea. HiTZ zentroa (UPV/EHU)|n helburu horrekin egiten dugu lan, lizentzia irekiko datuak erabiliz euskara eta euskal kultura hizkuntza-ereduei irakaten. Erronka honetan lagundu nahi? Ikusi 🧵
Esta semana hemos participado en #Pint25ES #pint25 con la charla "La IA en la torre de Babel". Ander Barrena Madinabeitia fue el ponente en #PINT25BIO y Eneko Agirre @eagirre.bsky.social en #pint25dss
Ostegunero, HiTZ zentroko kideak biltzen gara gure ikerketen berri emateko HiTZ mintegian. Aste honetan, bi tesi proiektu aurkeztu dira: Irune Zubiaga-k "Learning to Judge: Automated Multilingual Evaluation of LLM-Generated Text" eta Blanca C-F "Critical Questions Generation"
![[1/7]
#newHitzPaper
Many languages are underserved by open LLMs, and face the following question: Which is the best way to produce open instruction-tuned LLMs for low-resource languages?
We obtained great results for a cost-effective option!
📰 arxiv.org/abs/2506.07597](https://pbs.twimg.com/media/GtJ5uiHXQAEeMc0.jpg "HiTZ zentroa (UPV/EHU) (@hitz_zentroa) on Twitter photo [1/7]
#newHitzPaper
Many languages are underserved by open LLMs, and face the following question: Which is the best way to produce open instruction-tuned LLMs for low-resource languages?
We obtained great results for a cost-effective option!
📰 arxiv.org/abs/2506.07597")
![[3/7]
🔬 Our experimental setup: 17 model variants using different backbone models (base/instruct) and data combinations (Basque corpus, English/Basque synthetic instructions).
Evaluated with 🎯 benchmarks AND🫂human preferences from 1,285 Basque speakers (12,890 annotations).](https://pbs.twimg.com/media/GtJ6aveXgAApWEB.jpg "HiTZ zentroa (UPV/EHU) (@hitz_zentroa) on Twitter photo [3/7]
🔬 Our experimental setup: 17 model variants using different backbone models (base/instruct) and data combinations (Basque corpus, English/Basque synthetic instructions).
Evaluated with 🎯 benchmarks AND🫂human preferences from 1,285 Basque speakers (12,890 annotations).")
![[4/7]
Key findings:
1⃣Language corpora is essential: models need exposure to plain Basque text
2⃣Starting from instructed models beats the standard base→instruct pipeline
3⃣English-only instructions work well, but combining with Basque instructions yields the most robust models](https://pbs.twimg.com/media/GtJ8KXJWIAAyws_.png "HiTZ zentroa (UPV/EHU) (@hitz_zentroa) on Twitter photo [4/7]
Key findings:
1⃣Language corpora is essential: models need exposure to plain Basque text
2⃣Starting from instructed models beats the standard base→instruct pipeline
3⃣English-only instructions work well, but combining with Basque instructions yields the most robust models")
![[5/7]
🎉 Bonus results!
Our 70B model approaches the performance of frontier models like GPT-4o and Claude 3.5 Sonnet on both Basque benchmarks and human evaluation, even outperforming GPT-4o on local knowledge tasks.](https://pbs.twimg.com/media/GtJ7jeHWYAAPiaF.png "HiTZ zentroa (UPV/EHU) (@hitz_zentroa) on Twitter photo [5/7]
🎉 Bonus results!
Our 70B model approaches the performance of frontier models like GPT-4o and Claude 3.5 Sonnet on both Basque benchmarks and human evaluation, even outperforming GPT-4o on local knowledge tasks.")
[7/7] 👤 Authors: Oscar Sainz, Naiara Perez, Julen Etxaniz Joseba Fernandez de Landa, Itziar Aldabe, Iker García-Ferrero, Aimar Zabala, Ekhi Azurmendi, German Rigau, Eneko Agirre @eagirre.bsky.social, Mikel Artetxe & AitorSoroa
We also had Maite Heredia present her PhD thesis so far, titled Evaluation of LLMs in Multilingual Settings: The Case of Code-Switching, which explores CS generation and evaluation for high- and low-resource language pairs.
