Deep Learning

Challenging the Abilities of Large Language Models in Italian: a Community Initiative featured image

Challenging the Abilities of Large Language Models in Italian: a Community Initiative

The rapid progress of Large Language Models (LLMs) has transformed natural language processing and broadened its impact across research and society. Yet, systematic evaluation of …

Malvina Nissim
,
Danilo Croce
,
Viviana Patti
,
Pierpaolo Basile
,
Giuseppe Attanasio
,
Elio Musacchio
,
Matteo Rinaldi
,
Federico Borazio
,
Maria Francis
,
Jacopo Gili
,
others
BERnaT: Basque Encoders for Representing Natural Textual Diversity featured image

BERnaT: Basque Encoders for Representing Natural Textual Diversity

Language models depend on massive text corpora that are often filtered for quality, a process that can unintentionally exclude non-standard linguistic varieties, reduce model …

Ekhi Azurmendi
,
Joseba Fernandez de Landa
,
Jaione Bengoetxea
,
Maite Heredia
,
Julen Etxaniz
,
Mikel Zubillaga
,
Ander Soraluze
,
Aitor Soroa
Multimodal Large Language Models for Low-Resource Languages: A Case Study for Basque featured image

Multimodal Large Language Models for Low-Resource Languages: A Case Study for Basque

Current Multimodal Large Language Models exhibit very strong performance for several demanding tasks. While commercial MLLMs deliver acceptable performance in low-resource …

Lukas Arana
,
Julen Etxaniz
,
Ander Salaberria
,
Gorka Azkune
Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque featured image

Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque

Instructing language models with user intent requires large instruction datasets, which are only available for a limited set of languages. In this paper, we explore alternatives to …

Oscar Sainz
,
Naiara Perez
,
Julen Etxaniz
,
Joseba Fernandez de Landa
,
Itziar Aldabe
,
Iker García-Ferrero
,
Aimar Zabala
,
Ekhi Azurmendi
,
German Rigau
,
Eneko Agirre
,
Mikel Artetxe
,
Aitor Soroa
BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data featured image

BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data

We present BabyBabelLM, a multilingual collection of datasets modeling the language a person observes from birth until they acquire a native language. We curate developmentally …

Jaap Jumelet
,
Abdellah Fourtassi
,
Akari Haga
,
Bastian Bunzeck
,
Bhargav Shandilya
,
Diana Galvan-Sosa
,
Faiz Ghifari Haznitrama
,
Francesca Padovani
,
Francois Meyer
,
Hai Hu
,
Julen Etxaniz
,
others
Truth Knows No Language: Evaluating Truthfulness Beyond English featured image

Truth Knows No Language: Evaluating Truthfulness Beyond English

We introduce a professionally translated extension of the TruthfulQA benchmark designed to evaluate truthfulness in Basque, Catalan, Galician, and Spanish. Truthfulness evaluations …

Blanca Calvo Figueras
,
Eneko Sagarzazu
,
Julen Etxaniz
,
Jeremy Barnes
,
Pablo Gamallo
,
Iria de-Dios-Flores
,
Rodrigo Agerri
HiTZ at VarDial 2025 NorSID: Overcoming Data Scarcity with Language Transfer and Automatic Data Annotation featured image

HiTZ at VarDial 2025 NorSID: Overcoming Data Scarcity with Language Transfer and Automatic Data Annotation

In this paper we present our submission for the NorSID Shared Task as part of the 2025 VarDial Workshop (Scherrer et al., 2025), consisting of three tasks: Intent Detection, Slot …

Jaione Bengoetxea
,
Mikel Zubillaga
,
Ekhi Azurmendi
,
Maite Heredia
,
Julen Etxaniz
,
Markel Ferro
,
Jeremy Barnes
GITA4CALAMITA - Evaluating the Physical Commonsense Understanding of Italian LLMs in a Multi-layered Approach: A CALAMITA Challenge featured image

GITA4CALAMITA - Evaluating the Physical Commonsense Understanding of Italian LLMs in a Multi-layered Approach: A CALAMITA Challenge

In the context of the CALAMITA Challenge, we investigate the physical commonsense reasoning capabilities of large language models (LLMs) and introduce a methodology to assess their …

Giulia Pensa
,
Ekhi Azurmendi
,
Julen Etxaniz
,
Begoña Altuna
,
Itziar Gonzalez-Dios
Latxa Euskarazko Hizkuntza-Eredua featured image

Latxa Euskarazko Hizkuntza-Eredua

Artikulu honetan Latxa hizkuntza-ereduak (HE) aurkeztuko ditugu, egun euskararako garatu diren HE handienak. Latxa HEek 7.000 miloi parametrotik 70.000 milioira bitartean dituzte, …

Naiara Perez
,
Julen Etxaniz
,
Oscar Sainz
,
Itziar Aldabe
,
German Rigau
,
Eneko Agirre
,
Ahmed Salem
,
Aitor Ormazabal
,
Mikel Artetxe
,
Aitor Soroa
BertaQA: How Much Do Language Models Know About Local Culture? featured image

BertaQA: How Much Do Language Models Know About Local Culture?

Large Language Models (LLMs) exhibit extensive knowledge about the world, but most evaluations have been limited to global or anglocentric subjects. This raises the question of how …

Julen Etxaniz
,
Gorka Azkune
,
Aitor Soroa
,
Oier Lopez de Lacalle
,
Mikel Artetxe