2nd Shared Task on Multi-lingual Multi-task Information Retrieval at EMNLP 2024
The 2nd Shared Task on Multi-lingual Multi-task Information Retrieval is organized by the 4th Workshop on Multilingual Representation Learning, collocated with EMNLP 2024. The shared task aims to assess the multilingual characteristics of inference capability of multilingual LLMs in understanding and generating language based on logical, factual or causal relationships between knowledge contained over long contexts of text, especially under low-resource settings.
The task provides a multi-task evaluation format that assesses reading comprehension capabilities of language models in terms of two subtasks: named entity recognition (NER) and question answering (QA). The NER task identifies phrases in a text that refer to entities or predefined categories, and the QA task enables responding to natural language questions with answers found in text.
The test sets for official evaluation will be released one week before the submission date, and will be in Turkish, Uzbek, Indonesian, Alemannic, Yoruba, and Igbo languages.
Important dates:
- June 15, 2024: Release of validation data
- August 1, 2024: Release of test data
- September 1, 2024: Deadline for submission of systems
- September 20, 2024: Release of rankings and results
- September 30, 2024: Deadline for submitting system description papers
Organizers:
- Duygu Ataman
- David Ifeoluwa Adelani
- Mammad Hajili
- Francesco Tinner
- Inder Khatri
The winning team will receive an award of 500 USD and will be given a presentation during the workshop.