NTCIR-19 Task Proposals: Call for Submissions

Call for task proposals for NTCIR-19, a series of evaluation conferences focusing on information access with East Asian languages and English.

Call for task proposals for NTCIR-19, a series of evaluation conferences focusing on information access with East Asian languages and English. The conference will start in September 2025 and conclude in December 2026.

Researchers are invited to submit task proposals in the field of information access, including but not limited to:

• Evaluation of the quality of content generated by large language models (LLMs)
• Information access exploiting LLMs, including generative information retrieval (IR), IR using generative queries, conversational search using generated utterances, evaluation using LLM (relevance judgements or language annotation using LLM), and RAG

Two types of task proposals are accepted: Core tasks and Pilot tasks. Core tasks focus on fostering research on a particular information access problem, while Pilot tasks focus on novel information access problems with uncertainties in task design or organization.

Proposals should be submitted as a PDF file via EasyChair by March 31, 2025 (Anywhere on Earth). The proposal should not exceed four pages in A4 single-column format and should include the task name, abstract, motivation, methodology, expected results, and details about the data to be used.

Review criteria include importance of the task, timeliness, organizers’ commitment, financial sustainability, soundness of the evaluation methodology, detailed description of the data, and language scope.

Tags: NTCIR-19, task proposals, information access, large language models, evaluation conferences, East Asian languages, English