Join the 2nd Call for Participation in LLMs4Subjects Shared Task at SemEval 2025
Join the 2nd Call for Participation in LLMs4Subjects Shared Task organized by SemEval 2025. The shared task aims to drive technological innovation in libraries, both traditional and modern digital libraries. It provides an opportunity for the research community to creatively utilize language models for subject tagging of the Leibniz University’s Technical Library’s open-access collection. Successful solutions may be directly integrated into the TIB Leibniz Information Centre for Science and Technology University Library’s operational workflows. Participants will be provided with a human-readable form of a subject’s taxonomy, the GND or Gemeinsame Normdatei, the integrated authority file used for cataloging in German-speaking countries, and a large collection of technical records tagged with these subjects from the TIB’s open-access collection, TIBKAT.
We are excited to announce the 2nd Call for Participation in the LLMs4Subjects Shared Task, a task organized by SemEval 2025. This shared task is focused on driving technological innovation in libraries, both traditional and modern digital libraries. It aims to provide an opportunity for the research community to creatively utilize language models for subject tagging of the Leibniz University’s Technical Library’s open-access collection. Successful solutions may be directly integrated into the TIB Leibniz Information Centre for Science and Technology University Library’s operational workflows. Participants will be provided with a human-readable form of a subject’s taxonomy, the GND or Gemeinsame Normdatei, the integrated authority file used for cataloging in German-speaking countries, and a large collection of technical records tagged with these subjects from the TIB’s open-access collection, TIBKAT. The task defines three main tasks, including learning the GND, aligning subject tagging to the TIBKAT collection, and developing elegant frontend interfaces for subject tagging. The shared task will have three separate evaluations, quantitative metrics-based evaluations, qualitative evaluations by human subject specialists, and optional HCI evaluations for subject indexing interfaces. The task is open for participation until January 10, 2025, and the training and validation datasets are available until October 2, 2024. Test data will be available from January 10, 2025, and evaluation starts from January 31, 2025. Participant paper submissions are due on February 28, 2025, and notification to authors will be on March 31, 2025. Camera-ready due is April 21, 2025, and the SemEval workshop is to be determined.
Tags: SemEval 2025, LLMs4Subjects Shared Task, language models, subject tagging, Leibniz Technical Library, TIBKAT, GND, Gemeinsame Normdatei, TIB’s open-access collection, human-readable form, HCI evaluations, frontend interfaces, dataset download