GenBench: The Second Workshop on Generalisation (Benchmarking) in NLP @ EMNLP 2024

The GenBench workshop, co-located with EMNLP 2024, aims to serve as a platform for discussing challenging questions related to generalisation in NLP. It brings together different expert communities to crowd-source challenging generalisation benchmarks for large language models (LLMs) and make progress on open questions related to generalisation. The workshop calls for two types of submissions: regular workshop submissions and collaborative benchmarking task (CBT) submissions. The CBT focuses on generating versions of existing evaluation datasets for LLMs that have a larger distribution shift than the original test set, evaluating generalisation to a stronger degree. Submissions for CBT consist of a data/task artefact and a companion paper. The workshop accepts archival papers and extended abstracts for both categories. The submission deadline is August 15, 2024, and the notification deadline is September 20, 2024. For more information, visit https://genbench.org/workshop.