SemEval-2026 Task 5: Rating Plausibility of Word Senses

Participate in SemEval-2026 Task 5 to develop systems that rate word sense plausibility in ambiguous sentences using narrative understanding.

SemEval-2026 Task 5 invites participants to develop systems that rate the plausibility of word senses in ambiguous sentences through narrative understanding.

The task utilizes the AmbiStory dataset, comprising five-sentence English short stories with human plausibility judgements.

• Given a short story and a pair of candidate word senses, systems must predict plausibility on a scale from 1 (implausible) to 5 (highly plausible).
• Evaluation metrics include accuracy within standard deviation and ranked correlation between model predictions and human judgments.

For more information and to participate, visit the task website and code repository. Register and submit your results on Codabench.

Tags: SemEval-2026, Word Sense Disambiguation, NLP, Narrative Understanding, AI Research, Computational Linguistics, Natural Language Processing