QuAIL - Question Answering for Artificial Intelligence

What is QuAIL?

A new kind of question-answering dataset that combines commonsense, text-based, and unanswerable questions, balanced for different genres and reasoning types. Reasoning type annotation for 9 types of reasoning: temporal, causality, factoid, coreference, character properties, their belief states, subsequent entity states, event durations, and unanswerable. Genres: CC license fiction, Voice of America news, blogs, user stories from Quora 800 texts, 18 questions for each (~14K questions).

Getting started

Questions?

Contact Matt Downey regarding the leaderboard, and Anna Rogers regarding QuAIL data.

Leaderboard (Last updated: 05 Feb 20 15:07 EST)

Rank All Temp. Caus. Fact. Char. Ent. Belief Sub. Dur. Unans. Team Submitted Model
1 25.3 25.1 19.2 27.4 16.6 23.2 20.9 24.4 21.1 28.8 Matt Downey 05 Feb 20 14:51 EST Submission description

Notes

After submission, please allow a few hours for your results to show up in the leaderboard.