QuAIL - Question Answering for Artificial Intelligence

What is QuAIL?

A new kind of question-answering dataset that combines commonsense, text-based, and unanswerable questions, balanced for different genres and reasoning types. Reasoning type annotation for 9 types of reasoning: temporal, causality, factoid, coreference, character properties, their belief states, subsequent entity states, event durations, and unanswerable. Genres: CC license fiction, Voice of America news, blogs, user stories from Quora 800 texts, 18 questions for each (~14K questions).

Getting started

Questions?

Send an email to Matt Downey (matt.downey18@gmail.com) or Anna Rogers (anna.gld@gmail.com).

Leaderboard (Last updated: 13 Feb 20 15:16 EST)

Rank All Temp. Caus. Fact. Char. Ent. Belief Sub. Dur. Unans. Team Submitted Model
1 29.7 30.4 30.8 30.8 32.1 29.2 26.7 24.2 17.9 45.4 matt.downey18 11 Feb 20 14:47 EST PMI baseline UML Text Machine Lab

Notes

After submission, please allow a few days for your results to show up in the leaderboard.