Capturing the Varieties of Natural Language Inference: A Systematic Survey of Existing Datasets and Two Novel Benchmarks
Journal
Journal of Logic, Language and Information
Type
journal article
Date Issued
2023-11-20
Author(s)
Abstract
Transformer-based Pre-Trained Language Models currently dominate the field of Natural Language Inference (NLI). We first survey existing NLI datasets, and we systematize them according to the different kinds of logical inferences that are being distinguished. This shows two gaps in the current dataset landscape, which we propose to address with one dataset that has been developed in argumentative writing research as well as a new one building on syllogistic logic. Throughout, we also explore the promises of ChatGPT. Our results show that our new datasets do pose a challenge to existing methods and models, including ChatGPT, and that tackling this challenge via fine-tuning yields only partly satisfactory results.
Language
English (United States)
Keywords
NLI
Inference
Transformer
MNLI
Survey
ChatGPT
File(s)![Thumbnail Image]()
Name
s10849-023-09410-4.pdf
Size
656.19 KB
Format
Adobe PDF
Checksum (MD5)
049fae1e4755ab0db1f5a35f6b296967