Research

Enabling machines with the ability of reasoning and inference over text is one of the core missions of natural language understanding. Although deep learning models have shown strong performance on various cross-sentence inference benchmarks, recent work has shown that they tend to leverage spurious statistical cues rather than capturing deeper relations between pairs of sentences.

​

We show that state-of-the-art language encoding models are especially bad at modeling directional relations between sentences.

​

To remedy this issue, we incorporate a mutual attention mechanism with a transformer-based model to better capture directional relations between sentences.  We further curate CER, a Cause-and-Effect Relation corpus, to facilitate the model embeds commonsense causal relations in sentence representations. 

​

Experiment results show that the proposed approach improves performance on downstream applications, such as abductive reasoning.