McClure, David, Shayne O’Brien, and Deb Roy. “Context Is Key: New Approaches to Neural Coherence Modeling.” ArXiv:1812.04722 [Cs, Stat], December 5, 2018. http://arxiv.org/abs/1812.04722.

Abstract

We formulate coherence modeling as a regression task and propose two novel methods to combine techniques from our setup with pairwise approaches. The first of our methods is a model that we call "first-next," which operates similarly to selection sorting but conditions decision-making on information about already-sorted sentences. The second consists of a technique for adding context to regression-based models by concatenating sentence-level representations with an encoding of its corresponding out-of-order paragraph. This latter model achieves Kendall-tau distance and positional accuracy scores that match or exceed the current state-of-the-art on these metrics. Our results suggest that many of the gains that come from more complex, machine-translation inspired approaches can be achieved with simpler, more efficient models.