The End is Near: Anticipating the end of a sentence

Abstract

Hahn (2011) demonstrated that anticipating a final word from local context benefitted from the knowledge that the sentence was ending. In this work, I hypothesized that statistical regularities of text usage would show a building anticipation of the sentence’s end as the ending word sequence is acquired. A text corpus analysis was conducted focusing on 570 common and 570 uncommon sentence endings. A single word (w1) followed by three unspecified intervening words generated a weak average anticipation (p(w5=“.” | w1 ? ? ?) = 0.1) that steadily increased with the accumulation of subsequent words in the ending sequence. The complete four-word context generated a strong average anticipation (p(w5=“.” | w1 w2 w3 w4 ) = 0.8) of the subsequent period that renders the period fairly redundant.


Back to Table of Contents