Exploring Distributional Semantics In Lexical Representations And Narrative Modeling

We are interested in the computational modeling of lexico-conceptual and narrative knowledge (e.g. how to represent the meaning of cat to reflect facts such as: it is similar to a dog, and it is typically larger than a mouse; how to characterize story, and how to identify different narratives on the same topic). On the lexico-conceptual front, we learn lexical representations with strong interpretability, as well as integrate commonsense knowledge into lexical representations. For narrative modeling, we study how to identify, extract, and generate narratives/stories acceptable to human intuition. As a methodological framework we apply the methods of Distributional Semantics (DS) — “a subfield of Natural Language Processing that learns meaning from word usages” (Herbelot, 2019) — where semantic representations (on any levels such as word, phrases, sentences, etc.) are learned at scale from data through machine learning models (Erk and Padó, 2008; Baroni and Lenci, 2010; Mikolov et al., 2013; Pennington et al., 2014). To infuse interpretability and commonsense into semantic representations (specifically lexical and event), which are typically lacking in previous works (Doran et al., 2017; Gusmao et al., 2018; Carvalho et al., 2019), we complement the data-driven scalability with a minimal amount of human knowledge annotation on a selected set of tasks and have obtained empirical evidence in support of our techniques. For narrative modeling, we draw insights from the rich body of work on scripts and narratives started from Schank and Abelson (1977) and Mooney and DeJong (1985) to Chambers and Jurafsky (2008, 2009), and proposed distributional models for the tasks narrative identification, extraction, and generation which produced state-of-the-art performance. Symbolic approaches to lexical semantics (Wierzbicka, 1996; Goddard and Wierzbicka, 2002) and narrative modeling (Schank and Abelson, 1977; Mooney and DeJong, 1985) have been fruitful on the front of theoretical studies. For example, in theoretical linguistics, Wierzbicka defined a small set of lexical semantic primitives from which complex meaning can be built compositionally; in Artificial Intelligence, Schank and Abelson formulated primitive acts which are conceptualized into semantic episodes (i.e. scripts) understandable by humans. Our focus, however, is primarily on computational approaches that need wide lexical coverage, for which DS provides a better toolkit, especially in practical applications. In this thesis, we innovate by building on the “vanilla” DS techniques (Landauer and Dumais, 1997; Mikolov et al., 2013) to address the issues listed above. Specifically, we present empirical evidence that • On the building block level, with the framework of DS, it is possible to learn highly interpretable lexical and event representations at scale and introduce human commonsense knowledge at low cost. • On the narrative level, well-designed DS modeling offers a balance of precision and scalability, solutions which are empirically stronger to complex narrative modeling questions (e.g. narrative identification, extraction and generation). Further, conducting case-studies on lexical and narrative modeling, we showcase the viability of integrating DS with traditional methods in complementation to retain the strengths of both approaches Concretely, the contributions of this thesis are summarized as follows: • Evidence from analyzing/modeling a small set of common concepts which indicates that interpretable representations can be learned for lexical concepts with minimal human annotation to realize one/few-shot learning. • Commonsense integration in lexical semantics: with carefully designed crowdsourcing, and combined with distributional methods, it is possible to substantially improve inference related to physical knowledge of the world. • Neural distributional methods perform strongly in complex narrative modeling tasks, where we demonstrate that the following techniques are particularly useful: 1) human intuition inspired iterative algorithms; 2) integration of graphical and distributional modeling; pre-trained large-scale language models