Learning Document-Level Semantic Properties from Free-Text Annotations

 S.R.K. Branavan, Harr Chen, Jacob Eisenstein, Regina Barzilay

Abstract

JAIR volume 34, pages 569-603

This paper presents a new method for inferring the semantic properties of documents by leveraging free-text keyphrase annotations. Such annotations are becoming increasingly abundant due to the recent dramatic growth in semi-structured, user-generated online content. One especially relevant domain is product reviews, which are often annotated by their authors with pros/cons keyphrases such as ``a real bargain'' or ``good value.'' These annotations are representative of the underlying semantic properties; however, unlike expert annotations, they are noisy: lay authors may use different labels to denote the same property, and some labels may be missing. To learn using such noisy annotations, we find a hidden paraphrase structure which clusters the keyphrases. The paraphrase structure is linked with a latent topic model of the review texts, enabling the system to predict the properties of unannotated documents and to effectively aggregate the semantic properties of multiple reviews. Our approach is implemented as a hierarchical Bayesian model with joint inference. We find that joint inference increases the robustness of the keyphrase clustering and encourages the latent topics to correlate with semantically meaningful properties. Multiple evaluations demonstrate that our model substantially outperforms alternative approaches for summarizing single and multiple documents into a set of semantically salient keyphrases.

Code

The source code for this work can be downloaded from the link below. For clarity and ease of maintenance, auxiliary functionality has been split out into a set of separate dynamic link libraries.

        [ Source code for model & libraries ]
        [ A complete ready-to-run archive containing the compiled model & libraries, and a dataset in the required format ]

This code makes extensive use of the GNU scientific library (GSL). You can download binary & source distributions of this library from the GSL home page.

All of the code is in C++, and our compilation environment was gcc 4.1.2 on linux. However, the code should compile correctly with versions of GCC higher than 3.4 and run on any environment which supports dynamic link libraries and pthread based multithreading. If required, the requirement for DLL and multithreading support can be removed through straighforward changes to the code.

The source archives include the necessary makefiles in the GNU make format.

Data

The datasets used in this work are available in XML format from the links below:

       [ Complete archive ]

You can also download the individual datasets:

       [ Cameras dataset ]
       [ Cellphones dataset ]
       [ Restaurants dataset ]
       [ Dataset XML format ]

Annotations

The manual annotations used in this work are available from the link below. Please check the README file in the archive for the format of the annotations.

       [ Annotations archive ]