Improving Event Detection by Automatically Assessing Validity of Event Occurrence in Text

ABSTRACT

Manually inspecting text to assess whether an event occurs in a document collection is an onerous and time consuming task. Although a manual inspection to discard the false events would increase the precision of automatically detected sets of events, it is not a scalable approach. In this paper, we automatize event validation, defined as the task of determining whether a given event occurs in a given document or corpus. The introduction of automatic event validation as a post-processing step of event detection can boost the precision of the detected event set, discarding false events and preserving the true ones. We propose a novel automatic method for event validation, which relies on a supervised model to predict the occurrence of events in a non-annotated corpus. The data for training the model is gathered by exploiting the crowdsourcing paradigm. Experiments on real-world events and documents show that our proposed method (i) outperforms the state-of-the-art event validation approach and (ii) increases the precision of event detection while preserving recall.

References

1. J. Allan, R. Papka, and V. Lavrenko. On-line new event detection and tracking. In SIGIR, 1998.
2. J. Araki and J. Callan. An annotation similarity model in passage ranking for historical fact validation. In SIGIR, 2014.
3. A. Ceroni and M. Fisichella. Towards an entity-based automatic event validation. In ECIR, 2014.
4. A. Das Sarma, A. Jain, and C. Yu. Dynamic relationship and event discovery. In Proc. of WSDM '11, 2011.
5. G. P. C. Fung, J. X. Yu, P. S. Yu, and H. Lu. Parameter free bursty events detection in text streams. In Proc. of VLDB '05, 2005.
6. Q. He, K. Chang, and E.-P. Lim. Analyzing feature trajectories for event detection. In Proc. of SIGIR '07, 2007.
7. J. Hoffart, F. Suchanek, K. Berberich, and G. Weikum. Yago2: A spatially and temporally enhanced knowledge base from Wikipedia. Artificial Intelligence, 2012.
8. D. Hovy, J. Fan, A. Gliozzo, S. Patwardhan, and C. Welty. When did that happen? linking events and relations to timestamps. EACL'12.
9. J. R. Landis and G. G. Koch. The measurement of observer agreement for categorical data. Biometrics, 1977.
10. A. J. McMinn, Y. Moshfeghi, and J. M. Jose. Building a large-scale corpus for evaluating event detection on twitter. In CIKM, 2013.
11. T. Tran, A. Ceroni, M. Georgescu, K. Djafari Naini, and M. Fisichella. Wikipevent: Leveraging wikipedia edit history for event detection. In WISE. 2014.

Index Terms

• Information systems

  • Information retrieval