JORD: A System for Collecting Information and Monitoring Natural Disasters by Linking Social Media with Satellite Imagery

ABSTRACT

Gathering information, and continuously monitoring the affected areas after a natural disaster can be crucial to assess the damage, and speed up the recovery process. Satellite imagery is being considered as one of the most productive sources to monitor the after effects of a natural disaster; however, it also comes with a lot of challenges and limitations, due to slow update. It would be beneficiary to link remote sensed data with social media for the damage assessment, and obtaining detailed information about a disaster. The additional information, which are obtainable by social media, can enrich remote-sensed data, and overcome its limitations.

To tackle this, we present a system called JORD that is able to autonomously collect social media data about natural disasters, and link it automatically to remote-sensed data. In addition, we demonstrate that queries in local languages that are relevant to the exact position of natural disasters retrieve more accurate information about a disaster event. We also provide content based analysis along with temporal and geo-location based filtering to provide more accurate information to the users. To show the capabilities of the system, we demonstrate that a large number of disaster events can be detected by the system. In addition, we use crowdsourcing to demonstrate the quality of the provided information about the disasters, and usefulness of JORD from potential users point of view

References

1. Martín Abadi, Ashish Agarwal, Paul Barham, and others. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016).
2. Kashif Ahmad, Nicola Conci, Giulia Boato, and Francesco GB De Natale. 2016. USED: a large-scale social event detection dataset. In Proceedings of the 7th International Conference on Multimedia Systems. ACM, 50.
3. Farzindar Atefeh and Wael Khreich. 2015. A survey of techniques for event detection in twitter. Computational Intelligence 31, 1 (2015), 132--164.
4. Mohamed Bakillah, Ren-Yu Li, and Steve HL Liang. 2015. Geo-located community detection in Twitter with enhanced fast-greedy optimization of modularity: the case study of typhoon Haiyan. IJGIS 29, 2 (2015), 258--279.
5. François Chollet and others. 2015. Keras: Deep learning library for theano and tensorflow. URL: https://keras.io/k (2015).
6. Xiaowen Dong, Dimitrios Mavroeidis, Francesco Calabrese, and Pascal Frossard. 2015. Multiscale event detection in social media. IJDMKD 29, 5 (2015), 1374--1405.
7. Berna Erol, Jonathan J Hull, and Dar-Shyang Lee. 2003. Linking multimedia presentations with their symbolic source documents: algorithm and applications. In Proc. of ACM MM. 498--507.
8. Debarati Guha-Sapir, Regina Below, and Philippe Hoyois. 2015. EM-DAT: International disaster database. Catholic University of Louvain: Brussels, Belgium (2015).
9. Adrien Guille and Cécile Favre. 2015. Event detection, tracking, and visualization in twitter: a mention-anomaly-based approach. Social Network Analysis and Mining 5, 1 (2015), 1--18.
10. Mark Hall, Eibe Frank, Geoffrey Holmes, and others. 2009. The WEKA data mining software: an update. ACM SIGKDD explorations newsletter 11, 1 (2009), 10--18.
11. Chunyang He, Qiaofeng Zhang, Yuechen Li, Xiaobing Li, and Peijun Shi. 2005. Zoning grassland protection area using remote sensing and cellular automata modeling---a case study in Xilingol steppe grassland in northern China. IJAE 63, 4 (2005), 814--826.
12. Karen E Joyce, Stella E Belliss, Sergey V Samsonov, Stephen J McNeill, and Phil J Glassey. 2009. A review of the status of satellite remote sensing and image processing techniques for mapping natural hazards and disasters. Progress in Physical Geography (2009).
13. Chenliang Li, Aixin Sun, and Anwitaman Datta. 2012. Twevent: segment-based event detection from tweets. In Proc. of ACM IKM. ACM, 155--164.
14. Mathias Lux, Michael Riegler, Pål Halvorsen, Konstantin Pogorelov, and Nektarios Anagnostopoulos. 2016. LIRE: open source visual information retrieval. In Proc. of ACM MMSys.
15. TN Manjunath, Ravindra S Hegadi, and GK Ravikumar. 2010. A survey on multimedia data mining and its relevance today. IJCSNS 10, 11 (2010), 165--170.
16. Michael Mathioudakis and Nick Koudas. 2010. Twittermonitor: trend detection over the twitter stream. In Proc. of ACM SIGMOD. 1155--1158.
17. Polykarpos Meladianos, Giannis Nikolentzos, François Rousseau, Yannis Stavrakas, and Michalis Vazirgiannis. 2015. Degeneracy-based real-time sub-event detection in twitter stream. In Proc. of AAAI ICWSM. 248--257.
18. Frank Paul and Liss M Andreassen. 2009. A new glacier inventory for the Svartisen region, Norway, from Landsat ETM+ data: challenges and change assessment. Journal of Glaciology 55, 192 (2009), 607--618.
19. M. Riegler, V. R. Gaddam, M. Larson, R. Eg, P. Halvorsen, and C. Griwodz. 2016. Crowdsourcing as self-fulfilling prophecy: Influence of discarding workers in subjective assessment tasks. In Proc. of CBMI. 1--6. DOI:http://dx.doi.org/10.1109/CBMI.2016.7500256
20. Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. 2010. Earthquake shakes Twitter users: real-time event detection by social sensors. In Proc. of ACM WWW. 851--860.
21. Brian Stelter and Noam Cohen. 2008. Citizen journalists provided glimpses of Mumbai attacks. The New York Times 30 (2008).
22. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, and others. 2015. Rethinking the inception architecture for computer vision. arXiv preprint arXiv:1512.00567 (2015).
23. C. Tzelepis, Z. Ma, V. Mezaris, B. Ionescu, I. Kompatsiaris, G. Boato, N. Sebe, and S. Yan. 2016. Event-based Media Processing and Analysis: A Survey of the Literature. Image and Vision Computing (2016).
24. H Wood. 2002. The Use of Earth Observing Satellites for Hazard Support: Assessments and Scenarios. Final Report of the CEOS Disaster Management Support Group. Available from< http://disaster.ceos.org/legal.cfm (2002).

Index Terms

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

      • Information extraction