Jiaoyan Chen
Ningyu Zhang
ABSTRACT
Recently, the appearing disaster of severe smog has been attacking many cities in China such as the capital Beijing. The chief culprit of China smog, namely PM2.5, is affected by various factors including air pollutants, weather, climate, geographical location, urbanization, etc. To analyze the factors, we collect about 35,000,000 air quality records and about 30,000,000 weather records from the sensors in 77 China's cities in 2013. Moreover, two big data sets named Geoname and DBPedia are also combined for the data of climate, geographical location and urbanization. To deal with big spatio-temporal data for big smog analysis, we propose a MapReduce-based framework named BigSmog. It mainly conducts parallel correlation analysis of the factors and scalable training of artificial neural networks for spatio-temporal approximation of the concentration of PM2.5. In the experiments, BigSmog displays high scalability for big smog analysis with big spatio-temporal data. The analysis result shows that the air pollutants influence the short-term concentration of PM2.5 more than the weather and the factors of geographical location and climate rather than urbanization play a major role in determining a city's long-term pollution level of PM2.5. Moreover, the trained ANNs can accurately approximate the concentration of PM2.5.
- D. Borthakur. Hdfs architecture guide. Hadoop Apache Project. http://hadoop.apache.org/common/docs/current/hdfs_design.pdf, 2008.Google Scholar
- H. Che, X. Zhang, Y. Li, Z. Zhou, J. Qu, and X. Hao. Haze trends over the capital cities of 31 provinces in china, 1981âĂŞ2005. Theoretical and Applied Climatology, 97(3-4):235--242, 2009.Google ScholarCross Ref
- D. A. Chu, Y. Kaufman, G. Zibordi, J. Chern, J. Mao, C. Li, and B. Holben. Global monitoring of air pollution over land from the earth observing system-terra moderate resolution imaging spectroradiometer (modis). Journal of Geophysical Research: Atmospheres (1984--2012), 108(D21), 2003.Google Scholar
- J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. Communications of the ACM, 51(1):107--113, 2008. Google ScholarDigital Library
- D. Dominick, H. Juahir, M. T. Latif, S. M. Zain, and A. Z. Aris. Spatial assessment of air quality patterns in malaysia using multivariate analysis. Atmospheric Environment, 60(0):172--181, 2012.Google ScholarCross Ref
- G. Grivas and A. Chaloulakou. Artificial neural network models for prediction of {PM10} hourly concentrations, in the greater area of athens, greece. Atmospheric Environment, 40(7):1216--1229, 2006.Google ScholarCross Ref
- D. Hasenfratz, O. Saukh, S. Sturzenegger, and L. Thiele. Participatory air pollution monitoring using smartphones. In Proc. 1st IntâĂŹl Workshop on Mobile Sensing: From Smartphones and Wearables to Big Data, 2012.Google Scholar
- J. Hooyberghs, C. Mensink, G. Dumont, F. Fierens, and O. Brasseur. A neural network forecast for daily average {PM10} concentrations in belgium. Atmospheric Environment, 39(18):3279--3289, 2005.Google ScholarCross Ref
- G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew. Extreme learning machine: a new learning scheme of feedforward neural networks. In International Symposium on Neural Networks, volume 2, 2004.Google Scholar
- G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew. Extreme learning machine: Theory and applications. Neurocomputing, 70(1âĂŞ3):489--501, 2006.Google Scholar
- H. Jing, X. Lihua, G. Yuling, and W. Xiaoxuan. Remote sensing monitoring on photochemical pollution caused by haze in pearl river delta. In Urban Remote Sensing Joint Event, 2009.Google Scholar
- O. Kunii, S. Kanagawa, I. Yajima, Y. Hisamatsu, S. Yamamura, T. Amagai, and I. T. S. Ismail. The 1997 haze disaster in indonesia: Its air quality and health effects. Archives of Environmental Health: An International Journal, 57(1):16--22, 2002. PMID: 12071356.Google Scholar
- J. Lee Rodgers and W. A. Nicewander. Thirteen ways to look at the correlation coefficient. The American Statistician, 42(1):59--66, 1988.Google ScholarCross Ref
- S. Li, L. Chen, F. Zheng, D. Han, and Z. Wang. Design and application of haze optic thickness retrieval model for beijing olympic games. In Geoscience and Remote Sensing Symposium, 2009 IEEE International, IGARSS 2009, volume 2, pages II--507--II--510, 2009.Google Scholar
- I. G. McKendry. Evaluation of artificial neural networks for fine particulate pollution (pm10 and pm2.5) forecasting. Journal of the Air & Waste Management Association, 52(9):1096--1101, 2002.Google ScholarCross Ref
- M. Patience. Beijing smog: When growth trumps life in china. BBC News Magazine, 2013.Google Scholar
- Y. Sun, G. Zhuang, A. Tang, Y. Wang, and Z. An. Chemical characteristics of pm2. 5 and pm10 in haze-fog episodes in beijing. Environmental science & technology, 40(10):3148--3155, 2006.Google Scholar
- S. N. Syed Abdul Mutalib, H. Juahir, A. Azid, S. Mohammad Sharif, M. T. Latif, A. Z. Aris, S. Md Zain, and D. Dominick. Spatial and temporal air quality pattern recognition using environmetric techniques: a case study in malaysia. Environ. Sci.: Processes Impacts, pages --, 2013.Google Scholar
- J.-H. Tan, J.-C. Duan, D.-H. Chen, X.-H. Wang, S.-J. Guo, X.-H. Bi, G.-Y. Sheng, K.-B. He, and J.-M. Fu. Chemical characteristics of haze during summer and winter in guangzhou. Atmospheric Research, 94(2):238--245, 2009.Google ScholarCross Ref
- A. Van Donkelaar, R. V. Martin, and R. J. Park. Estimating ground-level pm2. 5 using aerosol optical depth determined from satellite remote sensing. Journal of Geophysical Research: Atmospheres (1984--2012), 111(D21), 2006.Google Scholar
- S. Vardoulakis, B. E. Fisher, K. Pericleous, and N. Gonzalez-Flesca. Modelling air quality in street canyons: a review. Atmospheric Environment, 37(2):155--182, 2003.Google ScholarCross Ref
- C. Xu, Z. Wang, S. Li, and H. Chen. A haze monitoring over north china plain. In Geoscience and Remote Sensing Symposium (IGARSS), 2012 IEEE International, pages 2474--2477, 2012.Google ScholarCross Ref
- Y. Zheng, F. Liu, and H.-P. Hsie. U-air: When urban air quality inference meets big data. 2013.Google Scholar
Index Terms
- When big data meets big smog: a big spatio-temporal data framework for China severe smog analysis
Recommendations
Big Data Management: Advanced Issues and Approaches
The objective of this article is to provide the advanced issues and approaches of big data management. The literature review indicates the overview of big data management; the aspects of Big Data Analytics BDA; the importance of big data management; the ...
Where big data meets 5G?
ICC '17: Proceedings of the Second International Conference on Internet of things, Data and Cloud ComputingDue to massive increase in data collection from wireless devices, wireless sensor networks, and network operators, data processing has become a challenge. The massive data can broadly be categorized into raw data and right data. Future generation ...
Comments