The role of visualization in effective data cleaning

ABSTRACT

Using visualization techniques to assist conventional data mining tasks has attracted considerable interest in recent years. This paper addresses a challenging issue in the use of visualization for data mining: choosing appropriate parameters for spatial data cleaning methods. On one hand, algorithm performance is improved through visualization. On the other hand, characteristics and properties of methods and features of data are visualized as feedbacks to the user. A 3-D visualization model, called Waterfall, is proposed to assist spatial data cleaning in four important aspects: dimension-independent data visualization, visualization of data quality, algorithm parameter selection, and measurement of noise removing methods on parameter sensitiveness.

REFERENCES

• 1: Mihael Ankerst , Markus M. Breunig , Hans-Peter Kriegel , Jörg Sander, OPTICS: ordering points to identify the clustering structure, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.49-60, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 2: Vladimir Batagelj , Andrej Mrvar , Matjaz Zaversnik, Partitioning Approach to Visualization of Large Graphs, Proceedings of the 7th International Symposium on Graph Drawing, p.90-97, September 01, 1999
• 3: Ertoz, L., Steinbach, M., and Kumar, V., Finding clusters of different sizes, shapes, and densities in noisy, high dimensional data, In Proc. of SIAM DM'03.
• 4: Ester, M., Kriegel, H. P., Sander, J., and Xu, X., A density-based algorithm for discovering clusters in large spatial databases with noise, in Proc. of 2nd Int. Conf. on Knowledge Discovery and Data Mining (KDD-96), AAAI Press, 1996, pp. 226--231.
• 5: Usama M. Fayyad , Gregory Piatetsky-Shapiro , Padhraic Smyth , Ramasamy Uthurusamy, Advances in knowledge discovery and data mining, American Association for Artificial Intelligence, Menlo Park, CA, 1996
• 6: Usama Fayyad , Georges G. Grinstein , Andreas Wierse, Information Visualization in Data Mining and Knowledge Discovery, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2001
• 7: Usama Fayyad , Ramasamy Uthurusamy, Evolving data into mining solutions for insights, Communications of the ACM, v.45 n.8, August 2002 [doi>10.1145/545151.545174]
• 8: Han, J., Kamber, M., and Tung, A. K. H., Spatial clustering methods in data mining: A survey, H. Miller and J. Han (eds.), Geographic Data Mining and Knowledge Discovery, Taylor and Francis, 2001.
• 9: David Harel , Yehuda Koren, Clustering spatial data using random walks, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p.281-286, August 26-29, 2001, San Francisco, California [doi>10.1145/502512.502552]
• 10: Hinneburg, A., and Keim, D. A., An efficient approach to clustering in large multimedia databases with noise. In Proc. 1998 Int. Conf. Knowledge Discovery and Data Mining, pp. 58--65.
• 11: George Karypis , Eui-Hong (Sam) Han , Vipin Kumar, Chameleon: Hierarchical Clustering Using Dynamic Modeling, Computer, v.32 n.8, p.68-75, August 1999 [doi>10.1109/2.781637 ]
• 12: Yu Qian , Gang Zhang , Kang Zhang, FAÇADE: a fast and effective approach to the discovery of dense clusters in noisy spatial data, Proceedings of the 2004 ACM SIGMOD international conference on Management of data, June 13-18, 2004, Paris, France [doi>10.1145/1007568.1007696]
• 13: Yu Qian , Kang Zhang, Discovering spatial patterns accurately with effective noise removal, Proceedings of the 9th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery, June 13, 2004, Paris, France [doi>10.1145/1008694.1008701]
• 14: Seidman, S. B., Network structure and minimum degree, Social Networks, 5, 1983, pp. 269--287.
• 15: Shekhar, S., Schrater, P. R., Vatsavai, R. R., Wu, W., and Chawla, S., Spatial contextual classification and prediction models for mining geospatial data, IEEE Trans. on Multimedia, Vol. 4, No. 2, pp. 174--188.
• 16: Ward, M. O. and Zheng, J., Visualization of spatio-temporal data quality, in Proc. of GIS/LIS, 1993, pp. 727--737.
• 17: Tian Zhang , Raghu Ramakrishnan , Miron Livny, BIRCH: an efficient data clustering method for very large databases, Proceedings of the 1996 ACM SIGMOD international conference on Management of data, p.103-114, June 04-06, 1996, Montreal, Quebec, Canada

INDEX TERMS

Keywords:
clustering, data mining, information visualization, noise removal