The role of visualization in effective data cleaning

ABSTRACT

Using visualization techniques to assist conventional data mining tasks has attracted considerable interest in recent years. This paper addresses a challenging issue in the use of visualization for data mining: choosing appropriate parameters for spatial data cleaning methods. On one hand, algorithm performance is improved through visualization. On the other hand, characteristics and properties of methods and features of data are visualized as feedbacks to the user. A 3-D visualization model, called Waterfall, is proposed to assist spatial data cleaning in four important aspects: dimension-independent data visualization, visualization of data quality, algorithm parameter selection, and measurement of noise removing methods on parameter sensitiveness.

References

1. Ankerst, M., Breunig, M., Kriegel, H. P., and Sander, J., OPTICS: Ordering Points To Identify the Clustering Structure, in Proc. of 1999 ACM-SIGMOD Conf. on Management of Data (SIGMOD'99), pp. 49--60.]]
2. Batagelj, V., Mrvar, A., and Zaversnik, M., Partitioning approaches to clustering in graphs, Proc. Graph Drawing'1999, LNCS, 2000, pp. 90--97.]]
3. Ertoz, L., Steinbach, M., and Kumar, V., Finding clusters of different sizes, shapes, and densities in noisy, high dimensional data, In Proc. of SIAM DM'03.]]
4. Ester, M., Kriegel, H. P., Sander, J., and Xu, X., A density-based algorithm for discovering clusters in large spatial databases with noise, in Proc. of 2nd Int. Conf. on Knowledge Discovery and Data Mining (KDD-96), AAAI Press, 1996, pp. 226--231.]]
5. Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., and Uthurusamy, R. (eds.), Advances in Knowledge Discovery and Data Mining, AAAI/MIT press, 1996.]]
6. Fayyad, U. and Grinstein, G., Information Visualization in Data Mining and Knowledge Discovery, Morgan Kaufmann, 2001, pp. 182--190.]]
7. Fayyad, U. and Uthurusamy, R., Evolving data mining into solutions for insights, Communications of ACM, 45 (8), 2002, pp. 28--31.]]
8. Han, J., Kamber, M., and Tung, A. K. H., Spatial clustering methods in data mining: A survey, H. Miller and J. Han (eds.), Geographic Data Mining and Knowledge Discovery, Taylor and Francis, 2001.]]
9. Harel, D. and Keren, Y., Clustering spatial data using random walks, In Proc. 7th Int'l Conf. Knowledge Discovery and Data Mining (KDD-2001), ACM Press, New York, pp. 281--286.]]
10. Hinneburg, A., and Keim, D. A., An efficient approach to clustering in large multimedia databases with noise. In Proc. 1998 Int. Conf. Knowledge Discovery and Data Mining, pp. 58--65.]]
11. Karypis, G., Han, E., and Kumar, V., CHAMELEON, A hierarchical clustering algorithm using dynamic modeling, IEEE Computer, Vol. 32, 1999, pp. 68--75.]]
12. Qian, Y., Zhang, G. and Zhang, K., FACADE: A Fast and Effective Approach to the Discovery of Dense Clusters in Noisy Spatial Data, in Proc. ACM SIGMOD 2004 Conference, Paris, France, 13-18 June 2004, ACM Press, pp. 921--922.]]
13. Qian, Y. and Zhang, K., Discovering spatial patterns accurately with effective noise removal, in Proc. 9th ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery (DMKD'04), Paris, France, 13 June 2004, ACM press, pp. 43--50.]]
14. Seidman, S. B., Network structure and minimum degree, Social Networks, 5, 1983, pp. 269--287.]]
15. Shekhar, S., Schrater, P. R., Vatsavai, R. R., Wu, W., and Chawla, S., Spatial contextual classification and prediction models for mining geospatial data, IEEE Trans. on Multimedia, Vol. 4, No. 2, pp. 174--188.]]
16. Ward, M. O. and Zheng, J., Visualization of spatio-temporal data quality, in Proc. of GIS/LIS, 1993, pp. 727--737.]]
17. Zhang, T., Ramakrishnan, R., and Linvy, M., BIRCH: an efficient data clustering method for very large databases, in Proc. ACM SIGMOD 1996 Conference, ACM Press, pp. 103--114.]]