Multi-resolution modeling of large scale scientific simulation data

ABSTRACT

To provide scientists and engineers with the ability to explore and analyze tera-scale size data-sets we are using a twofold approach. First, we model the data with the objective of creating a compressed yet manageable representation. Second, with that compressed representation, we provide the ability to query the resulting approximation in order to obtain approximate yet sufficient answers; a process called ad-hoc querying. This paper is concerned with a wavelet modeling technique that seeks to capture the important physical characteristics of the target scientific data. Our approach is driven by the compression, which is necessary for viable throughput, along with the end user requirements from the discovery process. Our work contrasts existing research which applies wavelets to range querying, change detection, and clustering problems by working directly with the wavelet decomposition of the data. The difference in this procedure is due primarily to the nature of the data and the requirements of the scientists and engineers. Our approach directly uses the wavelet coefficients of the data to compress as well as query. We describe how the wavelet decomposition is used to facilitate data compression and how queries are posed on the resulting compressed model. Results of this process will be shown for several problems of interest.

REFERENCES

• 1: Ghaleb Abdulla , Chuck Baldwin , Terence Critchlow , Roy Kamimura , Ida Lozares , Ron Musick , Nu Ai Tang , Byung S. Lee , Robert Snapp, Approximate ad-hoc query engine for simulation data, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.255-256, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379673]
• 2: Kaushik Chakrabarti , Minos Garofalakis , Rajeev Rastogi , Kyuseok Shim, Approximate query processing using wavelets, The VLDB Journal — The International Journal on Very Large Data Bases, v.10 n.2-3, p.199-223, September 2001
• 3: Ingrid Daubechies, Ten lectures on wavelets, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1992
• 4: R. DeVore, B. Jawerth, and B. Lucier. Image compression through wavelet transform coding. IEEE Trans. Image Processing, 38:719--746, 1992.
• 5: R. DeVore, B. Jawerth, and V. Popov. Compression of wavelet decompositions. American Journal of Mathematics, 114:737-- 785, 1992.
• 6: U. M. Fayyad, D. Haussler, and P. E. Stolorz. KDD for science data analysis: Issues and examples. In Knowledge Discovery and Data Mining, pages 50--56, 1996.
• 7: Michael L. Hilton , Björn D. Jawerth , Ayan Sengupta, Compressing still and moving images with wavelets, Multimedia Systems, v.2 n.5, p.218-227, Dec. 1994 [doi>10.1007/BF01215399]
• 8: Björn Jawerth , Wim Sweldens, An overview of wavelet based multiresolution analyses, SIAM Review, v.36 n.3, p.377-412, Sept. 1994 [doi>10.1137/1036095]
• 9: Jpeg home page. http://www.jpeg.org/JPEG2000.htm,1999.
• 10: Eamonn Keogh , Kaushik Chakrabarti , Michael Pazzani , Sharad Mehrotra, Locally adaptive dimensionality reduction for indexing large time series databases, Proceedings of the 2001 ACM SIGMOD international conference on Management of data, p.151-162, May 21-24, 2001, Santa Barbara, California, United States
• 11: S. Mallat. A Wavelet Tour of Signal Processing. Academic Press,1998.
• 12: F. Murtagh, J.-L. Starck, and M. W. Berry. Overcoming the curse of dimensionality in clustering by means of the wavelet transform. Computer Journal, 43(2):107--120,2000.
• 13: R. Todd Ogden, Essential wavelets for statistical applications and data analysis, Birkhauser Boston Inc., Cambridge, MA, 1996
• 14: N. Ramakrishnan and A. Gram. Mining scientific data. In Advances in Computers, pages 119--169. Academic Press,2001.
• 15: R. R. Schmidt and C. Shahabi. Polap: A fast wavelet-based technique for progressive evaluation of olap queries. Technical Report 01-744, Department of Computer Science, University of Southern California, 2001.
• 16: C. Shahabi, S. Chung, and M. Safar. A wavelet-based approach to improve the efficiency of multi-level surprise mining. In PAKDD International Workshop on Mining Spatial and Temporal Data 2001 ,2001.
• 17: Gholamhosein Sheikholeslami , Surojit Chatterjee , Aidong Zhang, WaveCluster: A Multi-Resolution Clustering Approach for Very Large Spatial Databases, Proceedings of the 24rd International Conference on Very Large Data Bases, p.428-439, August 24-27, 1998
• 18: J.-L. Starck , F. Murtagh , A. Bijaoui, Image processing and data analysis: the multiscale approach, Cambridge University Press, New York, NY, 1998
• 19: G. Strang and T. Nguyen. Wavelets and Filter Banks. Wellesley Cambridge, 1996.
• 20: David S. Taubman , Michael W. Marcellin, JPEG 2000: Image Compression Fundamentals, Standards and Practice, Kluwer Academic Publishers, Norwell, MA, 2001
• 21: A. Uhl. Wavelets and digital image compression I. Technical Report RIST++04/93, University of Salzburg,1993.
• 22: B. Vidakovic. Statistical Modeling by Wavelets. Wiley, 1999.
• 23: D. Yu, S. Chatterjee, G. Sheikholeslami, and Z. Aidong. Efficiently detecting arbitrary shaped clusters in very large datasets with high dimensions. Technical Report 98-08, Department of Computer Science and Engineering, SUNY Buffalo, 1998.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Scientific databases

Additional Classification:
  E. Data
  E.4 CODING AND INFORMATION THEORY
      Subjects: Data compaction and compression

  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.2 Approximation
          Subjects: Wavelets and fractals

General Terms:
Algorithms, Management, Performance

Keywords:
compression, data modeling, scientific data processing, wavelets