Methods for the assessment of fused images

Abstract

The prevalence of image fusion---the combination of images of different modalities, such as visible and infrared radiation---has increased the demand for accurate methods of image-quality assessment. The current study used a signal-detection paradigm, identifying the presence or absence of a target in briefly presented images followed by an energy mask, which was compared with computational metric and subjective quality assessment results. In Study 1, 18 participants were presented with fused infrared-visible light images, with a soldier either present or not. Two independent variables, image-fusion method (averaging, contrast pyramid, dual-tree complex wavelet transform) and JPEG compression (no compression, low and high compression), were used in a repeated-measures design. Participants were presented with images and asked to state whether or not they detected the target. In addition, subjective ratings and metric results were obtained. This process was repeated in Study 2, using JPEG2000 compression. The results showed a significant effect for fusion but not compression in JPEG2000 images, while JPEG images showed significant effects for both fusion and compression. Subjective ratings differed, especially for JPEG2000 images, while metric results for both JPEG and JPEG2000 showed similar trends. These results indicate that objective and subjective ratings can differ significantly, and subjective ratings should, therefore, be used with care.

References

1. Anderson, N. 1974. Algebraic models in perception. In Handbook of Perception: Psychophysical Judgement and Measurement, E. Carterette and M. Friedman, Eds. Vol. 2. Academic Press, New York, 216--298.]]
2. Breitmeyer, B. G. and Ogmen, H. 2000. Recent models and findings in backward visual masking: A comparison, review, and update. Perception & Psychophysics 62, 8, 1572--1595.]]
3. Canga, E. F., Dixon, T. D., Nikolov, S. G., Canagarajah, C. N., Bull, D. R., Noyes, J. M., and Troscianko, T. 2005. Characterisation of image-fusion quality metrics for surveillance applications over bandlimited channels. Proceedings of the Eighth International Conference on Information Fusion 1, 483--490.]]
4. CCIR Recommendation 500-3. 1986. Method for the subjective assessment of the quality of television pictures. Tech. rep., International Telecommunications Union, Geneva, Switzerland.]]
5. Cedrus. 1999. Superlab pro version 2.0. website http://www.superlab.com/.]]
6. de Ridder, H. and Majoor, G. 1990. Numerical category scaling: An efficient method for assessing digital image coding impairments. In Proceedings of SPIE Conference on Human Vision and Electronic Imaging: Models, Methods and Applications. SPIE, Santa Clara, CA. 65--77.]]
7. Dixon, T., Canga, E., Noyes, J., Troscianko, T., Bull, D., and Canagarajah, C. 2005. Psychophysical and metric assessment of fused-images. Proceedings of the Second Symposium on Applied Perception in Graphics and Visualisation. 43--50.]]
8. Eskicioglu, A. 2000. Quality measurement for monochorome compressed images in the past 25 years. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1907--1910.]]
9. Ghafourian, M. and Huang, C. 1995. Comparison between several adaptive search vector quantization schemes and jpeg standard for image compression. IEEE Transactions on Communications 43, 24 (Feb./Mar./Apr.), 1308--1312.]]
10. Green, D. M. and Swets, J. A. 1966. Signal Detection Theory and Psychophysics. Wiley, New York.]]
11. Guihong, Q., Dali, Z., and Pingfan, Y. 2001. Medical image-fusion by wavelet transform modulus maxima. Optics Express 9, 4 (Aug.), 184--190.]]
12. ITU-R Recommendation BT. 500-10. 2000. Methodology for the subjective assessment of the quality of television pictures. Tech. Rep., International Telecommunications Union, Geneva, Switzerland.]]
13. Kingsbury, N. 1999. Image processing with complex wavelets. In Wavelets: The Key to Intermittent Information, B. Silverman and J. Vassilicos, Eds. Oxford University Press, New York, 165--185.]]
14. Kingsbury, N. G. 1998. The dual-tree complex wavelet transform: A new technique for shift invariance and directional filters. In IEEE Digital Signal Processing Workshop. 86, (Aug.).]]
15. Loza, A., Dixon, T. D., Canga, E. F., Nikolov, S. G., Bull, D. R., Canagarajah, C. N., Noyes, J. M., and Troscianko, T. 2005. Methods of fused-image analysis and assessment. In Proceedings of the Advanced Study Institute Conference (NATO-ASI 2005): Multisensor Data and Information Processing for Rapid and Robust Situation and Threat Assessment. NATO, Albena, Bulgaria.]]
16. Luce, R. 1986. Response Times: Their Role in Inferring Elementary Mental Organization. Oxford Psychology Series, Vol. 8. Oxford University Press, New York.]]
17. Macmillan, N. and Creelman, C. 1991. Detection Theory: A User's Guide. Cambridge University Press, New York.]]
18. Miyahara, M., Kotani, K., and Algazi, V. 1998. Objective picture quality scale (pqs) for image coding. IEEE Transactions on Communications 46, 9 (Sept.), 1215--1226.]]
19. Nikolov, S., Hill, P., Bull, D., and Canagarajah, N. 2001. Wavelets for image-fusion. In Wavelets in Signal and Image Analysis, A. Petrosian and F. Meyer, Eds. Computational Imaging and Vision Series. Kluwer Academic Publishers, Dordrecht, Boston, MA, 213--244.]]
20. Petrovic, V. and Xydeas, C. 2000. On the effects of sensor noise in pixel-level image-fusion performance. Proceedings of the Third International Conference on Information Fusion 2, 14--19.]]
21. Petrovic, V. and Xydeas, C. 2003. Sensor noise effects on signal-level image-fusion performance. Information Fusion 4, 3 (Sept.), 167--183.]]
22. Piella, G. 2003. A general framework for multiresolution image-fusion: From pixels to regions. Information Fusion 4, 4 (Dec.), 259--280.]]
23. Piella, G. and Heijmans, H. A. 2003. A new quality metric for image-fusion. International Conference on Image Processing, ICIP, Barcelona.]]
24. Pohl, C. and Van Genderen, J. 1998. Multisensor image-fusion in remote sensing: Concepts, methods and applications. International Journal of Remote Sensing 19, 5, 823--854.]]
25. Rajae-Joordens, R. and Engel, J. 2005. Paired comparisons in visual perception studies using small sample sizes. Displays 26, 1, 1--7.]]
26. Rigolin, V., Robiolio, P., Spero, L.A., Harrawood, B., Morris, K., Fortin, D., Baker, W., Bashore, T., and Cusma, J. 1996. Compression of digital coronary angiograms does not affect visual or quantitative assessment of coronary artery stenosis severity. American Journal of Cardiology 78, 2 (July), 131--135.]]
27. Samet, A., Ayed, M., Loulou, M., and Masmoudi, N. 2004. Perceptual evaluation of jpeg2000. European Transactions on Telecommunications 15, 135--143.]]
28. Santa-Cruz, D., Grosbois, R., and Ebrahimi, T. 2002. Jpeg 2000 performance evaluation and assessment. Signal Processing: Image Communication 17, 1 (Jan.), 113--130.]]
29. Skodras, A. N., Christopoulos, C. A., and Ebrahimi, T. 2001. Jpeg2000: The upcoming still image compression standard. Pattern Recognition Letters 22, 12 (Oct.), 1337--1345.]]
30. Steingrímsson, U. and Simon, K. 2003. Perceptive quality estimations: Jpeg 2000 versus jpeg. Journal of Imaging Science and Technology 47, 6 (Nov./Dec.), 572--585.]]
31. Stiles, P. and Hofmann, M. 1997. Demonstrated value of data fusion. American Helicopter Society Avionics and Crew Systems Technical Specialist's Meeting, 23--25.]]
32. Sung, M., Kim, H., Kim, E., Kwak, J., Yoo, J., and Yoo, H. 2002. Clinical evaluation of jpeg2000 compression for digital mammography. IEEE Transactions on Nuclear Science 49, 3, 827--832.]]
33. Taubman, D. and Marcellin, M. 2002. JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer International Series in Engineering and Computer Science. Kluwer Academic Publishers, Boston, MA.]]
34. Thurstone, L. 1927. A law of comparative judgement. Psychological Review 34, 273--286.]]
35. Toet, A. 1990. Hierarchical image-fusion. Machine Vision and Applications 3, 1--11.]]
36. Toet, A. 1992. Multiscale contrast enhancement with applications to image-fusion. Optical Engineering 31, 1026--1031.]]
37. Toet, A. and Franken, E. M. 2003. Perceptual evaluation of different image-fusion schemes. Displays 24, 1 (Feb.), 25--37.]]
38. Toet, A., IJspeert, J., Waxman, A., and Aguilar, M. 1997. Fusion of visible and thermal imagery improves situational awareness. Displays 18, 85--95.]]
39. Tomura, N., Watanabe, O., Omachi, K., Sakuma, I., Takahashi, S., Otani, T., Kidani, H., and Watarai, J. 2004. Image fusion of thallium-201 spect and mr imaging for the assessment of recurrent head and neck tumors following flap reconstructive surgery. European Radiology 14, 7 (July), 1249--1254.]]
40. Van Dijk, A. and Martens, J. 1997. Subjective quality assessment of compressed images. Signal Processing 58, 235--252.]]
41. Wald, L. 2002. Data Fusion: Definitions and Architectures---Fusion of images of different spatial resolutions. Les Presses de l'Ecole des Mines, Paris.]]
42. Wallace, G. 1992. The jpeg still picture compression standard. IEEE Transactions on Consumer Electronics 38, 1 (Feb.), 18--34.]]
43. Wang, Z. and Bovik, A. 2002. A universal image quality index. Signal Processing Letters, IEEE 9, 3 (Mar.), 81--84.]]
44. Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (Apr.), 600--612.]]
45. Wickens, T. D. 2002. Elementary Signal Detection Theory. Oxford University Press, New York.]]
46. Wilson, T., Rogers, S., and Kabrisky, M. 1995. Perceptual based hyperspectral image fusion using multi---spectral analysis. Optical Engineering 34, 11, 3154--3164.]]
47. Wixted, J. and Lee, K. 2004. Signal detection theory. http://psy.ucsd.edu/~kang/sdt/sdt.htm.]]
48. www.ImageFusion.org. The online resource for research in image-fusion. http://www.imagefusion.org.]]

Index Terms

• Applied computing

  • Computers in other domains

  • Law, social and behavioral sciences

  • Physical sciences and engineering