skip to main content
research-article

Enabling multimedia using resource-constrained video processing techniques: A node-centric perspective

Published: 06 February 2008 Publication History

Abstract

Successful proliferation of multimedia-enabled devices and advances in very large-scale integration (VLSI) technology has spawned new research efforts in migrating video processing applications onto ever smaller and more inexpensive devices. This article focuses on the technical challenges associated with that migration.
Due to limitations in size, battery lifetime, and, ultimately, cost, mapping complex video applications onto resource-constrained systems is a very challenging proposition. To this end, we first consider a technique, region-of-interest (ROI) processing, of defining a window within a video frame and only operating on the data inside that window, ignoring the rest of the frame. By using this lossy technique, the processing requirements can be reduced by roughly 80% while the error introduced in the quality of the results is roughly 10%. The other technique is adaptive data partitioning (ADP) combined with a content-based power management algorithm. By distributing video processing among multiple processors and shutting them down when they are not needed, the energy consumed per processor can be reduced by 60% without sacrificing the performance of the underlying video-based application.
Taken together, these novel techniques enable ambient multimedia systems and maintain the needed overall efficiency in video processing.

References

[1]
Aarts, E. 2004. Ambient intelligence: A multimedia perspective. IEEE Multimedia Mag. 11, 1(Jan.), 12--19.
[2]
Altilar, D. and Paker, Y. 2001. Minimum overhead data partitioning algorithms for parallel video processing. In Proceedings of the International Conference on Domain Decomposition Methods.
[3]
Atmel. 2007. AT91 arm-based microcontrollers. http://www.atmel.com/products/AT91/.
[4]
Bradley, A. and Stentiford, F. 2002. JPEG 2000 and region of interest coding. Digit. Image Comput. Tech. Appl.
[5]
Cavallaro, A., Steiger, O., and Ebrahimi, T. 2002. Multiple video object tracking in complex scenes. In Proceedings of the ACM International Conference on Multimedia.
[6]
Chen, K. and Hu, C. 1998. Performance and VDD scaling in deep submicrometer CMOS. IEEE J. Solid-State Circ. 33, 10, 1586--1589.
[7]
Ellis, T., Black, J., Xu, M., and Makris, D. 2005. A distributed multicamera surveillance system. In Ambient Intelligence---A Novel Paradigm, Springer, New York, 107--138.
[8]
Ferscha, A., Vogl, S., and Wolfgang, B. 2002. Ubiquitous context sensing in wireless environments. In Proceedings of the 4th Austrian-Hungarian Workshop on Distributed and Parallel Systems (DAPSYS).
[9]
Hamamoto, T., Nagao, S., and Aizawa, K. 2002. Real-time objects tracking by using smart image sensor and FPGA. In Proceedings of the IEEE International Conference on Image Processing.
[10]
Hariharakrishnan, K., Schonfeld, D., Raffy, P., and Yassa, F. 2003. Object tracking using adaptive block matching. In Proceedings of the IEEE International Conference on Multimedia and Expo.
[11]
Hu, X., Ogras, U. Y., Zamora, N. H., and Marculescu, R. 2004. Data partitioning techniques for pervasive multimedia platforms. In Proceedings of the IEEE International Conference on Multimedia and Expo.
[12]
Huang, Y., Hsieh, B., Chien, S., and Chen, L. 2002. Simple and effective algorithm for automatic tracking of a single object using a pan-tilt-zoom camera. In Proceedings of the IEEE International Conference on Multimedia and Expo.
[13]
Lee, C., Wang, Y., and Yang, T. 1994. Static global scheduling for optimal computer vision and image processing operations on distributed-memory multiprocessors. Tech. Rep., Computer Science Department. University of California at Santa Barbara.
[14]
Omnivision. 2007. OV7141 CMOS VGA. CameraChip. http://www.ovt.com/pdfs/pb_7141_7640.pdf.
[15]
Rabaey, J., Chandrakasan, A., and Nikolic', B. 2003. Digital Integrated Circuits, 2nd ed. Prentice Hall, Upper Saddle River, NJ, 213--231.
[16]
Razeeb, K., Bellis, S., O'Flynn, B., Barton, J., Delaney, K., and O'Mathuna, C. 2004. A hybrid network of autonomous sensor nodes. In Proceedings of the 2nd European Union Symposium on Ambient Intelligence.
[17]
Senior, A., Hampapur, A., Brown, L., Tian, Y.-L., Shu, C.-F., and Pankanti, S. 2005. Distributed active multicamera networks. In Ambient Intelligence---A Novel Paradigm. Springer, New York, 89--105.
[18]
Sezan, M. I. and Lagendijk, R. L. 1993. Motion Analysis and Image Sequence Processing. Kluwer Academic, Norwell, MA.
[19]
Sun, X., Foote, J., Kimber, D., and Manjunath, B. 2001. Panoramic video capturing and compressed domain virtual camera control. In Proceedings of the ACM International Conference on Multimedia.
[20]
Texas Instruments. 2007. TMS320VC5510. http://focus.ti.com/docs/prod/folders/print/tms320vc 5510a.html.
[21]
Wolf, W., Ozer, B., and Lv, T. 2002. Smart cameras as embedded systems. Comput. 35, 9, 48--53.
[22]
Zamora, N. H., Hu, X., Ogras, U. Y., and Marculescu, R. 2004. Resource-aware video processing techniques for ambient multimedia systems. In Proceedings of the IEEE International Conference on Multimedia and Expo.

Recommendations

Reviews

Henk Sips

The problem of video object tracking in a multi-camera scenario, where the cameras and the associated video processing nodes have power constraints, is covered in this paper. The basic application scenario is a grid of identical video processing nodes that wirelessly communicate with each other and a base station. An example of such a system can be used in surveillance systems. Because the power consumption of the individual nodes is an important design criterion, optimizations in the video processing chain or alternative hardware architectures can substantially contribute to an increase in battery lifetime. Basically, the paper is an extensive study of two different methods for power saving in the described application scenario: the region of interest (ROI) method and the adaptive data partitioning (ADP) method. In the ROI method, a single moving object, bounded by a rectangular region, is processed, ignoring the rest of the frame. The ADP method is totally different. In this method, a node has multiple central processing units (CPUs) and the scene observed by the camera is subdivided among the processors. Only the processors that have a moving scene part are switched on, and the others are switched off. Because of the multiprocessor architecture in the ADP method, each individual processor can operate at a much lower speed compared to the single processor scenario in the ROI method; this results in much lower power consumption per processor. The authors describe a number of possible subframe-to-processor mapping scenarios and formulate a performance model for selecting the optimal number of processors. Both methods are described clearly and in detail. Various experiments are described to validate and compare the methods. A real system was built for the ROI method, and a multiprocessor simulator was used for the ADP method. The authors conclude that the ROI method results in power savings of 80 percent, compared with 60 percent for the ADP method. What I like about the paper is the elaborate discussion of all of the decision points and parameter settings involved in selecting the right video object tracking method. Personally, I prefer real experiments because, unfortunately, researchers in academia often rely on simulation only, where the validity of the applied models remains open. This is a somewhat weak point of the paper: the ADP multi-processor experiments are simulation only. I know from personal experience that, in terms of power savings, all system components have to be taken into account to obtain realistic results, something that is not always obvious in a simulation. On the other hand, one could argue that the overall results measured in real systems will generally be worse than those derived from simulations. In this case, the presented simulation results in the paper could be viewed as an upper bound and the conclusions would still be valid. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Design Automation of Electronic Systems
ACM Transactions on Design Automation of Electronic Systems  Volume 13, Issue 1
January 2008
496 pages
ISSN:1084-4309
EISSN:1557-7309
DOI:10.1145/1297666
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

Publication History

Published: 06 February 2008
Accepted: 01 July 2007
Revised: 01 April 2007
Received: 01 August 2006
Published in TODAES Volume 13, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Region-of-interest (ROI)
  2. data partitioning
  3. lossy and lossless video processing
  4. real-time video processing

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 312
    Total Downloads
  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media