Context-based video retrieval system for the life-log applications

ABSTRACT

Recently, we have often heard the terms "Wearable computing" and "Ubiquitous computing". Our expectation for the future of such new computing environments is growing. One of the characteristics of these computing environments is that they embed computers in our lives. In such environments, digitization of personal experiences will be made possible by continuous recordings using a wearable video camera[6, 7]. This could lead to the "automatic life-log application". However, it is evident that the resulting amount of video content will be enormous. Accordingly, to retrieve and browse desired scenes, a vast quantity of video data must be organized using structural information.In this paper, we attempt to develop a "context-based video retrieval system for life-log applications". This wearable system is capable of continuously capturing data not only from a wearable camera and a microphone, but also from various kinds of sensors such as a brain-wave analyzer, a GPS receiver, an acceleration sensor, and a gyro sensor to extract the user's contexts. In addition, the system provides functions that make efficient video browsing and retrieval possible by using data from these sensors and some databases. For example, we can use the following query using this system. "I talked with Kenji while walking at a shopping center in Shinjuku on a cloudy day in mid-May. The conversation was very interesting! I want to see the video of our outing to remember the contents of the conversation."

REFERENCES

• 1: K. Aizawa, K. Ishijima, and M. Shiina. Summarizing wearable video. In Proceedings of ICIP 2001, pages 398--401. IEEE, October 2001.
• 2: Hisashi Aoki , Bernt Schiele , Alex Pentland, Realtime Personal Positioning System for Wearable Computers, Proceedings of the 3rd IEEE International Symposium on Wearable Computers, p.37, October 18-19, 1999
• 3: B. Clarkson and A. Pentland. Unsupervised clustering of ambulatory audio and video. In Proceedings of ICASSP'99. IEEE, March 1999.
• 4: Jim Gemmell , Gordon Bell , Roger Lueder , Steven Drucker , Curtis Wong, MyLifeBits: fulfilling the Memex vision, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641053]
• 5: Jennifer Healey , Rosalind W. Picard, StartleCam: A Cybernetic Wearable Camera, Proceedings of the 2nd IEEE International Symposium on Wearable Computers, p.42, October 19-20, 1998
• 6: Steve Mann, `WearCam' (The Wearable Camera): Personal Imaging Systems for long--term use in wearable tetherless computer--mediated reality and personal Photo/Videographic Memory Prosthesis, Proceedings of the 2nd IEEE International Symposium on Wearable Computers, p.124, October 19-20, 1998
• 7: Y. Sawahata and K. Aizawa. Wearable imaging system for summarizing personal experiences. In Proceedings of ICME 2003. IEEE, July 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.0 General

General Terms:
Design, Human Factors, Management

Keywords:
video indexing, video retrieval, wearable computing