skip to main content
10.1145/1111449.1111463acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
Article

Posture and activity silhouettes for self-reporting, interruption management, and attentive interfaces

Published: 29 January 2006 Publication History

Abstract

In this paper we present a novel system for monitoring a computer user's posture and activities in front of the computer (e.g., reading, speaking on the phone, etc.) for self-reporting. In our system, a camera and a microphone are placed in front of a computer work area (e.g., on top of the computer screen). The system can be used as a component in an attentive interface, or for giving the user real time feedback on the goodness of his current posture, and generating summaries of postures and activities over a specified period of time (e.g., hours, days, months, etc.). All elements of the system are highly customizable: the user decides what "good" postures are, what alarms and interruptions are triggered, if any, and what activity and posture summaries are generated. We present novel algorithms for posture measurement (using geometric features of the user's silhouette), and activity classification (using machine learning). Finally, we present experiments that show the feasibility of our approach.

References

[1]
A Guide to Occupational Health and Safety in the New Zealand Sex Industry. Occupational Safety and Health Serv., Dept. of Labor, Wellington, New Zealand, June 2004.
[2]
M.D. Abramoff, P.J. Magelhaes, and S.J. Ram, S.J. "Image Processing with ImageJ," Biophotonics International, volume 11, issue 7, pp. 36--42, 2004.
[3]
P.D. Adamczyk and B.P. Bailey, "A Method and System for Intelligent Interruption Management," in proc. 4th International Workshop on TAsk MOdels and DIAgrams for user interface design For Work and Beyond, Gdansk, Poland, September 26-27, 2005.
[4]
A. Adamek, N.E. O'Connor, and G. Jones, "An Integrated Approach for Object Shape Registration and Modeling," in 2005 Intl. Workshop on Multimedia Info. Retrieval in conjunction with SIGIR 2005, Rio de Janeiro, Brazil, 2005.
[5]
O. Basir, D. Bullock, and E. Breza, "Visual classification and posture estimation of multiple vehicle occupants," US Patent #20040220705, Nov. 4, 2004.
[6]
B. Boulay, F. Bremond, and M. Thonnat, "Human Posture Recognition in Video Sequence," in pro. Joint IEEE International Workshop on VS-PETS, pp.23-29, Nice, France?Oct. 11-12?2003.
[7]
D. Chen, and R. Vertegaal, "Using Mental Load for Managing Interruptions in Psychologically Attentive User Interfaces," ACM CHI 2004, Montreal, Canada, April 2004.
[8]
M.-S. Chuang, and M.-S. Huang, "System and method for detecting unhealthy operation posture of using keyboard," US Patent #20030151595, Feb. 7, 2002.
[9]
M.D. Cordea, E.M. Petriu, N.D. Georganas, D.C.Petriu, and T.E. Whalen, "Real-time 2(1/2)-D Head Pose Recovery for Model-Based Video Coding," IEEE Trans. on Instrument. and Measurement, Vol. 50, No. 4, August 2001.
[10]
R. Cucchiara, C. Grana, A. Prati, and R. Vezzani, "Probabilistic Posture Classification for Human-Behavior Analysis," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 35, No.1, January 2005.
[11]
R. Donkin, "Typing Injury FAQ: Software Monitoring Tools," (http://www.faqs.org/faqs/typing-injury-faq/software), 1995, accessed Aug. 20. 2005.
[12]
A. Elgammal, V. Shet, Y. Yacoob, L. S. Davis, "Gesture Recognition using a Probabilistic Framework for Pose Matching," The 7th Int. Conf. on Control, Automation, Robotics and Vision, ICARCV, Singapore in Dec. 2-5, 2002.
[13]
J. Gemmell, R. Lueder, and B. Gordon, "The MyLifeBits Lifetime Store," ACM SIGMM 2003 Workshop on Experiential Telepresence (ETP 2003), November 7, 2003, Berkeley, CA.
[14]
S.R. Gunn and M.S. Nixon, "Snake Head Boundary Extraction using Global and Local Energy Minimisation," in proc. IEEE Int. Conf. on Pattern Recognition, pp. 581--585. Vienna, 1996.
[15]
I. Haritaoglu, D. Harwood, and L. Davis, "Ghost: A Human Body Part Labeling System Using Silhouettes," in Proc. Intl. Conf. On Pattern Recognition, Vol. 1, pp. 77--82, 1998.
[16]
A. Hedge, and E.J. Ray, "Effects of an electronic height-adjustable worksurface on self-assessed musculoskeletal discomfort and productivity among computer workers," proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting, New Orleans, Sept. 20-24, HFES, Santa Monica, 1091--1095, 2004.
[17]
A. Hedge, "Effects of Ergonomic Management Software on Employee Performance," Cornell Human Factors Laboratory Technical Report #/RP9991, 1991.
[18]
E. Horvitz, P. Koch, and J. Apacible, "BusyBody: Creating and Fielding Personalized Models of the Cost of Interruption," in CSCW 2004, Chicago, IL, Nov. 2004.
[19]
H. Ikeda, M. Maeda, N. Kato, and H. Kashimura, "Classification of Human Actions Using Face and Hands Detection," in proc. ACM Multimedia 2004, New York City, Oct. 2004.
[20]
S. Illic, M. Salzmann, and P. Fua, "Implicit Surfaces Make for Better Silhouettes," Computer Vision and Pattern Recognition, (1) 2005: 1135--1141. San Diego, CA, June 2005.
[21]
A. Jaimes and N. Sebe, "Multimodal Human Computer Interaction: A Survey," IEEE Int. Workshop on Human-Computer Interaction at IEEE Int. Conf. on Computer Vision (ICCV 2005), Beijing, China, Oct. 15-21, 2005.
[22]
A. Jaimes, J. Liu, and N. Sebe, "Affective Meeting Video Analysis," in proc. IEEE ICME 2005, Amsterdam, The Netherlands, July 2005.
[23]
A. Jaimes, and J. Liu, "Sit Straight (and tell me what I did today): A Human Posture Alarm and Activity Summarization System," in ACM Workshop on Continuous Archival and Retrieval of Personal Experiences (CARPE 05) in conjunction with ACM MM 2005, Singapore, 2005.
[24]
M. Kölsch, A. C. B., and M. Turk, "Postural Comfort Zone for Reaching Gestures," in HFES Annual Meeting Notes, October 2003.
[25]
M.W. Lee and I. Cohen, "Human Upper Body Pose Estimation in Static Images, " in proc. ECCV 2004, Vol II, pp. 126--138, 2004.
[26]
J. Lee, B. Moghaddam, H. Pfister, and R. Machiraju, "Finding Optimal Views for 3D Face Shape Modeling," in proc. International Conference on Automatic Face and Gesture Recognition, pp. 31--36?Seoul, Korea, May 2004.
[27]
H. Nomura and T. Shima, "Image feature extraction apparatus, method of extracting image characteristic, monitoring and inspection system, exposure system, and interface system," US Patent #20020015526, Feb. 7, 2002.
[28]
R. Rosales and S. Sclaroff, "Inferring Body Pose Without Tracking Body Parts," in IEEE Conf. on Comp. Vision and Pattern Recognition, Hilton Head Island, USA, June 2000.
[29]
G. Shakhnarovich, P. Viola, and T. Darrell, "Fast Pose Estimation with Parameter-Sensitive Hashing," IEEE International Conference on Computer Vision (ICCV), Vol. 2, pp. 750--757, October 2003.
[30]
C. Sminchisescu and A. Telea, "Human Pose Estimation From Silhouettes A Consistent Approach Using Distance Level Sets," WSCG International Conference on Computer Graphics, Visualization and Computer Vision, 2002.
[31]
P. Smith, N.d.V. Lobo, and M. Shah, "Temporal Boost for Event Recognition," in proc. ICCV 2005, Beijing, China, Oct. 2005.
[32]
N. Stanton, A. Hedge, K. Brookhuis, E. Salas, and H.W. Hendrick. Handbook of Human Factors and Ergonomics Methods. CRC Press, 2004.
[33]
T. Suzuki and D. Miwako, "Life support apparatus and method and method for providing advertisement information," US Patent #20010049471, Dec. 6, 2001.
[34]
R. Vertegaal (ed.), "Attentive User Interfaces" special issue, Communications of the ACM, Vol. 46, No. 3, March, 2003.
[35]
I.H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, 1999.
[36]
Y. Zheng, "Method and apparatus for sensing body gesture, posture and movement," US Patent # 20040024312, Feb. 5, 2004.
[37]
Ergonomix (http://www.publicspace.net/ergonomix/)
[38]
Stretch Break (http://www.paratec.com/)
[39]
RSI guard (http://www.rsiguard.com/)
[40]
Ergotimer (http://www.tropsoft.com/ergotimer/ergonomics.htm)
[41]
Break Time (http://kadmi.com/products.html)
[42]
Protector of Health (http://www.olympsoft.com/)

Cited By

View all
  • (2024)Sitting Posture Recognition and Feedback: A Literature ReviewProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642657(1-20)Online publication date: 11-May-2024
  • (2021)Notification Timing Control While Reading Text InformationHuman Interface and the Management of Information. Information Presentation and Visualization10.1007/978-3-030-78321-1_11(125-139)Online publication date: 3-Jul-2021
  • (2018)ActiveErgoProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3174132(1-8)Online publication date: 21-Apr-2018
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IUI '06: Proceedings of the 11th international conference on Intelligent user interfaces
January 2006
392 pages
ISBN:1595932879
DOI:10.1145/1111449
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 January 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. computer vision
  2. ergonomics
  3. posture

Qualifiers

  • Article

Conference

IUI06
IUI06: 11th International Conference on Intelligent User Interfaces
January 29 - February 1, 2006
Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Sitting Posture Recognition and Feedback: A Literature ReviewProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642657(1-20)Online publication date: 11-May-2024
  • (2021)Notification Timing Control While Reading Text InformationHuman Interface and the Management of Information. Information Presentation and Visualization10.1007/978-3-030-78321-1_11(125-139)Online publication date: 3-Jul-2021
  • (2018)ActiveErgoProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3174132(1-8)Online publication date: 21-Apr-2018
  • (2015)Nekoze!Proceedings of the 9th International Conference on Pervasive Computing Technologies for Healthcare10.5555/2826165.2826203(237-240)Online publication date: 20-May-2015
  • (2015)Peripheral Notification That Does Not Bother User's Operations by Peripheral Cognition TechnologyTransactions of the Japanese Society for Artificial Intelligence10.1527/tjsai.30.44930:2(449-458)Online publication date: 2015
  • (2012)User state identification through desktop interactionThe 6th International Conference on Soft Computing and Intelligent Systems, and The 13th International Symposium on Advanced Intelligence Systems10.1109/SCIS-ISIS.2012.6505185(395-398)Online publication date: Nov-2012
  • (2012)Applying key typing pressure to estimate a user's state of activity2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication10.1109/ROMAN.2012.6343751(185-190)Online publication date: Sep-2012
  • (2011)Human neck's posture measurement using a 3-axis accelerometer sensorProceedings of the 2011 international conference on Computational science and Its applications - Volume Part V10.5555/2029427.2029437(96-109)Online publication date: 20-Jun-2011
  • (2009)A vision-based attentive user interface with (semi-)automatic parameter calibrationProceedings of the International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing10.1145/1731740.1731777(1-6)Online publication date: 18-Jun-2009
  • (2009)Low-cost, Automated Assessment of Sit-To-Stand Movement in “Natural” Environments4th European Conference of the International Federation for Medical and Biological Engineering10.1007/978-3-540-89208-3_20(76-79)Online publication date: 2009
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media