skip to main content
10.1145/1502650.1502656acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
research-article

User-oriented document summarization through vision-based eye-tracking

Published: 08 February 2009 Publication History

Abstract

We propose a new document summarization algorithm which is personalized. The key idea is to rely on the attention (reading) time of individual users spent on single words in a document as the essential clue. The prediction of user attention over every word in a document is based on the user's attention during his previous reads, which is acquired via a vision-based commodity eye-tracking mechanism. Once the user's attentions over a small collection of words are known, our algorithm can predict the user's attention over every word in the document through word semantics analysis. Our algorithm then summarizes the document according to user attention on every individual word in the document. With our algorithm, we have developed a document summarization prototype system. Experiment results produced by our algorithm are compared with the ones manually summarized by users as well as by commercial summarization software, which clearly demonstrates the advantages of our new algorithm for user-oriented document summarization.

References

[1]
Bibliography summarization papers. http://www.summarization.com/summ.pdf, last updated on October 20, 2008. last visited on December 11, 2008.
[2]
R. A. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999.
[3]
A. Bulling, J. A.Ward, H. Gellersen, and G. Tröster. Robust recognition of reading activity in transit using wearable electrooculography. In Pervasive '08: Proceedings of the 6th International Conference on Pervasive Computing, pages 19--37, 2008.
[4]
A. Bulling, D. Roggen, and G. Tröster. It's in your eyes: towards context-awareness and mobile HCI using wearable EOG Goggles. In UbiComp '08: Proceedings of the 10th International Conference on Ubiquitous Computing, pages 84--93, New York, NY, USA, 2008. ACM.
[5]
E. H. Chi, M. Gumbrecht, and L. Hong. Visual foraging of highlighted text: An eye-tracking study. In HCII '07: Proceedings of HCI International Conference, pages 589--598, 2007.
[6]
The New York Times Company. The New York Times, http://www.nytimes.com/, last visited on December 11, 2008.
[7]
T. Darrell, N. Checka, A. Oh, and L. Morency. Exploring vision-based interfaces: How to use your head in dual pointing tasks. MIT AI Memo 2002-001, 2002.
[8]
Z. Dou, R. Song, and J.-R. Wen. A large-scale evaluation and analysis of personalized search strategies. In WWW '07: Proceedings of International Conference on World Wide Web, pages 581--590, New York, NY, USA, 2007. ACM.
[9]
G. Dupret, V. Murdock, and B. Piwowarski. Web search engine evaluation using clickthrough data and a user model. In WWW '07: Proceedings of International Conference on World Wide Web, Banff, Canada, 2007.
[10]
G. Erkan and D. Radev. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research (JAIR), 22:457--479, 2004.
[11]
American Association for the Advancement of Science. Science magazine, http://www.sciencemag.com/, last visited on December 11, 2008.
[12]
S. Fox, K. Karnawat, M. Mydland, S. Dumais, and T. White. Evaluating implicit measures to improve web search. ACM Transactions on Information Systems, 23(2):147--168, 2005.
[13]
X. Fu. Evaluating sources of implicit feedback in web searches. In RecSys '07: Proceedings of the 1st ACM International Conference on Recommender Systems, pages 191--194, New York, NY, USA, 2007. ACM.
[14]
Y. Gong and X. Liu. Generic text summarization using relevance measure and latent semantic analysis. In SIGIR '01: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 19--25, New York, NY, USA, 2001. ACM.
[15]
D. Gorodnichy. Perceptual cursor-a solution to the broken loop problem in vision-based hands-free computer control devices. National Research Council Canada Publication, NRC-48472:1--23, 2006.
[16]
L. A. Granka, T. Joachims, and G. Gay. Eye-tracking analysis of user behavior in www search. In SIGIR '04: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 478--479, New York, NY, USA, 2004. ACM.
[17]
Z. Guan and E. Cutrell. An eye tracking study of the effect of target rank on web search. In CHI '07: Proceedings of SIGCHI Conference on Human Factors in Computing Systems, pages 417--420, New York, NY, USA, 2007. ACM.
[18]
W. S. A. Halabi, M. Kubat, and M. Tapia. Time spent on a web page is sufficient to infer a user's interest. In IMSA '07: Proceedings of IASTED European Conference, pages 41--46, Anaheim, CA, USA, 2007. ACTA Press.
[19]
19. C.-K. Huang, Y.-J. Oyang, and L.-F. Chien. A contextual term suggestion mechanism for interactive web search. In WI '01: Proceedings of Asia-Pacific Conference on Web Intelligence: Research and Development, pages 272--281, London, UK, 2001. Springer-Verlag.
[20]
T. Joachims. A probabilistic analysis of the rocchio algorithm with tfidf for text categorization. In ICML '97: Proceedings of International Conference on Machine Learning, pages 143--151, San Francisco, CA, USA, 1997. Morgan Kaufmann Publishers Inc.
[21]
T. Joachims. Optimizing search engines using clickthrough data. In KDD '02: Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 133--142, New York, NY, USA, 2002. ACM.
[22]
T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR '05: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 154--161, New York, NY, USA, 2005. ACM.
[23]
K. S. Jones. What might be in a summary. In Information Retrieval 93: Von der Modellierung zur Anwendung, pages 9--26, 1993.
[24]
D. Kelly and N. J. Belkin. Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback. In SIGIR '01 Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 408--409, New York, NY, USA, 2001. ACM.
[25]
D. Kelly and N. J. Belkin. Display time as implicit feedback: understanding task effects. In SIGIR '04: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 377--384, New York, NY, USA, 2004. ACM.
[26]
K.-N. Kim and R. Ramakrishna. Vision-based eye-gaze tracking for human computer interface. SMC '99: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, 2:324--329, 1999.
[27]
Y. Li, Z. A. Bandar, and D. Mclean. An approach for measuring semantic similarity between words using multiple information sources. IEEE Transactions on Knowledge and Data Engineering, 15(4):871--882, 2003.
[28]
C.-Y. Lin. Training a selection function for extraction. In CIKM '99: Proceedings of the 8th ACM International Conference on Information and Knowledge Management, pages 55--62, New York, NY, USA, 1999. ACM.
[29]
Y.-P. Lin, Y.-P. Chao, C.-C. Lin, and J.-H. Chen. Webcam mouse using face and eye tracking in various illumination environments. EMBS '05: Proceedings of 27th IEEE Annual International Conference of Engineering in Medicine and Biology Society, pages 3738--3741, 2005.
[30]
F. Liu, C. Yu, and W. Meng. Personalized web search by mapping user queries to categories. In CIKM '02: Proceedings of the 11th ACM International Conference on Information and Knowledge Management, pages 558--565, New York, NY, USA, 2002. ACM.
[31]
Y. Lv, L. Sun, J. Zhang, J.-Y. Nie, W. Chen, and W. Zhang. An iterative implicit feedback approach to personalized search. In ACL '06: Proceedings of International Conference on Computational Linguistics, pages 585--592, Morristown, NJ, USA, 2006. Association for Computational Linguistics.
[32]
I. Mani. Advances in Automatic Text Summarization. MIT Press, Cambridge, MA, USA, 1999.
[33]
Microsoft. Word (software), http://office.microsoft.com/word/, Microsoft Corporation, last visited on December 11, 2008.
[34]
J. Pitkow, H. Sch¨utze, T. Cass, R. Cooley, D. Turnbull, A. Edmonds, E. Adar, and T. Breuel. Personalized search. Communications of the ACM, 45(9):50--55, 2002.
[35]
D. Radev, T. Allison, S. Blair-Goldensohn, J. Blitzer, A. C¸ elebi, S. Dimitrov, E. Drabek, A. Hakim, W. Lam, D. Liu, J. Otterbacher, H. Qi, H. Saggion, S. Teufel, M. Topper, A. Winkel, and Z. Zhang. MEAD--A platform for multidocument multilingual text summarization. In LREC '04: The 2nd International Conference on Language Resources and Evaluation, Lisbon, Portugal, May 2004.
[36]
F. Radlinski and T. Joachims. Query chains: learning to rank from implicit feedback. In KDD '05: Proceedings of ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, pages 239--248, New York, NY, USA, 2005. ACM.
[37]
R. W. Reeder, P. Pirolli, and S. K. Card. Webeyemapper and weblogger: tools for analyzing eye tracking data collected in web-use studies. In CHI '01: CHI '01 Extended Abstracts on Human Factors in Computing Systems, pages 19--20, New York, NY, USA, 2001. ACM.
[38]
R. Ruddarraju, A. Haro, K. Nagel, Q. T. Tran, I. A. Essa, G. Abowd, and E. D. Mynatt. Perceptual user interfaces using vision-based eye tracking. In ICMI '03: Proceedings of 5th International Conference on Multimodal Interfaces, pages 227--233, New York, NY, USA, 2003. ACM.
[39]
G. Salton and C. Buckley. Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4):288--297, 1990.
[40]
M. Speretta and S. Gauch. Personalized search based on user search histories. In WI '05: Proceedings of IEEE/WIC/ACM International Conference on Web Intelligence, pages 622--628, Washington, DC, USA, 2005. IEEE Computer Society.
[41]
M.-C. Su, S.-Y. Su, and G.-D. Chen. A low-cost vision-based human-computer interface for people with severe disabilities. Biomedical Engineering Applications, Basis, and Communications, 17:284--292, 2005.
[42]
R. White, J. M. Jose, and I. Ruthven. Comparing explicit and implicit feedback techniques for web retrieval: Trec-10 interactive track report. In TREC 2001.
[43]
R. White, I. Ruthven, and J. M. Jose. The use of implicit evidence for relevance feedback in web retrieval. In Proceedings of BCS-IRSG European Colloquium on IR Research, pages 93--109, London, UK, 2002. Springer-Verlag.
[44]
S. Xu, H. Jiang, and F. C. Lau. Personalized online document, image and video recommendation via commodity eye-tracking. In RecSys '08: Proceedings of the 2nd ACM International Conference on Recommender Systems, pages 83--90, New York, NY, USA, 2008. ACM.
[45]
S. Xu, Y. Zhu, H. Jiang, and F. C. M. Lau. A user-oriented webpage ranking algorithm based on user attention time. In AAAI '08: Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pages 1255--1260, 2008, AAAI Press.
[46]
J.-Y. Yeh, H.-R. Ke, W.-P. Yang, and I.-H. Meng. Text summarization using a trainable summarizer and latent semantic analysis. Information Processing and Management, 41(1):75--95, 2004.
[47]
P. Zielinski. Opengazer: open-source gaze tracker for ordinary webcams (software), Samsung and The Gatsby Charitable Foundation. http://www.inference.phy.cam.ac.uk/opengazer/, last visited on December 11 2008.

Cited By

View all
  • (2023)Marvista: Exploring the Design of a Human-AI Collaborative News Reading ToolACM Transactions on Computer-Human Interaction10.1145/360933130:6(1-27)Online publication date: 25-Sep-2023
  • (2023)Eye-Tracking Based Automatic Summarization of Lecture Slides2023 International Conference on Computer and Applications (ICCA)10.1109/ICCA59364.2023.10401718(1-7)Online publication date: 28-Nov-2023
  • (2023)Mining Eye-Tracking Data for Text SummarizationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222782740:17(4887-4905)Online publication date: 21-Jul-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IUI '09: Proceedings of the 14th international conference on Intelligent user interfaces
February 2009
522 pages
ISBN:9781605581682
DOI:10.1145/1502650
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 February 2009

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. commodity eye-tracking
  2. implicit user feedback
  3. personalized discourse abstract
  4. user attention
  5. user-oriented document summarization

Qualifiers

  • Research-article

Conference

IUI09
IUI09: 14th International Conference on Intelligent User Interfaces
February 8 - 11, 2009
Florida, Sanibel Island, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)13
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Marvista: Exploring the Design of a Human-AI Collaborative News Reading ToolACM Transactions on Computer-Human Interaction10.1145/360933130:6(1-27)Online publication date: 25-Sep-2023
  • (2023)Eye-Tracking Based Automatic Summarization of Lecture Slides2023 International Conference on Computer and Applications (ICCA)10.1109/ICCA59364.2023.10401718(1-7)Online publication date: 28-Nov-2023
  • (2023)Mining Eye-Tracking Data for Text SummarizationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222782740:17(4887-4905)Online publication date: 21-Jul-2023
  • (2022)Eye-Tracking Assistive Technologies for Individuals With Amyotrophic Lateral SclerosisIEEE Access10.1109/ACCESS.2022.316407510(41952-41972)Online publication date: 2022
  • (2021)Implicit Visual Attention Feedback System for Wikipedia UsersProceedings of the 17th International Symposium on Open Collaboration10.1145/3479986.3479993(1-11)Online publication date: 15-Sep-2021
  • (2021)Multi-Sensor Eye-Tracking Systems and Tools for Capturing Student Attention and Understanding Engagement in Learning: A ReviewIEEE Sensors Journal10.1109/JSEN.2021.310570621:20(22402-22413)Online publication date: 15-Oct-2021
  • (2021)SliceProcedia Computer Science10.1016/j.procs.2021.09.069192:C(2971-2979)Online publication date: 1-Jan-2021
  • (2020)Deep Learning-Enhanced Framework for Performance Evaluation of a Recommending Interface with Varied Recommendation Position and Intensity Based on Eye-Tracking Equipment Data ProcessingElectronics10.3390/electronics90202669:2(266)Online publication date: 5-Feb-2020
  • (2020)SuDocuProceedings of the VLDB Endowment10.14778/3415478.341549413:12(2861-2864)Online publication date: 14-Sep-2020
  • (2020)WikiGazeProceedings of the 3rd Workshop on Human Factors in Hypertext10.1145/3406853.3432662(1-9)Online publication date: 4-Dec-2020
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media