skip to main content
10.1145/1647314.1647354acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
technical-note

Realtime meeting analysis and 3D meeting viewer based on omnidirectional multimodal sensors

Published: 02 November 2009 Publication History

Abstract

This demo presents a realtime system for analyzing group meetings. Targeting round-table meetings, this system employs an omnidirectional camera-microphone system. The goal of this system is to automatically discover "who is talking to whom and when". To that purpose, the face pose/position of meeting participants are tracked on panorama images acquired from fisheye-based omnidirectional cameras. From audio signals obtained with microphone array, speaker diarization, i.e. the estimation of "who is speaking and when", is carried out. The visual focus of attention, i.e. "who is looking at whom", is esimated from the result of face tracking. The results are displayed based on a 3D visualization scheme. The advantage of our system is its realtimeness. We will demonstrate the portable version of the system consisting of two laptop PCs. In addition, we will showcase our meeting playback viewer with man-machine interfaces that allow users to freely control space and time of meeting scenes. With this viewer, users can also experince 3D positional sound effect linked with 3D viewpoint, using enhanced audio tracks for each participant.

References

[1]
S. Araki, H. Sawada, and S. Makino."Blind speech separation in a meeting situation with maximum SNR beamformers", In Proc. ICASSP, pages 41--44, 2007.
[2]
D. Mikami, K. Otsuka, and J. Yamato."Memory-based particle filter for face pose tracking robust under complex dynamics", In Proc. IEEE CVPR, 2009.
[3]
K. Otsuka, S. Araki, K. Ishizuka, M. Fujimoto, M. Heinrich, and J. Yamato. "A realtime multimodal system for analyzing group meetings by combining face pose tracking and speaker diarization", In Proc. 10th ICMI, pages 257--264, 2008.

Cited By

View all
  • (2023)Applications of Deep Learning for Top-View Omnidirectional Imaging: A Survey2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW59228.2023.00683(6421-6433)Online publication date: Jun-2023
  • (2021)Estimation of Empathy Skill Level and Personal Traits Using Gaze Behavior and Dialogue Act During Turn-ChangingHCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence10.1007/978-3-030-90963-5_4(44-57)Online publication date: 11-Nov-2021
  • (2019)Prediction of Who Will Be Next Speaker and When Using Mouth-Opening Pattern in Multi-Party ConversationMultimodal Technologies and Interaction10.3390/mti30400703:4(70)Online publication date: 26-Oct-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI-MLMI '09: Proceedings of the 2009 international conference on Multimodal interfaces
November 2009
374 pages
ISBN:9781605587721
DOI:10.1145/1647314

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. fisheye lens
  2. focus of attention
  3. meeting analysis
  4. microphone array
  5. omnidirectional cameras
  6. realtime system
  7. speaker diarization

Qualifiers

  • Technical-note

Conference

ICMI-MLMI '09
Sponsor:

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Applications of Deep Learning for Top-View Omnidirectional Imaging: A Survey2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW59228.2023.00683(6421-6433)Online publication date: Jun-2023
  • (2021)Estimation of Empathy Skill Level and Personal Traits Using Gaze Behavior and Dialogue Act During Turn-ChangingHCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence10.1007/978-3-030-90963-5_4(44-57)Online publication date: 11-Nov-2021
  • (2019)Prediction of Who Will Be Next Speaker and When Using Mouth-Opening Pattern in Multi-Party ConversationMultimodal Technologies and Interaction10.3390/mti30400703:4(70)Online publication date: 26-Oct-2019
  • (2019)Estimating Interpersonal Reactivity Scores Using Gaze Behavior and Dialogue Act During Turn-ChangingSocial Computing and Social Media. Communication and Social Communities10.1007/978-3-030-21905-5_4(45-53)Online publication date: 12-Jun-2019
  • (2018)Analyzing Gaze Behavior and Dialogue Act during Turn-taking for Estimating Empathy Skill LevelProceedings of the 20th ACM International Conference on Multimodal Interaction10.1145/3242969.3242978(31-39)Online publication date: 2-Oct-2018
  • (2017)Analyzing gaze behavior during turn-taking for estimating empathy skill levelProceedings of the 19th ACM International Conference on Multimodal Interaction10.1145/3136755.3136786(365-373)Online publication date: 3-Nov-2017
  • (2017)Analysis of Small GroupsSocial Signal Processing10.1017/9781316676202.025(349-367)Online publication date: 13-Jul-2017
  • (2016)Analyzing mouth-opening transition pattern for predicting next speaker in multi-party meetingsProceedings of the 18th ACM International Conference on Multimodal Interaction10.1145/2993148.2993189(209-216)Online publication date: 31-Oct-2016
  • (2016)Using Respiration to Predict Who Will Speak Next and When in Multiparty MeetingsACM Transactions on Interactive Intelligent Systems10.1145/29468386:2(1-20)Online publication date: 3-Aug-2016
  • (2016)Prediction of Who Will Be the Next Speaker and When Using Gaze Behavior in Multiparty MeetingsACM Transactions on Interactive Intelligent Systems10.1145/27572846:1(1-31)Online publication date: 5-May-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media