Automatic analysis of multimodal group actions in meetings

By Iain McCowan, Daniel Gatica-Perez, Samy Bengio, Guillaume Lathoud, Mark Barnard, Dong Zhang. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005

416 citations recorded by the source corpus.

This paper investigates the recognition of group actions in meetings. A framework is employed in which group actions result from the interactions of the individual participants. The group actions are modeled using different HMM-based approaches, where the observations are provided by a set of audiovisual features monitoring the actions of individuals. Experiments demonstrate the importance of taking interactions into account in modeling the group actions. It is also shown that the visual modality contains useful information, even for predominantly audio-based events, motivating a multimodal approach to meeting analysis.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/6893489f2fc9706bf278e49ff247f0a7254154aa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Detecting Human Behavior Models From Multimodal Observation in a Smart Home (2009, background): “Many approaches for learning and recognizing human (inter)actions and behavior models from sensor data have been proposed in recent years, with particular attention to applications in video surveillance [26], [28], [34], workplace tools [24], [30], and group entertainment [4].”
Multimodal Human Computer Interaction: A Survey (2005, methodology): “In [102], visual features are extracted from head and hand/forearm blobs: the head blob is represented by the vertical position of its centroid, and hand blobs are represented by eccentricity and angle with respect to the horizontal.”
Social signal processing: Survey of an emerging domain (2009, background): “, [33][68][111]), in order to limit parasitic effects and elicit the specific phenomena they want to observe.”
A survey of context modelling and reasoning techniques (2010, background): “[51] go further by proposing a two-layered framework for modelling and recognising individual and group actions in meetings.”
Human-Computer Interaction: Overview on State of the Art (2008, neutral): “• Smart Video Conferencing [59]”
Learning Influence among Interacting Markov Chains (2005, methodology): “We compute the SRP-PHAT measure using the signals from a 8-microphone array [12], which is a continuous value indicating the speech activity from a particular participant.”
Prediction of the Leadership Style of an Emergent Leader Using Audio and Visual Nonverbal Features (2017, neutral): “): [2], [13], [22], [23], [25], [29], [30], [32], [33], [36], [43], [45], [47], [48], [50] D, EL, G, PT”
Automatic nonverbal analysis of social interaction in small groups: A review (2009, background): “…has been made towards the detection, discovery, and recognition of patterns of multi-party interaction management, including turntaking [30] [91] [27] and addressing [74]; group members' internal states, including interest and attraction [131] [52] [102]; individuals' personality traits…”
Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Processing (2012, neutral): “For example, ICSI [126] includes annotation in terms of dialogue acts and interest level, M4 [127] is annotated in terms of interest and turn taking types, AMI [128] provides slides, hand-written and whiteboard notes, and it is annotated in terms of dialogue acts as well, the AMI-12 [124] includes…”