Unsupervised learning of event AND-OR grammar and semantics from video

By Zhangzhang Si, Mingtao Pei, Benjamin Z. Yao, Song-Chun Zhu. 2011

96 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning (2017, neutral): “There has been previous research (Si et al., 2011; Pirsiavash & Ramanan, 2014) using stochastic grammar models to capture such temporal relations.”
Joint Video and Text Parsing for Understanding Events and Answering Queries (2013, background): “These energy terms can be either manually speciﬁed by domain experts or learned from data [52], [53], [10].”
Parsing Videos of Actions with Segmental Grammars (2014, methodology): “Grammars: Much prior work has explored contextfree-grammars (CFGs) for gesture and event recognition [7, 15, 22, 26].”
Machine Common Sense Concept Paper (2018, background): “For example: Josh Tenenbaum (MIT) and his colleagues have developed rich probabilistic representations that mimic human learning [33][34]; and Song‐Chun Zhu (UCLA) has developed an array of techniques based on stochastic and‐or‐ graphs [35][36][37].”
Learning latent spatio-temporal compositional model for human action recognition (2013, background): “[31] studied the event grammar model for daily activities based on a predefined set of unary and binary relations.”
High-level event recognition in unconstrained videos (2012, methodology): “[44,125] are not able to detect complex events in situations where (basically) no domain knowledge is available.”
Learning a Generative Model for Multi‐Step Human‐Object Interactions from Videos (2019, methodology): “Although this literature is broad, it is largely based on fully supervised classiﬁcation of video clips [LMSR08,EBMM03,RA09] and only a few methods exist to synthesize interactions [SPYZ11,PJZ11] or to structure the causal dependencies of human actions [FZ16].”
Parsing video events with goal inference and intent prediction (2011, methodology): “The structures of the AoG are learned automatically from the training data as in our companion paper [16], the parameters and temporal relations are also learned from the training data.”
Published as a conference paper at ICLR 2018 S IMULATING A CTION D YNAMICS WITH N EURAL P ROCESS N ETWORKS (2018, background): “Additional work in tracking states with visual or multimodal context has focused on 1) building graph representations for how entities change in goal-oriented domains (Gao et al., 2016; Liu et al., 2016; Si et al., 2011) or 2) tracking visual state changes based on decisions taken by agents in envir”
Prediction of Human Activity by Discovering Temporal Sequence Patterns (2014, background): “In a smart room, people’s intention of activity can be predicted by a user-friendly sensor—camera, so that the system will adaptively provide services, even help if necessary.”
A database for fine grained activity detection of cooking activities (2012, background): “Many approaches have been researched ranging from low level image and video features [6, 15, 38], over semantic human pose detection [33], to temporal activity models [12, 22, 32].”
An end-to-end generative framework for video segmentation and recognition (2015, background): “More recently, FVs have been shown to yield not only higher classification accuracy, but also much more compact feature vectors [11].”
First-Person Activity Recognition: What Are They Doing to Me? (2013, background): “Researchers not only focused on developing reliable video features robust to noise and illumination changes [14, 3, 7], but also proposed various types of hierarchical approaches to recognize high-level activities with multiple actors [12, 9, 17] and even group activities [13].”
Representing Videos Using Mid-level Discriminative Patches (2013, background): “Therefore, such approaches have, for the most part, only been applied to restricted and structured domains such as baseball [11] and office scenes [30].”