Self-Organization of Spatio-Temporal Hierarchy via Learning of Dynamic Visual Image Patterns on Action Sequences

By Minju Jung, Jung-Mock Hwang, Jun Tani. 2015

33 citations recorded by the source corpus.

It is well known that the visual cortex efficiently processes high-dimensional spatial information by using a hierarchical structure. Recently, computational models that were inspired by the spatial hierarchy of the visual cortex have shown remarkable performance in image recognition. Up to now, however, most biological and computational modeling studies have mainly focused on the spatial domain and do not discuss temporal domain processing of the visual cortex. Several studies on the visual cortex and other brain areas associated with motor control support that the brain also uses its hierarchical structure as a processing mechanism for temporal information. Based on the success of previous computational models using spatial hierarchy and temporal hierarchy observed in the brain, the current report introduces a novel neural network model for the recognition of dynamic visual image patterns based solely on the learning of exemplars. This model is characterized by the application of both spatial and temporal constraints on local neural activities, resulting in the self-organization of a spatio-temporal hierarchy necessary for the recognition of complex dynamic visual image patterns. The evaluation with the Weizmann dataset in recognition of a set of prototypical human movement patterns showed that the proposed model is significantly robust in recognizing dynamically occluded visual patterns compared to other baseline models. Furthermore, an evaluation test for the recognition of concatenated sequences of those prototypical movement patterns indicated that the model is endowed with a remarkable capability for the contextual recognition of long-range dynamic visual image patterns.

Subfields: Artificial Intelligence, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

Lifelong learning of human actions with deep neural network self-organization (2017, background): “Our results outperform the overall accuracy reported by Jung et al. (2015) with three different deep learning models: convolutional neural network (CNN, 92.9%),multiple spatiotemporal scales neural network (MSTNN, 95.3%), and 3D CNN (96.2%).”
Neural and Computational Mechanisms of Action Processing: Interaction between Visual and Motor Representations. (2015, background): “Also, recently, deep architectures including dynamical neurons that learn hierarchical spatio-temporal representations have been proposed (Jung et al., 2015).”
Lifelong Learning of Action Representations with Deep Neural Self-Organization (2017, background): “Our results outperform the overall accuracy reported by (Jung, Hwang, and Tani 2015) with three different deep learning models: convolutional neural network (CNN, 92.9%), multiple spatiotemporal scales neural network (MSTNN, 95.3%), and 3D CNN (96.2%).”
Seamless Integration and Coordination of Cognitive Skills in Humanoid Robots: A Deep Learning Approach (2017, methodology): “an ability to recognize dynamic visual scenes [17], and the MTRNN to learn compositional actions [18].”
Answering Schrödinger's question: A free-energy formulation (2017, unknown): “This composition of temporal and spatial scales is evident in the hierarchical organisation of the brain [51,52], and more broadly, suggests that self-organisation should occupy a limited domain (along the diagonal) of a scale space with spatial and temporal dimensions [18] (see Fig.”