Memory Augmented Control Networks

By Arbaaz Khan, Clark Zhang, Nikolay A. Atanasov, Konstantinos Karydis, Vijay R. Kumar, Daniel D. Lee. 2017

80 citations recorded by the source corpus.

Planning problems in partially observable environments cannot be solved directly with convolutional networks and require some form of memory. But, even memory networks with sophisticated addressing schemes are unable to learn intelligent reasoning satisfactorily due to the complexity of simultaneously learning to access memory and plan. To mitigate these challenges we introduce the Memory Augmented Control Network (MACN). The proposed network architecture consists of three main parts. The first part uses convolutions to extract features and the second part uses a neural network-based planning module to pre-plan in the environment. The third part uses a network controller that learns to store those specific instances of past information that are necessary for planning. The performance of the network is evaluated in discrete grid world environments for path planning in the presence of simple and complex obstacles. We show that our network learns to plan and can generalize to new environments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Khan et al. (2017) design a network architecture that contains three components: a convolution network for feature extraction, a planner module to pre-plan in the environment, and a controller module that learns to selectively store past information that could be useful for planning.”
Cognitive Mapping and Planning for Visual Navigation (2017, methodology): “[10, 27, 45, 61] use similar 2D spatial representations, Mirowski et al.”
Visual Representations for Semantic Target Driven Navigation (2018, background): “[18] use value iteration networks [19] and imitation learning to learn navigation strategies.”
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning (2018, background): “ architecture and stores a feature vector for every observed location in the world. This approach does not require maintaining a distribution over likely maps. for navigation in discrete environments [11,12,23,37]. We work with continuous action and state spaces, and emphasize efﬁcient learning from”
Learning to Navigate in Cities Without a Map (2018, methodology): “These include Neural SLAM [44] that proposes an RL agent with an external memory to represent an occupancy map and a SLAM-inspired algorithm, Neural Map [36] which proposes a structured 2D memory for navigation, Memory Augmented Control Networks [27], which uses a hierarchical control strategy, and ”
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks (2019, methodology): “[23] apply the very general Differentiable Neural Computer [14] to control problems.”
Unifying Map and Landmark Based Representations for Visual Navigation (2017, result): “In contrast to the above methods, techniques based on learned map construction [12,17,25,35] and value iteration networks [31] have sought to directly incorporate elements of mapping and planning into end-to-end trained navigational policies.”
Benchmarking Classic and Learned Navigation in Complex 3D Environments (2019, background): “Given the planned path, the locomotion module selects an action to follow the path. for planning has been studied both in the fully observable scenario [39, 29] and under partial observability [23, 25].”
Combining Optimal Control and Learning for Visual Navigation in Novel Environments (2019, background): “There has been a recent interest in employing end-to-end learning for training policies for goal-driven navigation [16, 23, 45].”
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight (2019, background): “This approach is related to neural network models that construct maps [33, 34, 35, 36] or perform planning [37].”
Learning to Visually Navigate in Photorealistic Environments Without any Supervision (2020, unknown): “Other works replace SLAM entirely by deep models with no planning but explicit map-like or SLAM inspired memory structures [16, 22, 23, 36].”