Fast Reinforcement Learning for Vision-guided Mobile Robots

By Tomás Martínez-Marín, Tom Duckett. 2005

57 citations recorded by the source corpus.

This paper presents a new reinforcement learning algorithm for accelerating acquisition of new skills by real mobile robots, without requiring simulation. It speeds up Q-learning by applying memory-based sweeping and enforcing the “adjoining property”, a technique that exploits the natural ordering of sensory state spaces in many robotic applications by only allowing transitions between neighbouring states. The algorithm is tested within an image-based visual servoing framework on a docking task, in which the robot has to position its gripper at a desired configuration relative to an object on a table. In experiments, we compare the performance of the new algorithm with a hand-designed linear controller and a scheme using the linear controller as a bias to further accelerate the learning. By analysis of the controllability and docking time, we show that the biased learner could improve on the performance of the linear controller, while requiring substantially lower training time than unbiased learning (less than 1 hour on the real robot).

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Jan Peters
2. Jens Kober
3. Junbao Li
4. J. Bagnell
5. Hong Liu
6. R. Peters
7. Bin Wang
8. Karla Conn Welch

In Computer Vision and Pattern Recognition

1. Jan Peters
2. Jens Kober
3. J. Bagnell
4. Junbao Li
5. Hong Liu
6. R. Peters
7. Bin Wang
8. Karla Conn Welch

Citations in the corpus

• Reinforcement learning in robotics: A survey (2013, methodology): “For example, a vision-based mobile robot docking task can be learned faster with such a basic behavior than using Q-learning alone, as demonstrated in (Martínez-Marín and Duckett, 2005) Providing hand-coded, stable initial gaits can significantly help in learning robot locomotion, as shown on a six-”
• Reinforcement Learning with a Supervisor for a Mobile Robot in a Real-world Environment (2007, methodology): “Provost et al. [6] used a simulated environment in which a mobile robot had to maneuver around a corner.”
• Closed-Loop Learning of Visual Control Policies (2011, unknown): “This is actually the basic postulate behind local-appearance methods that have had much success in computer vision applications such as image matching, image retrieval and object recognition (Schmid & Mohr, 1997; Lowe, 2004 ).”
• A Heuristic Reinforcement Learning for Robot Approaching Objects (2006, methodology): “So the evaluation function f(n) is used, f(n) = g(n) +h(n) (2) where g(n) is the number of nodes of the path from the start node to node n, h(n) is the distance in workspace between node n to the goal node of goal.”