MDP robot grid-world example

Applies value iteration to learn a policy for a robot in a grid world.
Updated 24 Nov 2015

View License

Applies value iteration to learn a policy for a Markov Decision Process (MDP) -- a robot in a grid world.
The world is freespaces (0) or obstacles (1). Each turn the robot can move in 8 directions, or stay in place. A reward function gives one freespace, the goal location, a high reward. All other freespaces have a small penalty, and obstacles have a large negative reward. Value iteration is used to learn an optimal 'policy', a function that assigns a
control input to every possible location.
video at

This function compares a deterministic robot, one that always executes movements perfectly, with a stochastic robot, that has a small probability of moving +/-45degrees from the commanded move. The optimal policy for a stochastic robot avoids narrow passages and tries to move to the center of corridors.

From Chapter 14 in 'Probabilistic Robotics', ISBN-13: 978-0262201629,

Aaron Becker, March 11, 2015

Cite As

Aaron T. Becker's Robot Swarm Lab (2024). MDP robot grid-world example (, MATLAB Central File Exchange. Retrieved .

MATLAB Release Compatibility
Created with R2014b
Compatible with any release
Platform Compatibility
Windows macOS Linux

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!
Version Published Release Notes

added link to video