docker compose up
localhost:8888my-tokenwork/ folder, and open the GridWorld notebook in the JupyterLab environment.Link to Github repo for the implementation and notebook used to generate some figures:
https://github.com/shantanu/grid-world
The most basic Dynamic Programming application is Gridworld.

We can represent this grid as a 4x4 array, and actions are {u, d, l, r} in each state. This is an undiscounted, episodic task. The states 0 and 15 (gray above) are terminal. If an action takes the agent off the grid, the state is unchanged but we still get -1 reward.
For the first phase of the project, I want to evaluate the equiprobable random policy, where in each state the agent has a 0.25 chance of going in any direction.
Problem: given a policy $\pi$, I want to evaluate $V_\pi(s)$, the state value function, for all s.
This helps because once we have the value function, we can use it to do greedy policy improvement and policy iteration to find the optimal policy.
How: We can iterate the Bellman Equation in place: