This exercise deals with the formal handling of Markov chains and Markov decision processes.
- reading Markov chain graphs and determining state transitions probability
- determining stationary state of Markov chains
- determining state values of Markov reward processes
- reading Markov decision process graphs and determining state transition probability when actions are involved
- evaluating policies (determining state value functions) in Markov decision processes when the policy is given
- calculating action values in Markov decision processes when the policy is given
- state-value based comparison of different policies