stanford graduate
course project

AA 228 Decision Making Under Uncertainty

autumn 2021

In this course, I studied decision making under uncertainty as tools for designing autonomous and decision-support systems. The course spanned Bayesian networks, dynamic programming, RL, and POMDPs and was taught by Dr. Mykel Kochenderfer.

My project for the course consisted of framing the strategy game of Battleship as a POMDP and employing decision-making algorithms that can consistently outperform a human player.

my work

This diagram outlines the belief state updates that occur under sequential actions and observations, using the possible states availble in the game.

Through simulation and testing, framing the problem as a POMDP resulted in a large decrease in moves required to win against a naive player.

These are the performance results after repeated game simulations. We compared the results of each method and provided an analysis of the tradespace between hyperparameters.

This is the cover art for the popular strategy game, Battleship.

← →