\documentclass[11pt]{exam}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{xstring}

\title{\vspace{-5em}Reinforcement Learning---HW 9 \ifprintanswers Solution \fi}
\date{\vspace{-2em}April 13, 2020}

% If filename contains 'solution', then print answers
\IfSubStr*{\jobname}{solution}{\printanswers}{}

\begin{document}
\maketitle
\thispagestyle{empty} % no page numbers

Here's a partially-expanded Monte Carlo Search Tree for determining the next move in Tic-Tac-Toe:

 \includegraphics[scale=0.4]{hw9_image.pdf}

Assume that $c$, the constant used for UCT (Upper Confidence Bound applied to Trees) is 0.5.

\begin{questions}


\question In this Tic-Tac-Toe game, whose turn is it (X or O)?

\question From your knowledge of Tic-Tac-Toe, what is the best move for that player?


\question
For this question, you'll do one iteration of the MCST algorithm (Selection/Expansion/Simulation/Backup)
\begin{parts}

\part \emph{Selection} What leaf node  is the next to be selected (show your work)?

\part \emph{Expansion} What child node is expanded?

\part \emph{Simulation} Do a rollout (simulation). Use some form of randomization (coin flips, random number generation in Excel or on a website, coin toss, etc.) to do the simulation.  What is the result? If the result is a tie, rerun the simulation until you get a decisive winner.

\part \emph{Backup} Backup the result through the MCST. 


\part If this were the last simulation of the MCST, what action would be chosen and why?

\end{parts}
\end{questions}



\end{document}
