Question

(3) Consider the following game. There is a simple random walk on the half line S = {0,1,2, ...}. The game ends when the player chooses or the walk hits

A = {0}. The payoff function is f(k) = k². (a) Show that E(ƒ(Xk+1)|Xk) > f(Xk) holds for k > 0. (b) Conclude that any optimal strategy does not stop on any k > 0. (c) Since the chain reaches 0 with probability 1, any strategy which does not stop on positive integers results in payoff 0 with probability 1. (d) Explain this apparent contradiction.

Fig: 1

Fig: 2

Fig: 3

Fig: 4

Fig: 5