A = {0}. The payoff function is f(k) = k². (a) Show that E(ƒ(Xk+1)|Xk) > f(Xk) holds for k > 0. (b) Conclude that any optimal strategy does not stop on any k > 0. (c) Since the chain reaches 0 with probability 1, any strategy which does not stop on positive integers results in payoff 0 with probability 1. (d) Explain this apparent contradiction.
Fig: 1
Fig: 2
Fig: 3
Fig: 4
Fig: 5