You are trying to find the best parking space to use that minimizes the time needed to get to your restaurant. There are 50 parking spaces, and you see spaces 1, 2, ... , 50 in order. As you approach each parking space, you see whether it is full or empty. We assume, somewhat heroically, that the probability that each space is occupied follows an independent Bernoulli process. For example, each space will be occupied with probability p, but will be free with probability 1-p, and each outcome is independent of the other.

It takes two seconds to drive past each parking space and it takes eight seconds to walk past. That is, if we part in space n, it will require 8(50-n) seconds to walk to the restaurant. Furthermore, it would have taken you 2n seconds to get to this space. If you get to the last space without finding an opening, then you will have to drive into a special lot down the block, adding 30 seconds to your trip. 

We want to find an optimal strategy for accepting or rejecting a parking space. 

A. Give the sets of state and action spaces and the set of decision epochs.

B. Give the expected reward function for each time period and the expected terminal reward function.

C. Give a formal statement of the objective function.

D. Give the optimality equations for solving this problem.

E. You have just looked at space 45, which was empty. There are five more spaces remaining (46 through 50). What should you do? Using p=0.6, find the

optimal policy by solving your optimality equations for parking spaces 46 through 50.

F. Give the optimal value of the objective function in part E, corresponding to your optimal solution.