(4) Consider a Markov chain with infinite state space {1, 2, 3, ...} and assume its transition matrix satisfies p(i + 1, i). = 1 and p(1, i) > 0

for all i > 1. That is, it decreases deterministically when above 1 and jumps out to a random integer (according to some distribution) from 1. (a) Is the chain is irreducible and aperiodic? (b) When is the chain transient, null recurrent, and positively recurrent? (c) When can you find an invariant probability? That is, a function T \begin{aligned} &\text { on the state space such that } \pi(i) \geq 0, \sum_{i} \phi(i)=1, \text { and } \pi(i)=\\ &\sum_{j} \pi(j) p(j, i) \end{aligned}