Question

# 1. Question 1 (Bayesian inference, variational inference and sampling) Let y(y) be Lid. samples from the normal distribution N(µ, 7−¹). We specify normal-gamma prior distributions on and r A|TAN (Aer)-¹), 7|40, bo Gamma(a,b), f(a,b)- T(a) For a pair of random variables (X,T), if X\T ~ N{µ, (AT)¯¹) and T ~ Gamma(a, b), then (X,T) follows a normal-gamma distribution with parame- ters (u, A,a,d). The joint probability density function of (X, T) has the form te" exp x²(x = μ)²). 1(z. t|p. A, a. b)-1(a) √27 For the Python programming questions, we set - 0,-10, a-4-10 and observations y(y) are stored in Qty.com (a) Derive the joint prior (μ, 7) and likelihood function p/ya,7). Write down the probability density function of the posterior distribution (r) (no need to derive the exact distribution) Japa+n As +11 (b) In fact, for normal distributed data with unknown mean and precision (inverse of variance), the normal-gamma prior is a conjugate prior and the posterior (y) is also a normal-gamma distribution with parameters ܤܞ Ao +1.00 + + ²(x-8²+ 2(x+1) where is the sample mean. Derive the full conditional posterior distribution µly, and r. (need to obtain the exact distribution) (e) Write down the probability density function of the posterior predictive distribution yly. Describe how to approximate ply"ly) via the simple Monte Carlo approach. 1 (d) The mode of Normal-Gamma(, A, a., b) is (.). Consider the Laplace approximation on the joint posterior (p. 5|B): In (0) In (@Mary))-(0-0)¹ A(0-₁AP) - In 2(0|y). A--VV In z(y) where (7). Derive the approximated posterior distribution (u, ry). Draw two con- tour plots of x(u, rµ) and (µ,5%) respectively in Python. (Python scipy stats package does not provide direct functions to calculate the pdf of the normal-gamma distribution. You need to calculate it by yourself) (e) Assume a mean-field variational inference for the joint posterior (7) - (u)(r). Find the optimal mean-field factor q, and g. Write down the procedures to iteratively update the parameters of q and g; and implement them in Python to obtain the estimated parameters of q; and (set convergence criterion - 10-) (Hints: q;(0) x exp{E, [ln P(D, ¤)])  Fig: 1