1 question 1 bayesian inference variational inference and sampling let

Question

1. Question 1 (Bayesian inference, variational inference and sampling)
Let y(y) be Lid. samples from the normal distribution N(µ, 7−¹).
We specify normal-gamma prior distributions on and r
A|TAN (Aer)-¹),
7|40, bo Gamma(a,b),
f(a,b)-
T(a)
For a pair of random variables (X,T), if X\T ~ N{µ, (AT)¯¹) and T ~
Gamma(a, b), then (X,T) follows a normal-gamma distribution with parame-
ters (u, A,a,d). The joint probability density function of (X, T) has the form
te" exp
x²(x = μ)²).
1(z. t|p. A, a. b)-1(a) √27
For the Python programming questions, we set - 0,-10, a-4-10
and observations y(y) are stored in Qty.com
(a) Derive the joint prior (μ, 7) and likelihood function p/ya,7). Write down
the probability density function of the posterior distribution (r) (no
need to derive the exact distribution)
Japa+n
As +11
(b) In fact, for normal distributed data with unknown mean and precision
(inverse of variance), the normal-gamma prior is a conjugate prior and the
posterior (y) is also a normal-gamma distribution with parameters
ܤܞ
Ao +1.00 +
+ ²(x-8²+
2(x+1)
where
is the sample mean.
Derive the full conditional posterior distribution µly, and r. (need
to obtain the exact distribution)
(e) Write down the probability density function of the posterior predictive
distribution yly. Describe how to approximate ply"ly) via the simple
Monte Carlo approach.
1
(d) The mode of Normal-Gamma(, A, a., b) is (.). Consider the Laplace
approximation on the joint posterior (p. 5|B):
In (0) In (@Mary))-(0-0)¹ A(0-₁AP) - In 2(0|y).
A--VV In z(y)
where
(7).
Derive the approximated posterior distribution (u, ry). Draw two con-
tour plots of x(u, rµ) and (µ,5%) respectively in Python. (Python
scipy stats package does not provide direct functions to calculate the pdf
of the normal-gamma distribution. You need to calculate it by yourself)
(e) Assume a mean-field variational inference for the joint posterior (7) -
(u)(r). Find the optimal mean-field factor q, and g. Write down
the procedures to iteratively update the parameters of q and g; and
implement them in Python to obtain the estimated parameters of q; and
(set convergence criterion - 10-)
(Hints: q;(0) x exp{E, [ln P(D, ¤)])