Search for question
Question

1. Question 1 (Bayesian inference, variational inference and sampling)

Let y(y) be Lid. samples from the normal distribution N(µ, 7−¹).

We specify normal-gamma prior distributions on and r

A|TAN (Aer)-¹),

7|40, bo Gamma(a,b),

f(a,b)-

T(a)

For a pair of random variables (X,T), if X\T ~ N{µ, (AT)¯¹) and T ~

Gamma(a, b), then (X,T) follows a normal-gamma distribution with parame-

ters (u, A,a,d). The joint probability density function of (X, T) has the form

te" exp

x²(x = μ)²).

1(z. t|p. A, a. b)-1(a) √27

For the Python programming questions, we set - 0,-10, a-4-10

and observations y(y) are stored in Qty.com

(a) Derive the joint prior (μ, 7) and likelihood function p/ya,7). Write down

the probability density function of the posterior distribution (r) (no

need to derive the exact distribution)

Japa+n

As +11

(b) In fact, for normal distributed data with unknown mean and precision

(inverse of variance), the normal-gamma prior is a conjugate prior and the

posterior (y) is also a normal-gamma distribution with parameters

ܤܞ

Ao +1.00 +

+ ²(x-8²+

2(x+1)

where

is the sample mean.

Derive the full conditional posterior distribution µly, and r. (need

to obtain the exact distribution)

(e) Write down the probability density function of the posterior predictive

distribution yly. Describe how to approximate ply"ly) via the simple

Monte Carlo approach.

1

(d) The mode of Normal-Gamma(, A, a., b) is (.). Consider the Laplace

approximation on the joint posterior (p. 5|B):

In (0) In (@Mary))-(0-0)¹ A(0-₁AP) - In 2(0|y).

A--VV In z(y)

where

(7).

Derive the approximated posterior distribution (u, ry). Draw two con-

tour plots of x(u, rµ) and (µ,5%) respectively in Python. (Python

scipy stats package does not provide direct functions to calculate the pdf

of the normal-gamma distribution. You need to calculate it by yourself)

(e) Assume a mean-field variational inference for the joint posterior (7) -

(u)(r). Find the optimal mean-field factor q, and g. Write down

the procedures to iteratively update the parameters of q and g; and

implement them in Python to obtain the estimated parameters of q; and

(set convergence criterion - 10-)

(Hints: q;(0) x exp{E, [ln P(D, ¤)])

Fig: 1