3. In the class, we saw logistic regression without any regularization. This problem is a

variant that introduces regularization.

Given a training set {(x(0,r)} for a K-class classification problem where each element

of the class indicator vector ris defined as follows:

[ 1 if x € C₂

otherwise

+={1

We use the softmax function to compute the posterior class probabilities for a single-layer

perceptron as follows:

P(G₁|x)=0

exp(wx(0)

Σ exp(wx(0)

The likelihood function of the weights (w), is given by:

NK

L({w}) - (x)

8=16-1

(a) Give the cross-entropy error function E({w;}) that can be used for learning the

perceptron based on the batch learning approach.

(b) Instead of minimizing E({w}), suppose we minimize another error function defined

as follows:

E({w})=E({w})+Σ.².

where a > 0 is some constant. How will the second term of E({w;}) affect the learning

result?

(a) 10 points

(b) 10 points

(c) We use simple gradient descent for batch learning based on minimizing the error

function E({w}). How will the weight update rule for each element wij of we be

affected by the second term of E({w;})?

Hints: Recall that "Lec 05: Linear Discrimination", Slide # 29, we defined cross-entropy

error as the negative log-likelihood over all N training samples.