Search for question
Question

PROBLEM 2 2. Consider minimizing the function g(x) = x² = 2ax + \|x|~ - where > 0 and 0 < y < 1. (This problem arises, in a somewhat

more complicated form, in shrinkage estimation in regression.) The function |x| has a “cusp” at 0, which mean that if \ is sufficient large then g is minimized at x = = 0. (a) g is minimized at x = 0 if, and only if, -Y 2 2 - 2y > > Υ 2 γ (3) Otherwise, g is minimized at x* satisfying g′(x*) = 0. Using R, compare the following two iterative algorithms for computing (when condition (3) does not hold): x* (i) Set x0 = α and define xk = α- k = 1, 2, 3,... 2 xk-1 (ii) The Newton-Raphson algorithm with x0 = α. Use different values of a, Y, and \ to test these algorithms. Which algorithm is faster? (b) Functions like g arise in so-called bridge estimation in linear regression (which are gener- alizations of the LASSO) – such estimation combines the features of ridge regression (which 2 shrinks least squares estimates towards 0) and model selection methods (which produce ex- act 0 estimates for some or all parameters). Bridge estimates ß minimize (for some y > 0 and > 0), n Р Σy - κ β)2 + λ Σβ;1. i=1 j=1 (4) See the paper by Huang, Horowitz and Ma (2008) (“Asymptotic properties of bridge esti- mators in sparse high-dimensional regression models” Annals of Statistics. 36, 587–613) for details. Describe how the algorithms in part (a) could be used to define a coordinate descent algorithm to find ẞ minimizing (4) iteratively one parameter at a time. (c) Prove that g is minimized at 0 if, and only if, condition (3) in part (a) holds. (Hint: Define h(t) = g(at) and show that h'(t) < 0 for t < 0 and h'(t) > 0 for t > 1 so that h(t) is minimized for some t = [0,1].)