Please answer the following questions in complete sentences in submit the solution on Blackboard by the due date there.
Please identify anyone, whether or not they are in the class, with whom you discussed your homework. This problem is worth 1 point, but on a multiplicative scale.
Show that is a convex function. Feel free to use the result proved on the last homework.
Show that the null-space of a matrix is a convex set.
The Ridge Regression problem is a variant of least squares: This is also known as Tikhonov regularization.
Show that this problem always has a unique solution, for any if , using the theory discussed in class so far.
This property is one aspect of the reason that Ridge Regression and used. It is also a common regularization method that can help avoid overfitting in a regression problem.
Use the SVD of to characterize the solution as a function of .
What is the solution when ?
What is the solution when .
Consider the constrained least squares problem: where , and rank .
Convert this problem into the standard constrained least squares form.
Form the augmented system from the Lagrangian as we did in class.
Manipulate this problem to arrive at the normal equations
for a least-squares problem: .
Discuss any advantages of the systems at intermediate steps.