Issue
Can someone give me an understandable explanation of the parameter Alpha
in SKlearn's Ridge Regression? How does it influence the function etc.?
Examples would be helpful :)
Solution
Ridge regression minimizes the objective function:
||y - Xw||^2_2 + alpha * ||w||^2_2
This model solves a regression model where the loss function is the linear least squares function and regularization is given by the l2-norm. In simple words, alpha
is a parameter of how much should ridge regression tries to prevent overfitting!
Let say you have three parameter W = [w1, w2, w3]
. In overfitting situation, the loss function can fit a model with W=[0.95, 0.001, 0.0004]
which means it is highly biased to the first parameter. However, alpha * ||w||^2_2
increases the loss function in those cases and tries to keep all parameters in some sort of boundaries to prevent overfitting. For instance, with a regularizer, the W
could be W=[0.5, 0.2, 0.33]
. When you increase alpha
you are pushing the Ridge regression to be more robust against overfitting, but might be getting larger training error.
Answered By - aminrd
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.