Society of Actuaries (SOA) PA Practice Exam

What does the lambda hyperparameter in LASSO control?

1. The number of principal components generated

2. The severity of penalization for smaller coefficients

3. The fit of the model to the training data

4. The complexity of the original dataset

The correct answer is: The severity of penalization for smaller coefficients

The lambda hyperparameter in LASSO (Least Absolute Shrinkage and Selection Operator) plays a critical role in determining the degree of shrinkage applied to the coefficients of the regression model. Specifically, it controls the severity of penalization for smaller coefficients. By increasing the value of lambda, the penalty imposed on the absolute values of the coefficients becomes stronger, which encourages the model to reduce less important feature coefficients to zero, effectively performing variable selection. This leads to a simpler model that may improve prediction accuracy and interpretability. In contrast, the other options pertain to different concepts. The number of principal components generated relates to Principal Component Analysis (PCA) rather than LASSO. The fit of the model to the training data is connected to how well the model predicts the training data but does not specifically address the role of lambda. The complexity of the original dataset might refer to the number of features or interactions but does not directly relate to the penalization aspect that lambda influences in a LASSO model. Thus, understanding that lambda specifically manages the trade-off between fitting the training data and promoting simpler, sparse models is crucial in realizing its importance in LASSO regression.