Lasso

What is Lasso?

Lasso (Least Absolute Shrinkage and Selection Operator), also known as LASSO, is a regression analysis technique generally used in machine learning and statistics to improve prediction accuracy. This method performs both variable selection and regularization.

Variable or feature selection is a technique that entails removing redundant or irrelevant features, without losing important information. It should be distinguished from feature extraction, which consists of creating new features based on the original features. Regularization is the process of adding information in order to prevent overfitting or to redefine a badly specified problem.

Lasso was introduced in geophysics literature in 1986. Later Rober Tibshirani popularized it and coined its name in 1996.

Lasso was originally defined for linear regression and later extended to other statistical models, such as the generalized linear model, the generalized estimating equation model, proportional hazards models, and M-estimators.

In general, lasso consists of quadratic programming problems, which are usually solved via computational techniques. For example, the Python library sklearn considers the following equation:

where the first element of the equation is called the lost function, α is a constant, W₀ is the intercept and W_i are the coefficients of the estimated equation

Why is Lasso Important?

Lasso uses an L1 regularization that adds a penalty defined by the absolute value of the coefficients. The advantage of this technique is that it eliminates some coefficients, as large penalties can result in coefficients close to zero. This also makes Lasso easier to understand than methods, such as Ridge regression, which use L2 regularization.

Lasso + LogicPlum

Lasso is one of the many methods used to model phenomena. Although simple, its application requires heavy computation. Besides, the best way to know which method performs best is to try them all. This is where a tool like LogicPlum’s platform has an important advantage. Its automation capacity permits its users to find the best performing algorithm in a very short time, after comparing all possibilities according to a selected metric.

Additional Resources

For those wanting to implement Lasso in Python:

TutorialsPoint. (2020). LASSO (Least Absolute Shrinkage and Selection Operator) Available at https://www.tutorialspoint.com/scikit_learn/scikit_learn_lasso.htm