Variable selection and forecasting via automated methods for linear models: LASSO/adaLASSO and Autometrics

In this article we compare two approaches of model selection methods for linear regression models: classical approach—Autometrics (automatic general-to-specific selection)—and statistical learning—LASSO ((Formula presented.) -norm regularization) and adaLASSO (adaptive LASSO). In a simulation experiment, considering a simple setup with orthogonal candidate variables and independent data, we compare the performance of the methods concerning predictive power (out-of-sample forecast), selection of the correct model (variable selection) and parameter estimation. The case where the number of candidate variables exceeds the number of observation is considered as well. Finally, in an application using genomic data from a high-throughput experiment we compare the predictive power of the methods to predict epidermal thickness in psoriatic patients, and we perform a simulation experiment with correlated variables, based on the application.