The Penalized Trimmed Least Squares Method for Robust Variable Selection in Multiple Linear Regression

Abstract

The large number of independent variables often causes problems in the accuracy of the multiple linear regression model. This has motivated researchers to find or select the best model using methods such as forward selection, backward elimination, and stepwise regression. However, these methods have become time-consuming and ineffective when dealing with high-dimensional data. Therefore, statistical literature has proposed the Lasso method and its variants to address these issues. Nevertheless, these methods are sensitive to outliers, which has led to the emergence of several robust studies—particularly focusing on robustifying penalized methods for high-dimensional data, in order to achieve effective variable selection and robust parameter estimation. Among these methods is the Trimmed Penalized Least Squares (RPLTS), which aims to make the Lasso approach more robust against outliers appearing in the dependent variable or in the residuals. However, this method remains sensitive to the presence of leverage points. Accordingly, this research aims to weight the RPLTS method to select the best subset of variables by reducing the influence of leverage points and improving the model’s accuracy. Simulations and real data were used to evaluate the efficiency of the proposed method and compare it with the previous method. The comparison was based on several statistical criteria such as variable selection accuracy, sensitivity to influential variables, specificity of non-influential variables, and mean squared error (MSE) of the model. The method that achieves the highest accuracy, sensitivity, and specificity rates, along with the lowest MSE, is considered superior. The analysis of both real and simulated data demonstrated that the proposed method outperforms the previous one in terms of robustness and efficiency.