On Some New Hybridized Regression Estimation and Feature Selection Techniques

Conventional regularization techniques like LASSO, SCAD and MCP have been shown to perform poorly in the presence of extremely large or ultra-high dimensional covariates. This has created the need for and led to the development and reliance on filtering technique like screening. Screening techniques (such as SIS, DC-SIS, and DC – RoSIS) have been shown to reduce the computational complexity in selecting important covariates from ultrahigh dimensional candidates. To this end, there have been various attempts to hybridize the conventional regularization techniques. In this paper, we combine some regularization techniques (LASSO and SCAD) with a screening technique (DC – RoSIS) to form new hybrid methods with a view to achieving better dimension reduction and variable selection simultaneously. Extensive simulation results and real life data performance show that the proposed methods perform better than the conventional methods.

Keywords:- Regularization Techniques, Screening Technique, LASSO DC–RoSIS, SCAD DC – RoSIS.