Classification-based Ridge

Regression analysis is a study on relationship among variables. The dependent variable is believed to be related with one or more explanatory variables. It is often aimed at estimating the mean value of the former in terms of the known or fixed value of the latter. The Ordinary Least Squares (OLS) Estimator is the most popularly used estimator to estimate the parameters of regression model. Under certain assumptions, the estimator has some very attractive statistical properties which have made it one of the most powerful estimators of regression model. One of the assumptions is that the explanatory variables are independent. However, in practice, there may be strong linear relationships among the explanatory variables. This problem is often referred as multicollinearity problem. It is well known that performance of OLS estimator is unsatisfactory in the presence of multicollinearity in that the regression coefficients possess large standard errors and some will even have the wrong sign (Gujarati, 1995). In literature, there are various existing methods to solve this problem. Among them is the ridge regression estimator first introduced by Hoerl and Kennard (1970).