Application of Statistical Computing to Statistical Learning

This study focuses on supervised learning, an aspect of statistical learning. The supervised learning is concerned with prediction, and prediction problems are distinguished by the output predicted. The output of prediction is either a categorical or continuous variable. If the output is a categorical variable, we have classification otherwise what obtains is regression. We therefore identify classification and regression as two prediction tools.

We further identify many features commonly shared by these prediction tools, and as a result, opine that it may be possible to use a regression function in classification or vice versa. Thus, we direct our research towards classification,and intend to:
(i) Compare the differences and similarities between two main classifiers namely,
Fisher's Discriminant Analysis (FDA) and Support Vector Machine (SVM).
(ii) Introduce a regression based classification function, with acronym RDA (Regression
Discriminant Analysis).
(iii) Provide proof that RDA and FDA are identical.
(iv) Introduce other classification functions based on multiple regression variants (ridge regression and Lasso) namely, Lasso Discriminant Analysis (LaDA) and
Ridge Regression Discriminant Analysis (RRDA).

We further conduct experiments using real world datasets to verify if the error rates of RDA and FDA on the same datasets are identical or not. We also conduct similar experiments to verify if differences arising from the error rates of using LaDA, RRDA, FDA and Regularized Fisher's Discriminant Analysis (RFDA) on the same datasets are statistically different from each other or not. In the end, we explore benefits that may derive from the use of LaDA as a classifier, particularly in connection with variable selection.