MATH 30022 PROJECT (SEMESTER TWO) Exploring the interface between statistics and machine learning
The basic components of computer code are the implementations of statistical machine learning methods and CNN models, they are both repeated with all the three datasets. Pre-processing is done as preperation steps before implementing any specific machine learning methods, it involves: 1. import the datasets (MNIST and CIFAR-10 are loaded directly from the build-in datasets with Keras, Fruits 360 dataset is imported by ImageDataGenerator that converts image files into batches of processed tensors); 2. normalize the range of pixel values into the range of (0, 1); 3. convert all the classification labels into the form of class matrices (for example, [9] → [0, 0, 0, 0, 0, 0, 0, 0, 0, 1]).
For the implementations of statistical machine learning methods, dimensionality reduction tech- nique (PCA or autoencoder) is another pre-processing steps, for the purpose of comparison (it is pos- sible expect Fruits 360 dataset due to computational complexity). Afterwards, random forest, SVM, logistic regression and LDA are applied to the reshaped training data (with/without dimensionality reduction) respectively.
However, dimensionality reduction techniques are omitted with the implementations of CNN mod- els due to the features of convolutional layers. Firstly, the classification accuracies are examined on basic CNN models with fine-tunning to adapt each specific dataset, and the basic CNN model on CIFAR-10 is repeated with or without batch normalization to confirm its importance. Then, the per- formance of pretrained model in improving the accuracy is researched with VGGNet and ResNet (both imported from Keras directly). During the training, the weights of convolutional basis are frozen to aviod modification which is caused by backpropagation from densely connected layers.