CMSC 373 Artificial Intelligence
Fall 2023

Assignment 3: Recognizing Handwritten Digits using Backpropagation
(Due by start of class on Monday, November 13, 2023)

In this assignment, you will build a 3-layer backpropagation model to train it to classify handwritten digits. The dataset we will use is the MNIST Digits Dataset. That is available in Keras.
Make sure you have a Google account and are able to navigate to the Google Colab page: https://colab.research.google.com

Part 1: Building and Training the Model

Go to the link:https://colab.research.google.com/drive/1PYv6_a3bv9Nlw0H5Upwt_2ToFAkjDNfw?usp=sharing
First, create a copy in your own Drive by going to the File menu and selecting: Save a copy in drive.
Work through the entire workbook. Try some things of your own (this is why you saved a copy above).
Once done, try the following at the end:

1. Modify the model to use the Sigmoid ("sigmoid") activations in hidden layer, SGD optimizer ("sgd"), and "mean_squared_error" loss. How well does the model learn?
2. Modify the model to use Relu ("relu") along with SGD optimizer ("sgd"), and "mean_squared_error" loss. How well does the model learn?
3. Modify the model to use Relu ("relu") along with a RMSProp optimizer ("rmsprop") and "mean_squared_error" loss. How well does the model learn?
4. Modify the model to use Relu ("relu") along with a RMSProp optimizer ("rmsprop") and "sparse_categorical_crossentropy" loss. How well does the model learn?
5. For #4 above try changing the model design (use fewer or more hidden units). How well does the model learn?

The testing and validation Lab is available at: https://colab.research.google.com/drive/1c1YFdYZjEiCneHu0XubFdlMABUd0cUoE

Write a report (1-2 pages) on your observations for 1-5 above, followed by an analysis and reflection.

Extra Credit: Try another model - The MNIST Fashion Dataset

Go to the link: MNIST Fashion Dataset

Fashion MNIST is a dataset of Zolando's article images (see picture above) with a training set and image size exactly like the MNIST Digits dataset (60,000 training images, 10,000 testing images, each image of size 28x28 pixels, and ten labels). Each training and test sample is assigned one of the following labels (0,9), and names:

• 0 T-shirt/top
• 1 Trouser
• 2 Pullover
• 3 Dress
• 4 Coat
• 5 Sandal
• 6 Shirt
• 7 Sneaker
• 8 Bag
• 9 Ankle boot

Design your own model to train the Fashion Dataset. Once you have arrived at a good accuracy, stop. Use Relu activations for hidden layer, Softmax for output layer, Adam ("adam" or RMSProp ("rmsprop") optimizer, Sparse Categorical Cross Entropy ("sparse_categorical_crossentropy") loss, and Accuracy (% correct) metric ("accuracy").

Further, split the Testing Dataset into a 5000 image Validation Dataset and 5000 image Testing Dataset. Use the Validation Dataset as a way of exploring the hyperparameters. Try and achieve an accuracy of over 90% (higher than that the better). Once you have the hyperparameters determined, train the model on the Training Dataset, using Validation Dataset during training (as described in class), and then test the performance on the Testing Dataset. Does the model underfit? overfit? Estimate the robus fit by determining the number of epochs to train. Test again on the Testing Dataset.

Write a report (3-4 pages) describing your model, the choices you made, how you arrived at them, and finally your models performance, followed by a reflection.|

What to Hand In
Hand in a report containing the following:

• Your observations on 1-5 of Part 1. Write a an analysis and reflection based on your results (1-2 pages)
• Describe/draw your model, all its parameters used. And, the accuracy obtained.
• An overall reflection on the exercise.
• A report on the Extra Credit part as described above.
  
  Hand in a PDF by e-mail (named: LastFirst.pdf) by the due date.

Back to CMSC 373 Page.