Implementing, Tuning and Data Curation of Feedforward Networks
Overview
The purpose of this assignment is to learn the underpinnings of feedforward networks.
You will implement a perceptron network and a single hidden layer feedforward network
with a step-activation function. You will experiment with key hyper parameters as
well as key activation functions so as to understand how they interact and how
they influence the performance of the networks. You will additionally learn about the
task of curating data so that it may be used to successfully train neural networks.
You will be working with code that implements perceptrons, feed-forward networks and
the backpropagation algorithm.
Basics
- Review the materials from the days 4 and 5.
- Throughout this assignment, complete the lab manual
- Here is a link to a zipped copy of all software and the lab manual
for this assignment.
- For this portion, work with Perceptron.java. It is a
perceptron network with a single output node. It uses a step activation function.
Currently the training data is set up to learn the Boolean and
function.
- Implement the trainNetwork() method with a fixed number
of ten training episodes.
- Complete the experiments for questions 1-5 in the lab manual and
enter your results.
XOR Experiments
- For this portion, work with XOR.java. It is a
feedforward network with one output node. It uses a sigmoid
activation function. The weights are initialized to random
values in the range [0..1[.
- Study the code and conduct experiments 6 and 7 from the lab manual.
Parity Bit
- Use the given ParityNet.java and TrainingParity.java
files to engineer a network that
calculates an even parity bit for 7
bits. Please notice that one can implement a 2-bit parity net with
XOR. A three bit parity bit and any n-bit parity net can be
implemented through a cascading net of XOR "circuits." Here is a paper
that has some figures with architectures for parity bit NN
architectures. Here is a figure for an XOR net that uses
just 1 and -1 for weights.
Data Curation and Digit Recognition
- For this portion, work with FeedForwardNetwork.java and Training.java. Study the code. Notice that "Training.java" is
set-up to train the feed-forward network to train XOR. It serves to test your
installation. Please notice that there
is some testing code in the "FeedForwardNetwork.java" file. Feel
free to modify it to your needs. Modify the
training file so that it successfully recognizes handwritten
digits. Here is the MNIST training data
An explanation of it can be found here.
- There are two portions to this assignment. At
first, you need to curate the data from the MNIST data set so that it
can be used for training purposes. We recommend that you:
- Learn about the byte level layout of an idx-file.
- Study which Java classes and functions you will be using to read
bytes from an idx-file.
- Format and arrange the bytes so that they
can be used to train the given neural network.
- Here are some hints for curating
the data.
- For the second part of this assignment, conduct and document the
experiments of question 8.
Submission
Please submit a zipped copy of the following items to the appropriate
drop-box on Moodle.
- Your report for the experiments your conducted.
- Your modified Perceptron.java, XOR.java,
ParityNet.java, TrainingParity.java and Training.java files