Introduction to Neural Networks for Java, Session 3

Course Name	Introduction to Neural Networks for Java
Instructor	jeffheaton
Session Title	Hopfield Neural Network, Delta & Hebb's Rule
Session Number	3

Session Material

Introducing the Hopfield Neural Network

Hopfield neural networks are a very simple sort of neural network. They consist of a single layer of fully connected neurons. These neurons are connected to each other, but not to themselves. Hopfield neural networks can be trained to recognize patterns. You can see a hopfield neural network here.

Using a Hopfield Neural Network consists of the following two activities:

Training the Hopfield Neural Network
Recalling Patterns with the Hopfield Neural Network

Training the Hopfield neural network involves using several matrix manipulations that produce a weight matrix that will recognize the desired patterns.

Recalling a pattern from the Hopfield neural network involves using several matrix operations, particularly the dot product.

Programming a Hopfield Neural Network

The book provides the HopfieldNetwork class to implement a Hopfield neural network. This class defines the following methods.

Method Name	Purpose
getMatrix	Accesses the neural network's weight matrix.
getSize	Gets the size of the neural network.
present	Presents a pattern to the neural network.
train	Trains the neural network on a pattern.

Hopfield Network Examples

There are three different example programs provided. They are listed as follows.

Simple console example
Application that shows the weight matrix
Pattern recognition GUI application

All three of these applications make use of the HopfieldNetwork class.

Learning Methods and Error Calculation

Unsupervised training means that no anticipated outputs are provided for the sample input. The following flowchart demonstrates a simple unsupervised training algorithm.

Supervised training means that anticipated outputs are provided for the sample input. The following flowchart demonstrates a simple supervised training algorithm.

Hebb's Rule

Hebb's rule is a training method that reinforces what the neural network already knows. Hebb's rule is a form of unsupervised training.

Delta Rule

The delta rule is a training method that trains the neural network to produce the desired output from the specified input. The delta rule is a form of supervised training.

Videos for this Session

Video	Title
	Introduction to Neural Networks for Java(Class 3/16, Part 1/5)
	Introduction to Neural Networks for Java(Class 3/16, Part 2/5)
	Introduction to Neural Networks for Java(Class 3/16, Part 3/5)
	Introduction to Neural Networks for Java(Class 3/16, Part 4/5)
	Introduction to Neural Networks for Java(Class 3/16, Part 5/5)