Introduction to Neural Networks for C#, Session 3

Course Name	Introduction to Neural Networks for C#
Instructor	jeffheaton
Session Title	Using a Hopfield Neural Network
Session Number	3

Session Material

Introducing the Hopfield Neural Network

Hopfield neural networks are a very simple sort of neural network. They consist of a single layer of fully connected neurons. These neurons are connected to each other, but not to themselves. Hopfield neural networks can be trained to recognize patterns. You can see a hopfield neural network here.

Using a Hopfield Neural Network consists of the following two activities:

Training the Hopfield Neural Network
Recalling Patterns with the Hopfield Neural Network

Training the Hopfield neural network involves using several matrix manipulations that produce a weight matrix that will recognize the desired patterns.

Recalling a pattern from the Hopfield neural network involves using several matrix operations, particularly the dot product.

Programming a Hopfield Neural Network

The book provides the HopfieldNetwork class to implement a Hopfield neural network. This class defines the following methods and properties.

Property Name	Purpose
Matrix	Accesses the neural network's weight matrix.
Size	Gets the size of the neural network.

Methods defined by the HopfieldNetwork class:

Method Name	Purpose
Present	Presents a pattern to the neural network.
Train	Trains the neural network on a pattern.

Hopfield Network Examples

There are three different example programs provided. They are listed as follows.

Simple console example
Application that shows the weight matrix
Pattern recognition GUI application

All three of these applications make use of the HopfieldNetwork class.

Learning Methods and Error Calculation

Unsupervised training means that no anticipated outputs are provided for the sample input. The following flowchart demonstrates a simple unsupervised training algorithm.

Supervised training means that anticipated outputs are provided for the sample input. The following flowchart demonstrates a simple supervised training algorithm.

Hebb's Rule

Hebb's rule is a training method that reinforces what the neural network already knows. Hebb's rule is a form of unsupervised training.

Delta Rule

The delta rule is a training method that trains the neural network to produce the desired output from the specified input. The delta rule is a form of supervised training.

Videos for this Session

Video	Title
	Introduction to Neural Networks for C# (Class 3/16, Part 1/5)
	Introduction to Neural Networks for C# (Class 3/16, Part 2/5)
	Introduction to Neural Networks for C# (Class 3/16, Part 3/5)
	Introduction to Neural Networks for C# (Class 3/16, Part 4/5)
	Introduction to Neural Networks for C# (Class 3/16, Part 5/5)