The Hopfield Neural Network

    The Hopfield neural network is perhaps the simplest type of neural network. The Hopfield neural network is a fully connected single layer, autoassociative network. This means it has a single layer in which each neuron is connected to every other neuron. Autoassociative means that if the neural network recognizes a pattern, it will return that pattern.

    In this chapter we will examine a Hopfield neural network with just four neurons. It is small enough to be easily understood, yet it can recognize a few patterns. A Hopfield network, with connections, is shown in Figure 3.1.

Figure 3.1: A Hopfield neural network with 12 connections.

    We will build an example program that creates the Hopfield network shown in Figure 3.1. Since every neuron in a Hopfield neural network is connected to every other neuron, you might assume a four-neuron network contains 42 or 16 connections. However, 16 connections would require that every neuron be connected to itself, as well as to every other neuron. This is not the case in a Hopfield neural network, so the actual number of connections is 12.

    As we develop an example neural network program, we will store the connections in a matrix. Since each neuron in a Hopfield neural network is by definition connected to every other neuron, a two dimensional matrix works well. All neural network examples in this book will use some form of matrix to store their weights.

    Table 3.1 shows the layout of the matrix.

Table 3.1: Connections in a Hopfield Neural Network

    This matrix is called the weight matrix, and contains the weights associated with each connection. This matrix is the “memory” of the neural network, and will allow the neural network to recall certain patterns when they are presented. Many of the neural networks in this book will also contain threshold values, in addition to the weights. However, the Hopfield neural network contains only weights.

    For example, the values shown in Table 3.2 show the correct weight values to use to enable a network to recall the patterns 0101 and 1010. The method used to create a network with the values contained in Table 3.2 will be covered shortly. First, you will be shown how the values in Table 3.2 are used by the network to recall 0101 and 1010.

Table 3.2: Weights Used to Recall 0101 and 1010

Neuron 1 (N1) Neuron 2 (N2) Neuron 3 (N3) Neuron 4 (N4)

    Neuron 1 (N1) 0 -1 1 -1

    Neuron 2 (N2) -1 0 -1 1

    Neuron 3 (N3) 1 -1 0 -1

    Neuron 4 (N4) -1 1 -1 0