Neural network

    Error calculation is an important aspect of any neural network. Whether the neural network is supervised or unsupervised, an error rate must be calculated. The goal of virtually all training algorithms is to minimize the rate of error. In this section, we will examine how the rate of error is calculated for a supervised neural network. We will also discuss how the rate of error is determined for an unsupervised training algorithm. We will begin this section by examining two error calculation steps used for supervised training.

    Training is a very important process for a neural network. There are two forms of training that can be employed, supervised and unsupervised. Supervised training involves providing the neural network with training sets and the anticipated output. In unsupervised training, the neural network is also provided with training sets, but not with anticipated outputs. In this book, we will examine both supervised and unsupervised training. This chapter will provide a brief introduction to each approach. They will then be covered in much greater detail in later chapters.

    There are many different ways that a neural network can learn; however, every learning algorithm involves the modification of the weight matrix, which holds the weights for the connections between the neurons. In this chapter, we will examine some of the more popular methods used to adjust these weights. In chapter 5, “The Feedforward Backpropagation Neural Network,” we will follow up this discussion with an introduction to the backpropagation method of training. Backpropagation is one of the most common neural network training methods used today.

    1. A typical Hopfield neural network contains six neurons. How many connections will this produce?

    2. Convert the following matrix from binary to bipolar.

    3. Convert the following matrix from bipolar to binary.

    4. Consider a four-neuron Hopfield neural network with the following weight matrix.

     What output will an input of 1101 produce?

    Activation Function

    Autoassociation

    Bipolar

    Hopfield neural network

    Single layer neural network

    Neural networks are one of the most commonly used concepts in Artificial Intelligence. Neural networks are particularly useful for recognizing patterns. They are able to recognize something even when it is distorted.

    A neural network may have input, output, and hidden layers. The input and output layers are the only required layers. The input and output layers may be the same neurons. Neural networks are typically presented input patterns that will produce some output pattern.

    This second example is essentially the same as the first; however, this example uses an applet. Therefore, it has a GUI that allows the user to interact with it. The user interface for this program can be seen in Figure 3.2.

Figure 3.2: A Hopfield Applet (HopfieldApplet.java)

    Now you will see how to make use of the HopfieldNetwork class that was created in the last section. The first example implements a simple console application that demonstrates basic pattern recognition. The second example graphically displays the weight matrix using a Java applet. Finally, the third example uses a Java applet to illustrate how a Hopfield neural network can be used to recognize a grid pattern.

    The first example, which is a simple console application, is shown in Listing 3.2.

Listing 3.2: Simple Console Example (ConsoleHopfield.java)

    The Hopfield neural network is implemented using two classes. The first class, called HopfieldNetwork is the main class that performs training and pattern recognition. This class relies on the Matrix and MatrixMath classes, introduced in chapter 2, to work with the neural network's weight matrix. The second class, called HopfieldException, is an exception that is raised when an error occurs while processing the Hopfield network. This is usually triggered as a result of bad input.

    You will now be shown exactly how a neural network is used to recall patterns. We will begin by presenting 0101 to the Hopfield network. To do this, we present each input neuron, which in this case are also the output neurons, with the pattern. Each neuron will activate based upon the input pattern. For example, when Neuron 1 is presented with 0101, its activation will result in the sum of all weights that have a 1 in the input pattern. For example, we can see from Table 3.2 that Neuron 1 has connections to the other neurons with the following weights: