Learning Methods

    Training is a very important process for a neural network. There are two forms of training that can be employed, supervised and unsupervised. Supervised training involves providing the neural network with training sets and the anticipated output. In unsupervised training, the neural network is also provided with training sets, but not with anticipated outputs. In this book, we will examine both supervised and unsupervised training. This chapter will provide a brief introduction to each approach. They will then be covered in much greater detail in later chapters.

Unsupervised Training

    What does it mean to train a neural network without supervision? As previously mentioned, the neural network is provided with training sets, which are collections of defined input values. The unsupervised neural network is not provided with anticipated outputs.

    Unsupervised training is typically used to train classification neural networks. A classification neural network receives input patterns, which are presented to the input neurons. These input patterns are then processed, causing a single neuron on the output layer to fire. This firing neuron provides the classification for the pattern and identifies to which group the pattern belongs.

    Another common application for unsupervised training is data mining. In this case, you have a large amount of data to be searched, but you may not know exactly what you are looking for. You want the neural network to classify this data into several groups. You do not want to dictate to the neural network ahead of time which input pattern should be classified into which group. As the neural network trains, the input patterns fall into groups with other inputs having similar characteristics. This allows you to see which input patterns share similarities.

    Unsupervised training is also a very common training technique for self-organizing maps (SOM), also called Kohonen neural networks. In chapter 11, we will discuss how to construct an SOM and introduce the general process for training them without supervision.

    In chapter 12, “OCR and the Self-Organizing Map,” you will be shown a practical application of an SOM. The example program presented in chapter 12, which is designed to read handwriting, learns through the use of an unsupervised training method. The input patterns presented to the SOM are dot images of handwritten characters and there are 26 output neurons, which correspond to the 26 letters of the English alphabet. As the SOM is trained, the weights are adjusted so input patterns can then be classified into these 26 groups. As will be demonstrated in chapter 12, this technique results in a relatively effective method for character recognition.

    As you can see, unsupervised training can be applied to a number of situations. It will be covered in much greater detail in chapters 11 and 12. Figure 4.1 illustrates the flow of information through an unsupervised training algorithm.

Figure 4.1: Unsupervised training.

Supervised Training

    The supervised training method is similar to the unsupervised training method, in that training sets are provided. Just as with unsupervised training, these training sets specify input signals to the neural network. The primary difference between supervised and unsupervised training is that in supervised training the expected outputs are provided. This allows the neural network to adjust the values in the weight matrix based on the differences between the anticipated output and the actual output.

    There are several popular supervised training algorithms. One of the most common is the backpropagation algorithm. Backpropagation will be discussed in the next chapter. It is also possible to use simulated annealing or genetic algorithms to implement supervised training. Simulated annealing and genetic algorithms will be discussed in chapters 6, “Training using a Genetic Algorithm,” and chapter 7, “Training using Simulated Annealing.” We will now discuss how errors are calculated for both supervised and unsupervised training algorithms.

    Figure 4.2 illustrates the flow of information through a supervised training algorithm.

Figure 4.2: Supervised training.