Learning Methods
Training is a very important process of working with a neural network. As we saw in Chapter 1 there are two forms of training that can be employed with a neural network. Supervised training provides the neural network with training sets and the anticipated output. Unsupervised training supplies the neural network with training sets, but there is no anticipated output provided. In this book we will examine both supervised and unsupervised training. We will now briefly cover each type of training. Supervised and unsupervised training will be covered in much greater detail in later chapters of this book.
Unsupervised Training
Unsupervised training is a very common training technique for Kohonen neural networks. In Chapter 6 we will discuss how to construct a Kohonen neural network and the general process for training without supervision.
What is meant by training without supervision is that the neural network is provided with training sets, which are collections of defined input values. But the unsupervised neural network is not provided with anticipated outputs.
Unsupervised training is usually used in a classification neural network. A classification neural network takes input patterns, which are presented to the input neurons. These input patterns are then processed, and one single neuron on the output layer fires. This firing neuron can be thought of as the classification of which group the neural input pattern belonged to.
In Chapter 7 you will be shown a practical application of the Kohonen neural network. The example program presented in Chapter 7 learns by using the unsupervised training method. This example program is designed to be able to read handwriting. Handwriting recognition is a good application of a classification neural network.
The input patterns presented to the Kohonen neural network are the dot image of the character that was hand written. We may then have 26 output neurons, which correspond to the 26 letters of the English alphabet. The Kohonen neural network should classify the input pattern into one of the 26 input patterns.
During the training process the Kohonen neural network in Chapter 7 is presented with 26 input patterns. The network is configured to also have 26 output patterns. As the Kohonen neural network is trained the weights should be adjusted so that the input patterns are classified into the 26 output neurons. As you will see in Chapter 7, this technique results in a relatively effective method for character recognition.
Another common application for unsupervised training is data mining. In this case you have a large amount of data, but you do not often know exactly what you are looking for. You want the neural network to classify this data into several groups. You do not want to dictate, ahead of time, to the neural network which input pattern should be classified to which group. As the neural network trains the input patterns will fall into similar groups. This will allow you to see which input patterns were in common groups.
As you can see unsupervised training can be applied to a number of uses. Unsupervised training will be covered in much greater detail in Chapters 6 and 7.
Supervised Training
The supervised training method is similar to the unsupervised training method in that training sets are provided. Just as with unsupervised training these training sets specify input signals to the neural network. The neural network that was introduced in Chapter 3 used the supervised training method.
The primary difference between supervised and unsupervised training is that in supervised training the expected outputs are provided. This allows the supervised training algorithm to adjust the weight matrix based on the difference between the anticipated output of the neural network, and the actual output.
There are several popular training algorithms that make use of supervised training. One of the most common is the backpropagation algorithm. Back propagation will be discussed in Chapter 5. It is also possible to use an algorithm such as simulated annealing or a genetic algorithm to implement supervised training. Simulated annealing and genetic algorithms will be discussed in Chapters 8 through 10. We will now discuss how errors are calculated as a part of both the supervised and unsupervised training algorithms.
