Get the entire book!

The error of a neural network is a very important statistic to be used as a part of the training process. This chapter showed you how to calculate the output error for an individual training set element, as well as who to calculate the RMS error for the entire training set.

Layers are collections of related neurons. JOONE supports several layer types. In this chapter we examined the sigmoid layer, which can handle only positive output. We also examined the hyperbolic tangent layer which can handle both positive and negative outputs. Lastly, we examined the linear layer, which performs no modification of the input at all. The only memory maintained by the layer is “layer bias” which specifies a number that can be added to the input for each neuron.

Synapses form the connection between layers. Synapses maintain individual weights for each neuron that they connect. When you connect two layers with a synapses, each neuron is automatically connected to the neuron in the other layer. These connection weights make up the memory of the neural network. These weights determine the degree to which the neuron will pass information on to the connected neuron.

Training occurs when the weights of the synapse are modified to produce a more suitable output. Unsupervised training occurs when the neural network is left to determine the correct responses. Supervised training occurs when the neural network is provided with training data and anticipated outputs. Hebb’s rule can be used for supervised training. The delta rule is used for unsupervised training.

In this chapter we learned how a machine learns by modifying the connection weights between the neurons. This chapter only introduced the basic concepts of how a machine learns. In the next chapter we will explore back propagation and see how the neural network class implements it. We will also create an example of an artificial mouse that is trained to run a maze.