Get the entire book!

In this section you will see how to use a feed
forward backpropagation neural network from a Java program. In this chapter we
will see mainly how the program is structured. In Chapter 5 we will examine the
internals of how the feed forward backpropagation neural network was
implemented. In the next section you will be introduced the neural network class
that will be used in this book. Then in the following section you will be shown
how a simple example can be created that demonstrates the neural network class
solve the XOR problem. We begin by examining the neural network class.

Using the Network Class

All programs in this book that make use of the
feedforward neural network will make use of the Neural.java class. This class
implements a feedforward neural network with an input, hidden and output layer.
If you wish to use two hidden layers you must use the class NeuralDual.java,
which will be covered in Chapter 11. Most examples in this book will use only a
single layer of hidden neurons.

There are several methods that make up the
Network class. In the next sections we will examine each of these and see how
they are called.

The Network Constructor

  Like most classes in Java the network class
makes available a specialized constructor that will accept some basic
configuration information about how the neural network is to be constructed. The
signature for the neural network constructor is shown below.

  public Network(int inputCount,

int hiddenCount,

int outputCount,

double learnRate,

double momentum)

As you can see from the above constructor several
configuration parameters must be passed to the neural network class as it is
instanciated. These parameters are summarized as follows.

• inputCount – The number of neurons that are in
  the input layer.
• hiddenCount  - The number of neurons that are in
  the hidden layer.
• outputCount – The number of neurons that are in
  the output layer.
• learnRate – The learning rate for the
  backpropagation training algorithm. This parameter will be discussed in much
  greater detail in Chapter 5.
• momentum – The momentum for the backpropagation
  training algorithm. This parameter will be discussed in much greater detail in
  Chapter 5.

The input and output layers are determined by how you
represent your problem as a neural network. There are many ways to determine
this. Through this book you will be presented with problems that will be adapted
to neural networks. Chapter 14 focuses primarily on how to prepare data for
neural networks and process the results.

Correctly choosing the number of hidden neurons
was discussed earlier in this chapter. In addition, Chapter 11 will show you how
pruning algorithms can be used to decrease the neurons in the hidden layer.

  The learning rate and momentum are both
backpropagation algorithms that will affect the learning process of the neural
network. These two parameters will be covered in much greater detail in Chapter
5. In general both of these values should be set to values that are greater than
zero, but also less than 1.

  Once the neural network has been instantiated,
using the constructor, it is basically ready for use. At this point the neural
network has a random weight matrix and should be trained before any patterns are
to be recalled from the neural network. In the next section you will see how the
neural network is used to recall a pattern.

The Output Computation Method

The output computation method is allows you to
present a pattern to the input layer and receive a pattern back from the output
layer. Both the input and output patterns are passed using an array of doubles.
The signature for the output computation method is shown here.

public double []computeOutputs(double input[])

As you can see an array of doubles is passed to the output
computation method. The size of this array must correspond to the number of
input neurons for the neural network. An array will be returned from this
method. The array that is returned will have one element that represents the
output of each of the output neurons. The following code shows how you would
present a simple two number pattern to the output computation method and display
the result.

double input[] = { 0.0, 1.0 };
double output[];

output = neural.computeOutputs(input);

for(int i=0;i<output.length;i++)
{
  System.out.println( output[i] );
}

As you can see the above code passes the input array to the
output computation method and receives the output from the neural network. The
output from this method may not match the desired output. To calculate the
amount of error in the output of the neural network you will use the error
calculation methods. The error calculation methods will be discussed in the next
section.

The Error Calculation Method

The error calculation methods allow you to determine the
amount of variance between the expected and actual output of a neural network.
In the next section you will be shown how to train the neural network. Calling
the error calculation methods is the first step in training the neural network.
Once the error has been determined the network can be trained so that the next
time the error will likely be lower.

There are two methods that are used for error calculation.
The first is the error calculation method, which is called calcError. The
calcError's signature is shown here.

public void calcError(double ideal[]) 

This method is to be called after each output set is
presented during a training process. This method does not make any modifications
to the weight matrix. Yet this method does store the "deltas" needed to modify
the weight matrix so that the neural network will produce the ideal output
better next time. To actually implement these changes you must call the learn
method of the neural network. The learn method will be covered in the next
section.

You may wish to calculate the root mean-square (RMS) error
for a complete set of training data. Once you have submitted a number of a
complete training set through the learning process you can call the "getError"
method to calculate the average error for the entire set. The signature for the
"getError" method is shown below.

public double getError(int len)

Now that you have seen how to calculate errors you are
ready to see the methods that are used to train the neural network. These
methods are covered in the next section.

The Training and Resetting Methods

Once you have obtained the neural network output and
calculated the error for an input pattern, you can train the neural network to
better recognize this pattern next time. This is done very simply calling the
learn method. Before you call the learn method you must insure that you have
calculated the output and errors. The signature for the learn method is shown
below.

public void learn()

The neural network begins with a completely random weight
matrix. You may wish to completely reset the neural network back to this state.
This can be done at any time by calling the reset method. The signature for the
reset method is shown here.

public void reset()

As you can see you must use the output calculation, error
calculation and training methods in s specific order. The following code segment
shows how this would be done.

 
double input[] = {
 { 0,0 },
 { 0,1 },
 { 1,0 },
 { 1,1 } };

double ideal[] = { {0}, {1}, {1}, {0 }};

for( int i=0;i<input.length; i++)
{
  neural.calcOutputs( input[i] );
  neural.calcError(ideal[i]);
  neural.learn();
}

System.out.println("Total error: " + 
neural.getError(input.length) );

  As you can see from the above code the training
set used is for the XOR operator. This training set is then presented to the "calcOutputs"
method. Though the "calcOutputs" calculates the output for the neural network,
these outputs are discarded as they are not needed.  At this point we are only
calling the "calcOutputs" method to prepare to call the "calcError" method. 
This is because at this point we are simply training and we do not care about
the actual and outputs of the neural network.  Once the "calcOutputs" method has
been called the "calcError" method is called.  The ideal outputs, which are the
outputs we expected from the neural network, are passed into the "calcError"
method.  The "calcError" method will then determine how close the actual outputs
from the neural network match these ideal outputs.  With this all complete, we
can finally we call the "learn" method of the neural network.  The "learn"
method will make any adjustments to the weight matrix to allow an in network to
better recognize this training set and the ideal out and produce the a deal
outputs. 

The above code loops through all four of the possibilities
for the XOR problem.  Just as the human often does not learn to a new skill by
Trident only once so to the neural network learns through repetition.  The
example program that you will see in the next section runs through a similar
look up to 10,000 times to properly train the neural network.  Generally you'll
write neural network programs to evaluate the overall error and continue looping
through the training set so long as the error is above a desired amount, such as
10 percent. 

Activation Functions

You may recall from earlier in the chapter that JOONE
allowed you to choose different neuron layer types. These layer types differed
in the sort of threshold method that they used. By default the neural network
class uses a sigmoid activation method. This activation method is shown below.

public double threshold(double sum) {
  return 1.0 / (1 + Math.exp(-1.0 * sum));
}

You may wish to use other types of activation methods. To
do this you must override the neural network class and provide a new threshold
method. Appendix E lists some other equations you may wish to use as an
activation method.