Examining the Feedforward Process

    Earlier in this chapter you saw how to present data to a neural network and train that neural network. The remainder of this chapter will focus on how these operations were performed. The neural network classes presented earlier are not overly complex. We will begin by exploring how they calculate the output of the neural network. This is called the feedforward process.

Calculating the Output Mathematically

    Equation 5.4 describes how the output of a single neuron can be calculated.

Equation 5.4: Feedforward Calculations

     The above equation takes input values named x, and multiplies them by the weight w. As you will recall from chapter 2, the last value in the weight matrix is the threshold. This threshold is wn.

    To perform the above operation with matrix mathematics, the input is used to populate a matrix and a row is added, the elements of which are all ones. This value will be multiplied against the threshold value. For example, if the input were 0.1, 0.2, and 0.3, the following input matrix would be produced.

Equation 5.5: An Input Matrix

    The dot product would then be taken between the input matrix and the weight matrix. This number would then be fed to the activation function to produce the output from the neuron.

Calculating the Output for a Feedforward Neural Network

    To obtain the output from the neural network, the computeOutputs function should be called. This function will call the layers of the neural network and determine the output. The signature for the computeOutputs function is shown here:

public double[] computeOutputs(final double input[]) 

    The first thing that the computeOutputs function does is to check and see if the input array is of the correct size. The size of the input array must match the number of input neurons in the input layer.

if (input.length != this.inputLayer.getNeuronCount()) { 
  throw new NeuralNetworkError(
  "Size mismatch: Can't compute outputs for input size="  
  + input.length 
  + " for input layer size=" 						
  + this.inputLayer.getNeuronCount());
} 		

    Next, each of the FeedforwardLayer objects will be looped across.

for (final FeedforwardLayer layer : this.layers) { 

    The input layer will receive the input that was provided.

  if (layer.isInput()) { 				
    layer.computeOutputs(input); 			

    Each of the hidden layers will receive the output from the previous layer as input.

  } else if (layer.isHidden()) { 				
    layer.computeOutputs(null); 			
  } 		
} 		

    Finally, the output layer's result is returned.

return (this.outputLayer.getFire()); 	

    The computeOutputs method for the neural network obtains its output by calculating the output for each layer. The next section will explain how the output for a layer is calculated.

Calculating the Output for a Layer

    The FeedforwardLayer class contains a computeOutputs method as well. This method is used to calculate the output for each of the layers. The signature for this method is shown here:

public double[] computeOutputs(final double pattern[]) 

    First, the computeOutputs method checks to see whether or not a pattern was presented to it.

  if (pattern != null) { 			

    If a pattern was presented, then the pattern should be copied to the fire instance variable.

  for (int i = 0; i < getNeuronCount(); i++) { 				
    setFire(i, pattern[i]); 			
  } 		
} 		

    If a pattern was not presented, then the fire instance variable will have already been set by the previous layer.

    final Matrix inputMatrix = 
       createInputMatrix(this.fire);

    Next, an output value must be calculated to become the input value for each of the neurons in the next layer.

    for (i = 0; i < this.next.getNeuronCount(); i++) { 		

    To do this, we obtain a column matrix for each of the neurons in the next layer.

      final Matrix col = this.matrix.getCol(i); 			

    The dot product between the weight matrix column and input is then determined. This will be the value that will be presented to the next layer.

      final double sum = MatrixMath.dotProduct(col, inputMatrix); 

    The output is passed to the activationFunction and then stored in the fire instance variable of the next layer.

      this.next.setFire(i,     
      this.activationFunction.activationFunction(sum));
  } 		

    The fire instance variable is returned as the output for this layer.

return this.fire; 	

    This process continues with each layer. The output from the output layer is the output from the neural network.