Feedforward Neural Network

Feedforward network with bias and a hidden layer

The feedforward neural network, or perceptron, is a type of neural network first described by by Warren McCulloch and Walter Pitts in the 1940s. The feedforward neural network, and its variants, are the most widely used form of neural network. The connections are made in a forward direction. Typically each non-bias neuron is fully connected to the previous layer. However, it is also possible to have a network that includes Partial Connections.

The feedforward neural network is often trained with the backpropagation training technique, though there are other more advanced training techniques, such as resilient propagation. The feedforward neural network uses weighted connections from an input layer to zero or more hidden layers, and finally to an output layer. Feedforward Neural networks can be used for classification and regression. Feedforward networks remember what they learn by adjusting weights between the neurons.

Encog Factory Code

The following Encog Factory code can be used to create a feedforward neural network.

type: feedforward
architecture: ?:B->SIGMOID->4:B->SIGMOID->?

The above code creates a feedforward network with 4 hidden neurons and a variable number of input and output neurons. The input and output neuron counts will be defined when the Encog Factory is called. The above method makes use of a Sigmoid Activation Function. Other possible values include:

• SIGMOID
• HTAN
• LINEAR

Java Example

The following code creates a simple feedforward neural network for use with the XOR Operator. You should notice that the input layer has no activation function, but has bias. This is because there is no previous layer for the activation function to apply to. The output layer has an activation function and no bias. This is because there is no "next layer" for the bias to connect to. The activation function is applied to data leaving the layer. The bias neuron is connected to the next layer.

import org.encog.ml.data.MLDataSet;
import org.encog.ml.data.basic.BasicMLDataSet;
import org.encog.neural.activation.ActivationSigmoid;
import org.encog.neural.networks.BasicNetwork;
import org.encog.neural.networks.layers.BasicLayer;
import org.encog.util.simple.EncogUtility;
 
public class Test {
 
/**
 * The input necessary for XOR.
 */
  public static double XOR_INPUT[][] = { { 0.0, 0.0 }, { 1.0, 0.0 },
			{ 0.0, 1.0 }, { 1.0, 1.0 } };
 
/**
 * The ideal data necessary for XOR.
 */
  public static double XOR_IDEAL[][] = { { 0.0 }, { 1.0 }, { 1.0 }, { 0.0 } };
 
 
  public static void main(String[] args) {
    BasicNetwork method = new BasicNetwork();
    method.addLayer(new BasicLayer(null, true, 2));
    method.addLayer(new BasicLayer(new ActivationSigmoid(), true, 4));
    method.addLayer(new BasicLayer(new ActivationSigmoid(), false, 1));
    method.getStructure().finalizeStructure();
    method.reset();
 
    MLDataSet dataSet = new BasicMLDataSet(XOR_INPUT, XOR_IDEAL);
    // train to 1%
    EncogUtility.trainToError(method, dataSet, 0.01);
    // evaluate
    EncogUtility.evaluate(method, dataSet);
  }
}