Technology

This article shows how the Multi Propagation (MPROP) algorithm was implemented for Encog for Java. Though this article focuses on the Java implementation the C# version would be very similar. MPROP is based on resilient propagation, but is designed to work well with multicore computers and gain maximum performance.

Are you a Java or C# programmer? Would you like to contribute some time to the Encog project? We are always looking for volunteers, and at all skill levels. You do need to be proficient with Java or C#, but you by no means need to be an AI expert. We are, of course, glad to have AI experts help! But we usually have tasks available that do not require advanced knowledge of AI.

Suggesting New Features

    You are now familiar with all of the layer and synapse types supported by Encog. You will now be given a brief introduction to building ANNs with these neural network types. You will see how to construct several neural network types. They will be used to solve problems related to the XOR operator. For now, the XOR operator is a good enough introduction to several neural network architectures. We will see more interesting examples, as the book progresses. We will begin with the feedforward neural network.

    We will now look at how to structure a neural network for a very simple problem. We will consider creating a neural network that can function as an XOR operator. Learning the XOR operator is a frequent “first example” when demonstrating the architecture of a new neural network. Just as most new programming languages are first demonstrated with a program that simply displays “Hello World”, neural networks are frequently demonstrated with the XOR operator.

The Jordan Neural Network is a simple recurrent network (SRN) developed by Michael I. Jordan in 1986. The context layer holds the previous output from the output layer and then echos that value back to the hidden layer's input. The hidden layer then always receives input from the previous iteration's output layer. Jordan neural networks are generally trained using genetic, simulated annealing, or one of the propagation techniques. Jordan neural networks are typically used for prediction.

The ADALINE neural network was developed by professor Bernard Widrow and his graduate student Ted Hoff. It is a very simple neural network usually used for pattern recognition. ADALINE is short for Adaptive Linear Neuron or Adaptive Linear Element. It is considered a single layer neural network. Though an activation function forms a sort of primitive output layer. Weighted connections are made to this activation function. A threshold, or bias, is also provided for. The output from an ADALINE neural network is usually bipolar.

The Jordan Neural Network is a simple recurrent network (SRN) developed by Michael I. Jordan in 1986. The context layer holds the previous output from the output layer and then echos that value back to the hidden layer's input. The hidden layer then always receives input from the previous iteration's output layer. Jordan neural networks are generally trained using genetic, simulated annealing, or one of the propagation techniques. Jordan neural networks are typically used for prediction.

In the last table you saw the results from the training errors from 1000 RPROP iterations of each of the S&P 500. Some companies trained better than others. Perhaps the patterns in their price movements were more repetitive. The next step is to take these neural networks, that were trained with historical data, and use them to attempt to predict recent price fluctuations. We will use all 500 companies again and see how they can predict the last 60 days of stock prices. This table summarizes this.

This table shows what the training error was for training a neural network over 10 years of data, from 1999 to 2008. 1000 training iterations of RPROP training was used on a neural network with an input window of 10, predict window of 1, using a 2-hidden layer feedforward neural network. Hidden layer 1 had 29 neurons, hidden layer 2 had 13 neurons. It is important to note that the percent is the training error, ideally you want to get this as low as possible. The companies with the higher percentages are more difficult to train for.