Company Technology

    You are now familiar with all of the layer and synapse types supported by Encog. You will now be given a brief introduction to building ANNs with these neural network types. You will see how to construct several neural network types. They will be used to solve problems related to the XOR operator. For now, the XOR operator is a good enough introduction to several neural network architectures. We will see more interesting examples, as the book progresses. We will begin with the feedforward neural network.

    We will now look at how to structure a neural network for a very simple problem. We will consider creating a neural network that can function as an XOR operator. Learning the XOR operator is a frequent “first example” when demonstrating the architecture of a new neural network. Just as most new programming languages are first demonstrated with a program that simply displays “Hello World”, neural networks are frequently demonstrated with the XOR operator.

Counterpropagation Neural Networks (CPN) were devloped by Professor Robert Hecht-Nielsen in 1987. CPN neural networks are a hybrid neural network, employing characteristics of both a feedforward neural network and a self-organizing map (SOM). The CPN is composed of three layers, the input, the instar and the outstar. The connection from the input to the instar layer is competitive, with only one neuron being allowed to win. The connection between the instar and outstar is feedforward. The layers are trained separately, using instar training and outstar training.

In the last table you saw the results from the training errors from 1000 RPROP iterations of each of the S&P 500. Some companies trained better than others. Perhaps the patterns in their price movements were more repetitive. The next step is to take these neural networks, that were trained with historical data, and use them to attempt to predict recent price fluctuations. We will use all 500 companies again and see how they can predict the last 60 days of stock prices. This table summarizes this.

This table shows what the training error was for training a neural network over 10 years of data, from 1999 to 2008. 1000 training iterations of RPROP training was used on a neural network with an input window of 10, predict window of 1, using a 2-hidden layer feedforward neural network. Hidden layer 1 had 29 neurons, hidden layer 2 had 13 neurons. It is important to note that the percent is the training error, ideally you want to get this as low as possible. The companies with the higher percentages are more difficult to train for.

    A feedforward neural network is similar to the types of neural networks that we have already examined. Just like many other types of neural networks, the feedforward neural network begins with an input layer. The input layer may be connected to a hidden layer or directly to the output layer. If it is connected to a hidden layer, the hidden layer can then be connected to another hidden layer or directly to the output layer. There can be any number of hidden layers, as long as there is at least one hidden layer or output layer provided.

    Error calculation is an important aspect of any neural network. Whether the neural network is supervised or unsupervised, an error rate must be calculated. The goal of virtually all training algorithms is to minimize the rate of error. In this section, we will examine how the rate of error is calculated for a supervised neural network. We will also discuss how the rate of error is determined for an unsupervised training algorithm. We will begin this section by examining two error calculation steps used for supervised training.