Understanding Neural Logic

    Every Encog neural network must contain a neural logic class. The NeuralLogic classes define how a neural network will process its layers and synapses. All neural logic classes must implement the NeuralLogic interface. By default a BasicNetwork class will make use of the SimpleRecurrentLogic logic class. This class can be used for both feedforward and simple recurrent networks. Because these are some of the most common neural network types in use, the SimpleRecurrentLogic class was chosen as the default.

    The next few sections summarize the network logic classes provided by Encog.

The ART1Logic Class

    The ART1Logic class is used to implement an adaptive resonance theory neural network. Adaptive Resonance Theory (ART) is a form of neural network developed by Stephen Grossberg and Gail Carpenter. There are several versions of the ART neural network, which are numbered ART-1, ART-2 and ART-3. The ART neural network is trained using either a supervised or unsupervised learning algorithm, depending on the version of ART being used. ART neural networks are used for pattern recognition and prediction. Encog presently supports ART1.

    To create an ART1 neural network with Encog you should make use of the ART1Logic class. An example of an ART1 neural network will be provided in Chapter 14, “Common Neural Network Patterns”.

The BAMLogic Class

    The BAMLogic class is used to implement a Bidirectional Associative Memory (BAM) network. The BAM network is a type of neural network developed by Bart Kosko in 1988. The BAM is a recurrent neural network that works similarly to and allows patterns of different lengths to be mapped bidirectionally to other patterns. This allows it to act as almost a two-way hash map. During its training, the BAM network is fed pattern pairs. The two halves of each pattern do not have to be of the same length. However, all patterns must be of the same overall structure. The BAM network can be fed a distorted pattern on either side and will attempt to map to the correct value.

    To create a BAM neural network with Encog you should make use of the BAMLogic class. An example of an BAM neural network will be provided in Chapter 14, “Common Neural Network Patterns”.

The BoltzmannLogic Class

    The BoltzmannLogic class is used to implement a Boltzmann machine neural network. A Boltzmann machine is a type of neural network developed by Geoffrey Hinton and Terry Sejnowski. It appears identical to a Hopfield neural network except it contains a random nature to its output. A temperature value is present that influences the output from the neural network. As this temperature decreases so does the randomness. This is called simulated annealing. Boltzmann networks are usually trained in an unsupervised mode. However, supervised training can be used to refine what the Boltzmann machine recognizes.

    To create a Boltzmann machine neural network with Encog you should make use of the BoltzmannLogic class. An example of a BAM neural network will be provided in Chapter 14, “Common Neural Network Patterns”.

The FeedforwardLogic Class

    To create a feedforward with Encog the FeedforwardLogic class should be used. It is also possible to use the SimpleRecurrentLogic class as in place of the FeedforwardLogic class; however, the network will run slower. If there are no recurrent loops, the more simple FeedforwardLogic class should be used.

    The feedforward neural network, or perceptron, is a type of neural network first described by Warren McCulloch and Walter Pitts in the 1940s. The feedforward neural network, and its variants, is the most widely used form of neural network. The feedforward neural network is often trained with the backpropagation training technique, though there are other more advanced training techniques, such as resilient propagation. The feedforward neural network uses weighted connections from an input layer to zero or more hidden layers, and finally to an output layer. It is suitable for many types of problems.

    To create a feedforward neural network with Encog you should make use of the FeedforwardLogic class. An example of a BAM neural network will be provided in Chapter 14, “Common Neural Network Patterns”.

The HopfieldLogic Class

    To create a Hopfield neural network with Encog, you should use the HopfieldLogic class. The Hopfield neural network was developed by Dr. John Hopfield in 1979. The Hopfield network is a single layer recurrent neural network. The Hopfield network always maintains a "current state" which is the current output of the neural network. The Hopfield neural network also has an energy property, which is calculated exactly the same as the temperature property of the Boltzmann machine. The Hopfield network is trained for several patterns. The state of the Hopfield network will move towards the closest pattern, thus "recognizing" that pattern. As the Hopfield network moves towards one of these patterns, the energy lowers.

    To create a feedforward neural network with Encog you should make use of the HopfieldLogic class. An example of an BAM neural network will be provided in Chapter 14, “Common Neural Network Patterns”.

The SimpleRecurrentLogic Class

    To create a neural network where some layers are connected to context layers that connect back to previous layers, you should use the SimpleRecurrentLogic class. The Elman and Jordan neural networks are examples of the sort of networks where the SimpleRecurrentLogic class can be used. The SimpleRecurrentLogic class can also be used to implement a simple feedforward neural network, however, the FeedforwardLogic class will execute faster.

    To create either an Elman or Jordan type of neural network with Encog you should make use of the SimpleRecurrentLogic class. An example of an Elman neural network will be provided in Chapter 14, “Common Neural Network Patterns”.

The SOMLogic Class

    To create a Self Organizing Map with Encog the SOMLogic class should be used. The Self Organizing Map (SOM) is a neural network type introduced by Teuvo Kohonen. SOM's are used to classify data into groups.

    To create a SOM neural network with Encog you should make use of the SOMLogic class. An example of a BAM neural network will be provided in Chapter 14, “Common Neural Network Patterns”.