XOR Operator

    In the last chapter, backpropagation was used to solve the XOR operator problem. In this section, you will see how a genetic algorithm, the TrainingSetNeuralGeneticAlgorithm class, can be used to train for the XOR operator. This version of the XOR solver can be seen in Listing 6.2.

Listing 6.2: XOR with a Genetic Algorithm

/**
 * Introduction to Neural Networks with Java, 2nd Edition
 * Copyright 2008 by Heaton Research, Inc. 
 * http://www.heatonresearch.com/books/java-neural-2/
 * 
 * ISBN13: 978-1-60439-008-7  	 
 * ISBN:   1-60439-008-5
 *   
 * This class is released under the:
 * GNU Lesser General Public License (LGPL)
 * http://www.gnu.org/copyleft/lesser.html
 */
package com.heatonresearch.book.introneuralnet.ch6.xor;

import com.heatonresearch.book.introneuralnet.neural.feedforward.FeedforwardLayer;
import com.heatonresearch.book.introneuralnet.neural.feedforward.FeedforwardNetwork;
import com.heatonresearch.book.introneuralnet.neural.feedforward.train.genetic.TrainingSetNeuralGeneticAlgorithm;


/**
 * Chapter 6: Training using a Genetic Algorithm
 * 
 * XOR: Learn the XOR pattern with a feedforward neural network that
 * uses a genetic algorithm.
 * 
 * @author Jeff Heaton
 * @version 2.1
 */
public class GeneticXOR {
	public static double XOR_INPUT[][] = { { 0.0, 0.0 }, { 1.0, 0.0 },
			{ 0.0, 1.0 }, { 1.0, 1.0 } };

	public static double XOR_IDEAL[][] = { { 0.0 }, { 1.0 }, { 1.0 }, { 0.0 } };

	public static void main(final String args[]) {
		FeedforwardNetwork network = new FeedforwardNetwork();
		network.addLayer(new FeedforwardLayer(2));
		network.addLayer(new FeedforwardLayer(3));
		network.addLayer(new FeedforwardLayer(1));
		network.reset();

		// train the neural network
		final TrainingSetNeuralGeneticAlgorithm train = new TrainingSetNeuralGeneticAlgorithm(
				network, true, XOR_INPUT, XOR_IDEAL, 5000, 0.1, 0.25);

		int epoch = 1;

		do {
			train.iteration();
			System.out
					.println("Epoch #" + epoch + " Error:" + train.getError());
			epoch++;
		} while ((epoch < 5000) && (train.getError() > 0.001));

		network = train.getNetwork();

		// test the neural network
		System.out.println("Neural Network Results:");
		for (int i = 0; i < XOR_IDEAL.length; i++) {
			final double actual[] = network.computeOutputs(XOR_INPUT[i]);
			System.out.println(XOR_INPUT[i][0] + "," + XOR_INPUT[i][1]
					+ ", actual=" + actual[0] + ",ideal=" + XOR_IDEAL[i][0]);
		}
	}
}

    The above listing is very similar to the XOR solver from the last chapter. The main difference is in the following lines:

final TrainingSetNeuralGeneticAlgorithm train = new TrainingSetNeuralGeneticAlgorithm(
network, true, XOR_INPUT, XOR_IDEAL, 5000, 0.1, 0.25);

    As you can see, several parameters are passed to the constructor of the training class. First, the network to be trained is passed. The value of true specifies that all of the initial life forms should have their weight matrixes randomly initialized. The input and ideal arrays are also provided, just as they are with the backpropagation algorithm. The value of 5000 specifies the size of the population. The value 0.1 indicates that 10% of the population will be chosen to mate. The 10% will be able to mate with any life form in the top 25%.

Calculate Cost

    A specialized method is then used to calculate the cost for the XOR pattern recognition. The signature for the cost calculation method is shown here:

public void calculateCost() throws NeuralNetworkError {

    First, the contents of the chromosome are copied back into the neural network.

this.updateNetwork();

    The root mean square error of the neural network can be used as a cost. The RMS error will be calculated just as it was in previous chapters. The input and ideal arrays are copied to local variables.

final double input[][] = this.getGeneticAlgorithm().getInput();

final double ideal[][] = this.getGeneticAlgorithm().getIdeal();

    The neural network's calculateError function is used to calculate the error of the neural network given the input and ideal arrays.

setCost(getNetwork().calculateError(input, ideal));

    Although the XOR program uses a specialized cost calculation method, the program uses the same mating method as previously discussed.

Mutate

    Mutation is handled by a specialized mutate function. The signature for this mutate function is shown here:

public void mutate()

    The mutation method begins by obtaining the number of genes in a chromosome. Each gene will be scaled using a random percentage.

final int length = getGenes().length; 

    The function loops through all of the genes.

for (int i = 0; i < length; i++) { 

    It obtains a gene's value and multiplies it by a random ratio within a specified range.

  double d = getGene(i); 			 			
  final double ratio = (int) ((RANGE * Math.random()) - RANGE); 			 			 			 			
  d*=ratio; 			 			
  setGene(i,d); 			 		
}

    The result is that each weight matrix element is randomly changed.