You are here

Error never goes under 25% in modified XOR Problem

Here's a sample:
Epoch #122258 Error:0.2927216006463507
Epoch #122259 Error:0.24672919174738922
Epoch #122260 Error:0.24672919174738922
Epoch #122261 Error:0.24672919174738922
Epoch #122262 Error:0.24672919174738922
Epoch #122263 Error:0.24672919174738922
Epoch #122264 Error:0.24672919174738922
Epoch #122265 Error:0.24672919174738922
Epoch #122266 Error:0.254568263287881
Epoch #122267 Error:0.254568263287881
Epoch #122268 Error:0.254568263287881
Epoch #122269 Error:0.2543796589731955
Epoch #122270 Error:0.25420582845109413
Epoch #122271 Error:0.25420582845109413
Epoch #122272 Error:0.25420582845109413

This is even after I added this line:
train.addStrategy(new RequiredImprovementStrategy(5));

Here's my full source:

import org.encog.ml.data.MLData;
import org.encog.ml.data.MLDataPair;
import org.encog.ml.data.MLDataSet;
import org.encog.ml.data.basic.BasicMLDataSet;
import org.encog.ml.train.MLTrain;
import org.encog.neural.networks.BasicNetwork;
import org.encog.neural.networks.layers.BasicLayer;
import org.encog.neural.networks.training.propagation.back.Backpropagation;
import org.encog.engine.network.activation.ActivationSigmoid;
import org.encog.ml.train.strategy.RequiredImprovementStrategy;

public class XORHelloWorld {

public static double XOR_INPUT[][] = {
{0.0, 0.0},
{1.0, 0.0},
{0.0, 1.0},
{2.0, 1.0}
};

public static double XOR_IDEAL[][] = {
{0.0},
{1.0},
{1.0},
{0.0}
};

public static void main(String args[]) {
//SystemLoggingPlugin.stopConsoleLogging();

BasicNetwork network = new BasicNetwork();
network.addLayer(new BasicLayer(2));
network.addLayer(new BasicLayer(new ActivationSigmoid(), true, 3));
network.addLayer(new BasicLayer(new ActivationSigmoid(), true, 1));

network.getStructure().finalizeStructure();

MLDataSet trainingSet = new BasicMLDataSet(XOR_INPUT, XOR_IDEAL);

MLTrain train = new Backpropagation(network, trainingSet);

for (MLDataPair pair : trainingSet) {
System.out.println(pair.getInput().getData(0) + "," + pair.getInput().getData(1) + ",ideal=" + pair.getIdeal().getData(0));
}

int epoch = 1;

train.addStrategy(new RequiredImprovementStrategy(5));

do {
train.iteration();
System.out.println("Epoch #" + epoch + " Error:" + train.getError());
epoch++;
} while (train.getError() > 0.01);

for (MLDataPair pair : trainingSet) {
final MLData output = network.compute(pair.getInput());
System.out.println(pair.getInput().getData(0) + "," + pair.getInput().getData(1) + ", actual=" + output.getData(0) + ",ideal=" + pair.getIdeal().getData(0));
}
}
}

Neural Network Forums: 
jeffheaton's picture

That is not an XOR function. You have the line:

{2.0, 1.0}

Which is not actually an XOR function anymore. It is also un-normalized (neural networks expect an input in either -1 to 1 or 0 to 1, depending on activation function.

Try the example located here:

org.encog.examples.neural.xor.XORHelloWorld

pat.natali's picture

I can't believe I missed that! What a silly mistake.

Thanks Jeff

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer