Endless iterations with the sample code...

Hello,

Yes, I'm a neural network / encog newbie...

But I think I should be able to implement the most basic example from the XOR sample code.

Here's what I wrote:

private double[][] input = { { 0.0, 0.0 }, { 1.0, 0.0 }, { 0.0, 1.0 }, { 1.0, 1.0 } };
private double[][] ideal = { { 0.0 }, { 1.0 }, { 1.0 }, { 0.0 } };

BasicNetwork network = new BasicNetwork();
network.addLayer(new BasicLayer(2));
network.addLayer(new BasicLayer(2));
network.addLayer(new BasicLayer(1));

network.getStructure().finalizeStructure();
network.reset();

NeuralDataSet trainingSet = new BasicNeuralDataSet(input, ideal);

Train train = new ResilientPropagation(network, trainingSet);
int epoch = 1;
Logging.allConsoleLogging();
do
{
train.iteration();
System.out.println("Epoch #" + epoch + " Error:" + Format.formatPercent(train.getError()));
epoch++;
}
while(train.getError() > 0.02);

Using Encog 2.4 this do loop sometimes runs endless.

OK, not endless, but it runs until I stop it. The (truncated) output is:
...
Epoch #350402 Error:40,824829%
Epoch #350403 Error:40,824829%
Epoch #350404 Error:40,824829%
Epoch #350405 Error:40,824829%
Epoch #350406 Error:40,824829%
Epoch #350407 Error:40,824829%
Epoch #350408 Error:40,824829%
Epoch #350409 Error:40,824829%
Epoch #350410 Error:40,824829%
Epoch #350411 Error:40,824829%
Epoch #350412 Error:40,824829%
Epoch #350413 Error:40,824829%
...

Most likely the problem is sitting in front of the monitor and I'm sure I'm missing something simple...

Some other time the training succeds after a few hundred iterations.

And if I use my target data: An imput layer of 185 neurons, it also seems to be stuck...

André

yes, that will happen

jeffheaton's picture

With neural networks you are are starting from a random weight matrix. So every trainiis going to be different. Some converge really fast, others not so fast, still others never at all! Encog does not use "pure" random numbers, but rather weights them somewhat to try to get somewhat better results. This article talks about that.

http://www.heatonresearch.com/encog/articles/nguyen-widrow-neural-networ...

But you will still sometimes get a weight matrix that will not train. Adding this line can help.

train.addStrategy(new RequiredImprovementStrategy(5));

That will reset if there is minimal improvement over 5 iterations/epochs.


Copyright 2005 - 2010 by Heaton Research, Inc.. Heaton Research™ and Encog™ are trademarks of Heaton Research. Click here for copyright and trademark information.