You are here

Home » Neural Network Forums » Encog Project » Encog Developer Contributor (Java and C#)

C# trunk problems

jared's picture
published by jared on Tue, 05/04/2010 - 16:23

Hey everyone,

Trying to run the XOR sample using code from trunk:

BasicNetwork network = EncogUtility.SimpleFeedForward(2, 3, 0, 1, true);
BasicNeuralDataSet training = new BasicNeuralDataSet(XOR_INPUT, XOR_IDEAL);
ResilientPropagation train = new ResilientPropagation(network, training);

do {
train.Iteration();
Console.WriteLine("Train error: " + train.Error);
} while (train.Error > .001);

Console.WriteLine("Done");
Console.ReadKey();

network doesnt seem to be able to learn. error rate seems to cycle between .48 and .63

any ideas?

thanks!
jared

Neural Network Forums: 
Encog Developer Contributor (Java and C#)
jeffheaton's picture

Submitted by jeffheaton on Wed, 05/05/2010 - 09:23.

The trunk, being sort of the leading edge, of Encog development does sometimes develop a bug or two, and Encog 2.4 is still in the feature implementation phase... though nearly done... and then will be in bug fixing for awhile.

I will take a look and see if there is a bug thats tripping that up.

You might try running it a few times. The case with RPROP, and a network as small as the XOR, sometimes the random weights are in a bad enough state that RPROP fails to ever get the net to converge.

Jeff

jeffheaton's picture

Submitted by jeffheaton on Thu, 05/06/2010 - 04:50.

Thanks for the report!

jeffheaton's picture

Submitted by jeffheaton on Thu, 05/06/2010 - 17:45.

Okay, the bug has been fixed and checked into the trunk! Thanks for the report!

jared's picture

Submitted by jared on Tue, 05/11/2010 - 13:25.

hey jeff,

tested the simple function within sandbox and it looks much better! it took more than 30,000 iterations to reach the train error below .001 with a 6 neuron second layer, though. is that expected?

really looking forward to the new release, you're working on some really incredible stuff!

thanks again,
jared

jeffheaton's picture

Submitted by jeffheaton on Tue, 05/11/2010 - 20:27.

Actually that is expected. The sandbox code, which by the way is highly experimental :), is training XOR with a TANH activation function. Sigmoid works MUCH better than TANH for the XOR. Sigmoid only ranges between 0 and 1, just like the sigmoid. TANH ranges -1 to 1. Of course the neural network eventually figures it out, but it just takes longer.

Use this line and it will use a sigmoid activation function.

BasicNetwork network = EncogUtility.SimpleFeedForward(2, 3, 0, 1, false);

And thanks!

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer