You are here

Home » Neural Network Forums » Encog Project » Using Encog in Java

Identify a stuck situation

Clifte's picture
published by Clifte on Fri, 08/10/2012 - 13:13

Hi,
I am using a feed forward back propagation network to classify
manuscript data. I am using 15 samples of each class from
a total of 10 classes.

The network has 784 input neurons,392 hidden neuron in a single layer and 10 neurons in output layer.
The learn rate and momentum are both 0.1.

Can I say that the network won't converge?
What should I do now? Increase the number of hidden neurons or the number of layers?
How can I get out of a local minimum?

Epoch #1 Error:0.3162277613054124
Epoch #2 Error:0.31622776130517904
Epoch #3 Error:0.3162277613049436
Epoch #4 Error:0.31622776130470787
Epoch #5 Error:0.31622776130447205
Epoch #6 Error:0.3162277613042363
Epoch #7 Error:0.3162277613040005
Epoch #8 Error:0.3162277613037646
Epoch #9 Error:0.3162277613035288
Epoch #10 Error:0.3162277613032929
Epoch #11 Error:0.31622776130305696
Epoch #12 Error:0.316227761302821
Epoch #13 Error:0.31622776130258506
Epoch #14 Error:0.31622776130234903
Epoch #15 Error:0.316227761302113
Epoch #16 Error:0.31622776130187696
Epoch #17 Error:0.31622776130164093
Epoch #18 Error:0.31622776130140484
Epoch #19 Error:0.31622776130116875
Epoch #20 Error:0.31622776130093255
Epoch #21 Error:0.3162277613006964
Epoch #22 Error:0.31622776130046015
Epoch #23 Error:0.316227761300224
Epoch #24 Error:0.3162277612999877
Epoch #25 Error:0.31622776129975144
Epoch #26 Error:0.3162277612995152
Epoch #27 Error:0.3162277612992789
Epoch #28 Error:0.31622776129904256
Epoch #29 Error:0.3162277612988062
Epoch #30 Error:0.3162277612985698
Epoch #31 Error:0.31622776129833335
Epoch #32 Error:0.316227761298097
Epoch #33 Error:0.3162277612978605
Epoch #34 Error:0.31622776129762403
Epoch #35 Error:0.3162277612973875
Epoch #36 Error:0.31622776129715097
Epoch #37 Error:0.31622776129691443
Epoch #38 Error:0.31622776129667785
Epoch #39 Error:0.31622776129644126
Epoch #40 Error:0.3162277612962046
Epoch #41 Error:0.3162277612959679
Epoch #42 Error:0.31622776129573127
Epoch #43 Error:0.31622776129549457
Epoch #44 Error:0.3162277612952578
Epoch #45 Error:0.31622776129502106
Epoch #46 Error:0.31622776129478436

Neural Network Forums: 
Using Encog in Java
jeffheaton's picture

Submitted by jeffheaton on Thu, 08/16/2012 - 11:31.

First, make sure your data is normalized to 0 to 1 for sigmoid, or -1 to 1 for tanh. If the data is in a form the neural network can't learn from, then you will see it get stuck like this.

I would also try it on a global search such as PSO (particle swarm) or genetic. See if you can get below the 31%. Then perhaps use RPROP on one of the better trained networks from PSO.

Failing all of that, I would look at your data and see if it is overly noisy or contradictory.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer