You are here

Home » Neural Network Forums » Encog Project » Encog General Discussion

Retraining a trained network

chris_c's picture
published by chris_c on Thu, 08/02/2012 - 12:46

Just started experimenting with the java library and aint NN fascinating "creatures" ;)

Some questions occur to me.

Once trained is it possible (or advisable!) to train again with a different dataset, or would it be recommended to start from scratch with the old dataset and the new dataset together...

What is a sensible error target? for example is there any practical advantage to say 0.0005 and 0.005 in terms of the end accuracy of the net

Is there a rule of thumb for layer size and number of layers, for example if I have 144 inputs :o and 26 outputs.....

Sigmoid activation tends to get stuck in valleys (sometimes without escape)
where hyperbolic tangent does't seem to at least with my dataset (5,200 items), is this to be expected or just a peculiarities of my data/network?
(my inputs don't have -tive values)

Neural Network Forums: 
Encog General Discussion
jeffheaton's picture

Submitted by jeffheaton on Wed, 08/08/2012 - 08:09.

Retraining a neural network with a different dataset typically does not work too well. It will typically forget what it learned from the first dataset. A neural network will use all neurons to learn whatever the dataset at hand is.

The error target depends. Ideally it is the best error you can get without overfitting. TO ensure you are not overfitting you should evaluate the neural network with a dataset that it was not training on.

No real rule of thumb on hidden layer setup. Typically more than one hidden layer is unnecessary. I usually start with half my input neurons, and work from there. But depending on the complexity of the pattern, you might well get by with less.

Not sure on getting stuck in valleys with this particular dataset... would really have to experiment.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer