Home › Neural Network Forums › Neural Networks › General Neural Networks ›

Local Minima Escape Using Large Dataset

MarcF's picture
Submitted by MarcF on Mon, 07/26/2010 - 06:55
in

I am using the ScaledConjugate training method for a feed forward neural network on a large dataset ~300MB and it seems to continously get stuck in local minima. I have added simulated annealing to the training which does help minimise but the anneal does not seem to multi thread taking comparatively much much longer. Are there any better ways to get out of local minima which take advantage of multiple cores?

Regards,
Marc

‹ Smoothness of a curve Fitness criterion ›
jeffheaton's picture
Submitted by jeffheaton on Tue, 07/27/2010 - 04:14.

Simulated annealing is not yet multi-threaded. It is on the list, but have not gotten to it yet.

Another approach, I use somethimes, is to use a genetic algorithm, and then use scaled conj on say the top 30 of the population. Since the top members of the population are generally quite different from each other.

Copyright 2005 - 2012 by Heaton Research, Inc.. Heaton Research™ and Encog™ are trademarks of Heaton Research. Click here for copyright, license and trademark information.