You are here

Home » Neural Network Forums » Encog Project » Using Encog in Java

Can Encog neural networks work with map reduce?

jzhou's picture
published by jzhou on Sat, 06/30/2012 - 01:38

Hi There,

Is a grid implementation available for Encog 3.1?

I am just wondering if the current version of Encog neural networks for java library could work with map reduce in order to take the advantage of parallel computing.

Thanks
Joe

Neural Network Forums: 
Using Encog in Java
jeffheaton's picture

Submitted by jeffheaton on Fri, 07/13/2012 - 16:36.

Encog can be used inside of a map-reduce framework. Extending the training to map-reduce is not always straightforward. For example, the backpropagation based training (due to the nature of the algorithm) must bottleneck to a single thread at the end of each training iteration. This is not too big of a problem with threads, but when you must syncrhonize multiple computers the overhead can quickly eat of any gain you got.

jzhou's picture

Submitted by jzhou on Fri, 07/13/2012 - 22:03.

Thank you very much for the reply.
In the case of having a large training data set, I could divide the set into several smaller ones each of which will then be passed to the mapper function. The mapper propagates its set of data through the network.For each training example, the error is back propagated to calculate the partial gradient for each of the weights in the network. The reducer then sums the partial gradient from each mapper and does a batch gradient descent to update the weights of the network.

I am just wondering if the above process could be implemented using Encog 3.1. Can you please provide me some examples if it is possible?

Thank you very much!
Joe

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer