Using jdk-1.7 for encog

Submitted by SamLi on Thu, 12/17/2009 - 09:20

Using Encog in Java

jdk-1.7 has this parallelism feature where you could fork multiple subtasks and then joining them after they are finished.

The reason for this feature is allow users to leverage increasing number of cores on new CPUs these days while their
clock speed somehow remain constant.

I would like to apply this idea to the Matrix class so that we could do computations in faster ways.

Submitted by jeffheaton on Thu, 12/17/2009 - 20:44.

I have not gotten into much of the JDK 1.7 additions yet. We do use multithreading in Encog, particularly in MPROP, Genetic algorithms, and incremental pruning. I use these functions on a quadcore (i7) and it does speed things up greatly.

Picking the point at which you make things parallel is tricky. At first I tried to make the matrix multiplication and dot products parallel to speed things up. Perhaps on VERY large neural networks this would work, but the problem I ran into was that I can't process very many matrix operations before I have to wait for previous matrix operations to complete before moving forward.

So I ended up making things parallel higher than the matrix math. For MPROP this was in the calculation of the gradients. Basically break the training set up into a number of threads and then join together briefly near the end of each iteration to use these gradients however the learning algorithm specifies.

If you have any luck making things parallel at the matrix level, I would be very intersted in seeing how you set it up with Encog.