You are here

Home » Neural Network Forums » Neural Networks » General Neural Networks

A Few Questions

MaximusFromEarth's picture
published by MaximusFromEarth on Fri, 09/07/2012 - 22:10

Firstly, I'd just like to say thank you to Jeff Heaton and everyone involved with Encog, excellent work guys. I've been playing with it for a few weeks now, both the workbench and the java version. I am quite new to Neural Networks, but I am learning fast and enjoying all the videos and materials.

Ok, now the questions:

1. In the java sunspot examples, the data is normalized to a high of 0.9 and a low of 0.1. Why is this? Seems to me that it should be between 0 and 1, or between -1 and 1.

2. In the PredictSunspotSVM.java example, if I understand the code correctly, there is only one training iteration, yet this example seems to be the most accurate of the three PredictSunspot examples. What is going on here?

3. In the PredictSunspotElman.java example, in the console after running, I see that a Predict value is printed for each line of both the Train and Evalu sections. Why does it predict while still training? Or am I completely off course in my thinking about this?

I'll stop there for now, but I do have many more questions.

Max

Neural Network Forums: 
General Neural Networks
jeffheaton's picture

Submitted by jeffheaton on Sat, 09/08/2012 - 08:23.

Thanks, glad it is helpful!

1. Sometimes I will use slightly less than the entire range just so that more of the actual curve of the tanh function is used. This is something I was experimenting with for this example.

2. A SVM does much more with an iteration than a neural network. Typically a SVM training iteration is trying out two values a gamma and a "const". There is a SVMSearch trainer that runs for much longer, as it does a grid search on these two.

3. The prefix train: and eval: in this example specify where each year came from. Train means that it was part of the training set. The neural network will do well on these, because it has already seen this data. Eval means this is data that the network was not trained with.

MaximusFromEarth's picture

Submitted by MaximusFromEarth on Sun, 09/16/2012 - 00:29.

I've been testing the SVMSearchTrain for regression, attempting to predict the EURUSD, and I'm not sure exactly how to go about training process.

My main questions is How many iterations should I do? The error doesn't improve after 24 iterations (at least not for the 200 iterations I tried). But I will get a different output on the same query for different iteration counts.

For example:
1 iteration --- Error:0.001801608589709883 --- Prediction:1.30966
4 iterations --- Error:8.166579305708417E-4 --- Prediction:1.31038
20 iterations --- Error:2.47776550916589E-4 --- Prediction:1.31094
24 iterations --- Error:2.216230758783644E-4 --- Prediction:1.31197
50 iterations --- Error:2.216230758783644E-4 --- Prediction:1.31139
100 iterations --- Error:2.216230758783644E-4 --- Prediction:1.31327

As you can see, the Error is the same for 24, 50, and 100 iterations.

This is my train method

public void train(SVM theSVM, MLDataSet training)
{
final SVMSearchTrain search = new SVMSearchTrain(theSVM, training);
search.setGammaBegin(0.0001);
search.setGammaStep(0.0001);
//search.setGammaEnd(1);

//search.setConstBegin(0.1);
//search.setConstStep(0.1);
//search.setConstEnd(1);

int epoch = 1;

do {
search.iteration();
System.out
.println("Epoch #" + epoch + " Error:" + search.getError());
epoch++;
} while(search.getError() > MAX_ERROR && epoch < MAX_ITERATIONS);

System.out.println("Saving " + FILENAME);
EncogDirectoryPersistence.saveObject(new File(FILENAME), theSVM);
}

This is some more relevant code from a different method


SVM networkSVM = new SVM(sampleSize * 3,true);
MLDataSet trainingSet = new BasicMLDataSet(INPUT_DATA_NORMALIZED, IDEAL_DATA_NORMALIZED);
train(networkSVM,trainingSet);

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer