You are here

how accurate should predictions be ?

Hi,

I recently modified PredictSunspot.java so that the "sunspots" were just a simple array, counting from 1 to 1000 in integers.

I then trained the network with the first 700, and left 300 to predict.

My results look something like this :

Epoch #357 Error:6.0225646766598225E-6
Epoch #358 Error:5.9548271874202975E-6
Row Actual Predict Closed Loop Predict
701 0.7017 0.6899 0.6899
702 0.7027 0.6906 0.6910
703 0.7037 0.6914 0.6910

My question is - how accurate should the predictions be ? My data set is a very simple linear problem but it fails to spot the pattern that each number increments the previous value by 1.

I've seen NN's do what I consider much more complicated things. Am I doing something wrong, or is this as good as temporal predictions can get ?

Thanks for any replies !

Neural Network Forums: 
fxmozart's picture

Hi,

I get much better results with 1 loop analysis (SVM).

See : http://www.heatonresearch.com/node/2250

fxmozart's picture

But are you doing predictions analysis or pattern , the networks you could use are different..

correlator's picture

hmmm... definitely a lot for me to learn here. I've tried tailoring PredictSunspotSVM in the same way - ie a simple array of numbers that increment by one - but not seeing eye popping accuracy !

I'm definitely doing something wrong, but the subject area is possibly too big for me at the moment.

Should it be possible to get very accurate numbers if the training set is just a simple number series ? Using either NN or SVM ?

correlator's picture

basically I just alter the sunspot as follows:

public final static double[] SUNSPOTS = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61,
...
...
1000};

correlator's picture

hmmm... so I tried this with another package (WEKA) being familiar with R also. Used a simlar approach and found pretty much exactly the same results. I'm not sure what I'm doing wrong here because I've seen sine wave prediction work with little lag and lots of accuracy - possibly a linear progression problem doesnt give the network enough training with the input data - ie sine waves are cyclical ?

anyway I've now started playing with the NEAT networks having read that recurrant neural networks are more suited to prediction type problems, and the NEAT network as I understand it take care of the network topology for you.

Not sure of the population sizes to use, but am starting to get some interesting results, albeit on this rather trivial problem!

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer