You are here

Home » Neural Network Forums » Neural Networks » Financial Neural Networks

SVM..

fxmozart's picture
published by fxmozart on Tue, 08/23/2011 - 12:21

So SVM are supposed to be the "best" next thing after sex, but i just tried the sun spot example in c# (and to make sure my conversion was ok, i did the Java example at the same time).

SUNSPOTS prediction via SVM's:
Java :
Year Actual Predict Closed Loop Predict
1960 0.5723 0.5476 0.5476
1961 0.3267 0.3835 0.3754
1962 0.2577 0.2299 0.2458
1963 0.2173 0.1791 0.1764
1964 0.1429 0.2038 0.1879
1965 0.1635 0.2517 0.2670
1966 0.2977 0.3303 0.3679
1967 0.4946 0.4430 0.4637
1968 0.5455 0.5497 0.5300
1969 0.5438 0.5517 0.5397
1970 0.5395 0.4815 0.4798
1971 0.3801 0.4024 0.3829
1972 0.3898 0.2911 0.2960
1973 0.2598 0.2447 0.2254
1974 0.2451 0.1953 0.1931
1975 0.1652 0.1907 0.1873
1976 0.1530 0.1973 0.2151
1977 0.2148 0.2347 0.2615
1978 0.4891 0.3087 0.3194

C#:
Year\tActual\tPredict\tClosed Loop Predict
1960 0.5723 0.5474 0.5474
1961 0.3267 0.3837 0.3755
1962 0.2577 0.2301 0.2461
1963 0.2173 0.1790 0.1764
1964 0.1429 0.2037 0.1878
1965 0.1635 0.2517 0.2670
1966 0.2977 0.3304 0.3681
1967 0.4946 0.4428 0.4635
1968 0.5455 0.5488 0.5291
1969 0.5438 0.5503 0.5381
1970 0.5395 0.4802 0.4780
1971 0.3801 0.4016 0.3815
1972 0.3898 0.2909 0.2956
1973 0.2598 0.2448 0.2256
1974 0.2451 0.1957 0.1937
1975 0.1652 0.1914 0.1881
1976 0.1530 0.1982 0.2161
1977 0.2148 0.2351 0.2620
1978 0.4891 0.3083 0.3188
Press a key to continue ..

As you can see the results are next to identical (I didn't f--k up the conversion!) ..

-BUT here's the main question, this networks aren't TRAINING?..I run them , and both java or c# actually iterate once ? then even if i keep them running , the error stays the same?

Is there something to fiddle? the gamma etc?
If they don't train at all, it's more suited for statistics, classifications?
but the results are quite impressive i guess if we look at the time spent "learning" (1 iteration) versus other network types?

--Next question, why aren't they 100% identical in results?? Java or C# should be the same?

Are SVM actually suited for predictions at all??

Neural Network Forums: 
Financial Neural Networks
fxmozart's picture

Submitted by fxmozart on Tue, 08/23/2011 - 13:03.

SVM vs Basicnetwork..

Charts included..

Excel 2007:
http://uploading.com/files/3m7647d4/SVMPredict%2Bversus%2Bnetworks.xlsx/

Excel 2003:
http://uploading.com/files/1ef957fb/SVMPredict%2Bversus%2Bnetworks.xls/

Trained time for Basicnetwork : 7.6 seconds..

It beats the crap out of the svm??

jeffheaton's picture

Submitted by jeffheaton on Wed, 08/24/2011 - 06:22.

Okay, here is how that works. There are two ways to "train" a SVM. First, if you just use SVMTrain, that is a single iteration trainer. It converges VERY FAST to some error level. Often not a very good error level, however in the case of the sunspot example, a reasonably good error level.

If you tried SVM on the Iris example, it would not do nearly so good. To take SVM to the "next level" you have to use the search trainer. There are two paramaters used in SVM train that are very important. Gamma and a constant. The search method basically does a brute force rectangular search for a good Gamma and constant. It tracks which g/c produced the best SVM. There are even more advanced searches where you first do a fairly broad high-level search and then narrow in on "hot spots".

fxmozart's picture

Submitted by fxmozart on Wed, 08/24/2011 - 15:27.

OK..

Reading this : http://www.csie.ntu.edu.tw/~cjlin/talks/freiburg.pdf ..
http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf

Nice , i found : http://www.csie.ntu.edu.tw/~cjlin/papers/guide/data/ (will make a loader for this data for encog).

I found :
I can make this:

SVMSearchTrain search = new SVMSearchTrain(svm, training);
search.GammaStep = 0.1;
search.GammaBegin = 0.1;
search.GammaEnd = 1;
search.Strategies.Add(stop);
while (!stop.ShouldStop())
{
search.GammaStep = 0.1;
Console.WriteLine(@"Iterating maching network Error :" + search.Error + " iteration #:" +
search.IterationNumber);

There is however no example (and the above doesn't produce anything)...

This is the bruteforce right?

fxmozart's picture

Submitted by fxmozart on Sun, 09/25/2011 - 07:04.

OK found how to do it...

Updated my SVM predict sunspot pretty neat the SVMSearching.

public static void SVMSearch(SupportVectorMachine network, IMLDataSet training)
{
SVMSearchTrain bestsearch = new SVMSearchTrain(network, training);
StopTrainingStrategy stop = new StopTrainingStrategy(0.00000001, 1);
bestsearch.AddStrategy(stop);
while (bestsearch.IterationNumber < 500 && !stop.ShouldStop())
{
bestsearch.Iteration();
Console.WriteLine("Iteration #" + bestsearch.IterationNumber + " Error :" + bestsearch.Error);
}
}

If anyone is interested here's a small method i used to train the svm network.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer