Bias on Encog

Submitted by joaoffr on Tue, 02/02/2010 - 06:28

Using Encog in Java

Hello Jeff! I'm new in this forum and I recently start using Encog in my researches in the time series forecasting area (networking prevision).

In my opinion and until now, Encog is a very good tool, much better than others that I already tried.

I'm building several NN configurations and one of them is a NN with 3 layers: I input neurons; H hidden neurons and 1 output neuron.

My network might have shortcuts between input and output layers and a bias in all hidden layer nodes.

My question is: in Encog, how can i configure a +1 bias for all nodes from a specific layer?

PS: Sorry because my poor English... I'm Portuguese!

Grateful for your attention!
João Rodrigues

bias

Submitted by jeffheaton on Tue, 02/02/2010 - 08:57.

Okay, short cuts are something that I mean to start playing with soon in Encog. Especially, since one of the features that I hope to add in the next Encog release is the Cascade2 algorithm. I assume that they would work, but have not actually tried one.

As to bias, Encog does have an optional threshold value, at each layer, that is automatically added to each of the neurons for that later, which sort of acts as the "bias" in Encog. All of the training algorithms in Encog will modify this threshold, though, as the weights are modified.

Did you mean a fixed threshold value, that would not change with training?

Jeff

More or less...

Submitted by joaoffr on Tue, 02/02/2010 - 10:22.

Hello again!

First of all, thanks for the quick reply!

I know that Encog have threshold values for each layer, but is not what I want.

When I spoke in 'bias', I was talking about a kind of 'input static value' which serves as 'side input' to a neuron. The connection between this value and each neuron (in the hidden layer, for example) may be weighted as in the connections between common layers. This 'bias' value should help the network to have an output value much different from the input value and from the 'normal' value, allowing a small error in less epochs in cases that the inputs varying rapidly in a specified range.

I don't know if you are understanding my point, so, I found a image in this address with an example similar to my NN:
http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/fig/mlp3bias.gif

In this page we have an explanation about what I said:
http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-13-text.html

Thank you very much!