Summary

    Encog uses activation functions to scale the output from neural network layers. By default, Encog will use a hyperbolic tangent function, which is a good general purposes activation function. Any class that acts as an activation function must implement the ActivationFunction interface. This interface requires the implementation of several methods. First an activationFunction method must be created to actually perform the activation function. Secondly, a derivativeFunction method should be implemented to return the derivative of the activation function. If there is no way to take a derivative of the activation function, then an error should be thrown. Only activation functions that have a derivative can be used with propagation training.

    The ActivationBiPolar activation function class is used when your network only accepts bipolar numbers. The ActivationCompetitive activation function class is used for competitive neural networks, such as the Self-Organizing Map. The ActivationGaussian activation function class is used when you want a gaussian curve to represent the activation function. The ActivationLinear activation function class is used when you want to have no activation function at all. The ActivationLOG activation function class works similarly to the ActivationTANH activation function class except it will sometimes not saturate as a hidden layer. The ActivationSigmoid activation function class is similar to the ActivationTANH activation function class, except only positive numbers are returned. The ActivationSIN activation class can be used for periodic data. The ActivationSoftMax activation function class scales the output so that the sum is one.

    Up to this point we have covered all of the major components of neural networks. Layers contain the neurons and threshold values. Synapses connect the layers together. Activation functions sit inside the layers and scale the output. Tags allow special layers to be identified. Properties allow configuration values to be associated with the neural network. The next chapter will introduce the Encog Workbench. The Encog Workbench is a GUI application that let you build neural networks composed of all of these elements.