org.encog.neural.networks.training.propagation.resilient
Class ResilientPropagation

java.lang.Object
  org.encog.neural.networks.training.BasicTraining
      org.encog.neural.networks.training.propagation.Propagation
          org.encog.neural.networks.training.propagation.resilient.ResilientPropagation

All Implemented Interfaces:

Train

public class ResilientPropagation
extends Propagation

One problem with the backpropagation algorithm is that the magnitude of the partial derivative is usually too large or too small. Further, the learning rate is a single value for the entire neural network. The resilient propagation learning algorithm uses a special update value(similar to the learning rate) for every neuron connection. Further these update values are automatically determined, unlike the learning rate of the backpropagation algorithm. For most training situations, we suggest that the resilient propagation algorithm (this class) be used for training. There are a total of three parameters that must be provided to the resilient training algorithm. Defaults are provided for each, and in nearly all cases, these defaults are acceptable. This makes the resilient propagation algorithm one of the easiest and most efficient training algorithms available. The optional parameters are: zeroTolerance - How close to zero can a number be to be considered zero. The default is 0.00000000000000001. initialUpdate - What are the initial update values for each matrix value. The default is 0.1. maxStep - What is the largest amount that the update values can step. The default is 50. Usually you will not need to use these, and you should use the constructor that does not require them.

Author:

jheaton

Field Summary

static double	DEFAULT_INITIAL_UPDATE The starting update for a delta.
static double	DEFAULT_MAX_STEP The maximum amount a delta can reach.
static double	DEFAULT_ZERO_TOLERANCE The default zero tolerance.
static double	DELTA_MIN The minimum delta value for a weight matrix value.
static java.lang.String	LAST_GRADIENTS Continuation tag for the last gradients.
static double	NEGATIVE_ETA The NEGATIVE ETA value.
static double	POSITIVE_ETA The POSITIVE ETA value.
static java.lang.String	UPDATE_VALUES Continuation tag for the last values.

Constructor Summary

ResilientPropagation(BasicNetwork network, NeuralDataSet training)
Construct a resilient training object.

ResilientPropagation(BasicNetwork network, NeuralDataSet training, double zeroTolerance, double initialUpdate, double maxStep)
Construct a resilient training object, allow the training parameters to be specified.

Method Summary

boolean	canContinue()
double	getInitialUpdate()
double	getMaxStep()
double	getZeroTolerance()
boolean	isValidResume(TrainingContinuation state) Determine if the specified continuation object is valid to resume with.
TrainingContinuation	pause() Pause the training.
void	performIteration(CalculateGradient prop, double[] weights) Perform a training iteration.
void	resume(TrainingContinuation state) Resume training.

Methods inherited from class org.encog.neural.networks.training.propagation.Propagation

getNetwork, getNumThreads, iteration, setNumThreads

Methods inherited from class org.encog.neural.networks.training.BasicTraining

addStrategy, finishTraining, getError, getStrategies, getTraining, postIteration, preIteration, setError, setTraining

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_ZERO_TOLERANCE

public static final double DEFAULT_ZERO_TOLERANCE

The default zero tolerance.

See Also:

Constant Field Values

POSITIVE_ETA

public static final double POSITIVE_ETA

The POSITIVE ETA value. This is specified by the resilient propagation algorithm. This is the percentage by which the deltas are increased by if the partial derivative is greater than zero.

See Also:

Constant Field Values

NEGATIVE_ETA

public static final double NEGATIVE_ETA

The NEGATIVE ETA value. This is specified by the resilient propagation algorithm. This is the percentage by which the deltas are increased by if the partial derivative is less than zero.

See Also:

Constant Field Values

DELTA_MIN

public static final double DELTA_MIN

The minimum delta value for a weight matrix value.

See Also:

Constant Field Values

DEFAULT_INITIAL_UPDATE

public static final double DEFAULT_INITIAL_UPDATE

The starting update for a delta.

See Also:

Constant Field Values

DEFAULT_MAX_STEP

public static final double DEFAULT_MAX_STEP

The maximum amount a delta can reach.

See Also:

Constant Field Values

LAST_GRADIENTS

public static final java.lang.String LAST_GRADIENTS

Continuation tag for the last gradients.

See Also:

Constant Field Values

UPDATE_VALUES

public static final java.lang.String UPDATE_VALUES

Continuation tag for the last values.

See Also:

Constant Field Values

Constructor Detail

ResilientPropagation

public ResilientPropagation(BasicNetwork network,
                            NeuralDataSet training)

Construct a resilient training object. Use the defaults for all training parameters. Usually this is the constructor to use as the resilient training algorithm is designed for the default parameters to be acceptable for nearly all problems.

Parameters:

network - The network to train.

training - The training set to use.

ResilientPropagation

public ResilientPropagation(BasicNetwork network,
                            NeuralDataSet training,
                            double zeroTolerance,
                            double initialUpdate,
                            double maxStep)

Construct a resilient training object, allow the training parameters to be specified. Usually the default parameters are acceptable for the resilient training algorithm. Therefore you should usually use the other constructor, that makes use of the default values.

Parameters:

network - The network to train.

training - The training set to use.

zeroTolerance - The zero tolerance.

initialUpdate - The initial update values, this is the amount that the deltas are all initially set to.

maxStep - The maximum that a delta can reach.

Method Detail

canContinue

public boolean canContinue()

Overrides:

canContinue in class Propagation

Returns:

True, as RPROP can continue.

getInitialUpdate

public double getInitialUpdate()

Returns:

The initial update amount, set by the constructor.

getMaxStep

public double getMaxStep()

Returns:

The maximum step, set by the constructor.

getZeroTolerance

public double getZeroTolerance()

Returns:

The zero tolerance, set by the constructor.

isValidResume

public boolean isValidResume(TrainingContinuation state)

Determine if the specified continuation object is valid to resume with.

Overrides:

isValidResume in class Propagation

Parameters:

state - The continuation object to check.

Returns:

True if the specified continuation object is valid for this training method and network.

pause

public TrainingContinuation pause()

Pause the training.

Overrides:

pause in class Propagation

Returns:

A training continuation object to continue with.

performIteration

public void performIteration(CalculateGradient prop,
                             double[] weights)

Perform a training iteration. This is where the actual RPROP specific training takes place.

Specified by:

performIteration in class Propagation

Parameters:

prop - The gradients.

weights - The network weights.

resume

public void resume(TrainingContinuation state)

Resume training.

Overrides:

resume in class Propagation

Parameters:

state - The training state to return to.