Rprop stands for '** R**esilient back** prop**agation' and is a
local adaptive learning scheme, performing supervised batch learning
in multi-layer perceptrons. For a detailed discussion see also
[Rie93], [RB93].
The basic principle of Rprop is to eliminate the harmful influence of
the size of the partial derivative on the weight step. As a
consequence, only the sign of the derivative is considered to indicate
the * direction* of the weight update. The * size* of the
weight change is exclusively determined by a weight-specific,
so-called 'update-value' :

where denotes the summed gradient information over all patterns of the pattern set ('batch learning').

It should be noted, that by replacing the by a constant update-value , equation () yields the so-called 'Manhattan'-update rule.

The second step of Rprop learning is to determine the new update-values . This is based on a sign-dependent adaptation process.

In words, the adaptation-rule works as follows: Every time the partial derivative of the corresponding weight changes its sign, which indicates that the last update was too big and the algorithm has jumped over a local minimum, the update-value is decreased by the factor . If the derivative retains its sign, the update-value is slightly increased in order to accelerate convergence in shallow regions. Additionally, in case of a change in sign, there should be no adaptation in the succeeding learning step. In practice, this can be achieved by setting in the above adaptation rule (see also the description of the algorithm in the following section).

In order to reduce the number of freely adjustable parameters, often leading to a tedious search in parameter space, the increase and decrease factor are set to fixed values ().

For Rprop tries to adapt its learning process to the topology of the error function, it follows the principle of 'batch learning' or 'learning by epoch'. That means, that weight-update and adaptation are performed after the gradient information of the whole pattern set is computed.

Niels.Mache@informatik.uni-stuttgart.de

Tue Nov 28 10:30:44 MET 1995