org.joone.engine.extenders
Class WeightDecayExtender

java.lang.Object
  extended by org.joone.engine.extenders.LearnerExtender
      extended by org.joone.engine.extenders.DeltaRuleExtender
          extended by org.joone.engine.extenders.WeightDecayExtender

public class WeightDecayExtender
extends DeltaRuleExtender

Weight decay adds a penalty term to the error function. The penalty term penalizes large weights. The weight decay penalty term causes the weights to converge to smaller absolute values than they otherwise would. Smaller weights are expected to improve generalization. The update formula is changed in: Dw(t+1) = dw(t+1) - d x w(t) d is a weight decay value.

Author:
boris

Constructor Summary
WeightDecayExtender()
          Creates a new instance of WeightDecayExtender
 
Method Summary
 double getDecay()
          Gets the decay parameter.
 double getDelta(double[] currentGradientOuts, int j, double aPreviousDelta)
          Computes the delta value for a bias.
 double getDelta(double[] currentInps, int j, double[] currentPattern, int k, double aPreviousDelta)
          Computes the delta value for a weight.
 void postBiasUpdate(double[] currentGradientOuts)
          Gives extenders a change to do some post-computing after the biases are updated.
 void postWeightUpdate(double[] currentPattern, double[] currentInps)
          Gives extenders a change to do some post-computing after the weights are updated.
 void preBiasUpdate(double[] currentGradientOuts)
          Gives extenders a change to do some pre-computing before the biases are updated.
 void preWeightUpdate(double[] currentPattern, double[] currentInps)
          Gives extenders a change to do some pre-computing before the weights are updated.
 void setDecay(double aDecay)
          Sets the decay parameter.
 
Methods inherited from class org.joone.engine.extenders.LearnerExtender
getLearner, isEnabled, setEnabled, setLearner
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

WeightDecayExtender

public WeightDecayExtender()
Creates a new instance of WeightDecayExtender

Method Detail

getDelta

public double getDelta(double[] currentGradientOuts,
                       int j,
                       double aPreviousDelta)
Description copied from class: DeltaRuleExtender
Computes the delta value for a bias.

Specified by:
getDelta in class DeltaRuleExtender
Parameters:
currentGradientOuts - the back propagated gradients.
j - the index of the bias.
aPreviousDelta - a delta value calculated by a previous delta extender.

getDelta

public double getDelta(double[] currentInps,
                       int j,
                       double[] currentPattern,
                       int k,
                       double aPreviousDelta)
Description copied from class: DeltaRuleExtender
Computes the delta value for a weight.

Specified by:
getDelta in class DeltaRuleExtender
Parameters:
currentInps - the forwarded input.
j - the input index of the weight.
currentPattern - the back propagated gradients.
k - the output index of the weight.
aPreviousDelta - a delta value calculated by a previous delta extender.

postBiasUpdate

public void postBiasUpdate(double[] currentGradientOuts)
Description copied from class: LearnerExtender
Gives extenders a change to do some post-computing after the biases are updated.

Specified by:
postBiasUpdate in class LearnerExtender
Parameters:
currentGradientOuts - the back propagated gradients.

postWeightUpdate

public void postWeightUpdate(double[] currentPattern,
                             double[] currentInps)
Description copied from class: LearnerExtender
Gives extenders a change to do some post-computing after the weights are updated.

Specified by:
postWeightUpdate in class LearnerExtender
Parameters:
currentPattern - the back propagated gradients.
currentInps - the forwarded input.

preBiasUpdate

public void preBiasUpdate(double[] currentGradientOuts)
Description copied from class: LearnerExtender
Gives extenders a change to do some pre-computing before the biases are updated.

Specified by:
preBiasUpdate in class LearnerExtender
Parameters:
currentGradientOuts - the back propagated gradients.

preWeightUpdate

public void preWeightUpdate(double[] currentPattern,
                            double[] currentInps)
Description copied from class: LearnerExtender
Gives extenders a change to do some pre-computing before the weights are updated.

Specified by:
preWeightUpdate in class LearnerExtender
Parameters:
currentPattern - the back propagated gradients.
currentInps - the forwarded input.

setDecay

public void setDecay(double aDecay)
Sets the decay parameter.

Parameters:
aDecay - the decay parameter value.

getDecay

public double getDecay()
Gets the decay parameter.

Returns:
the decay parameter.


Submit Feedback to pmarrone@users.sourceforge.net