public class Nakayama extends java.lang.Object implements NeuralNetListener, NeuralValidationListener, ConvergenceListener, java.io.Serializable
K.Nakayama and Y.Kimura, "Optimization of activation functions in multilayer neural network applied to pattern classification", Proc. IEEE ICNN'94 Florida, pp.431-436, June 1994.
This techniques probably fails whenever the NeuralNet.join()
method
is called because this optimization technique stops the network to perform the
optimization, use a NeuralNetListener
instead.
Constructor and Description |
---|
Nakayama(NeuralNet aNet)
Creates a new instance of Nakayama.
|
Modifier and Type | Method and Description |
---|---|
void |
addLayer(Layer aLayer)
Adds layers to this optimizer.
|
void |
addLayers(NeuralNet aNeuralNet)
Adds all the hidden layers to this optimizer.
|
void |
cicleTerminated(NeuralNetEvent e) |
protected void |
cleanUp()
This method is called after optimization, e.g.
|
protected void |
doOptimize()
Optimizes the activation functions of the network.
|
void |
errorChanged(NeuralNetEvent e) |
protected void |
evaluateNeurons()
Evaluates neurons, that is, this function calculates information related to
the contribution of the activation functions, based on the following three
criteria:
Information from neurons to its output layers. |
protected int[] |
findCorrelation(java.util.List aCorrelations,
int aLayer,
int aNeuron)
Finds a correlation from a given list of correlations.
|
protected int |
findIndex(java.util.List aStatuses,
int aLayer,
int aNeuron)
Finds the index of a neuron taking into account the deletion of previous neurons.
|
protected Layer |
findInputLayer(Synapse aSynapse)
Finds the input layer of a synapse.
|
protected Layer |
findOutputLayer(Synapse aSynapse)
Finds the output layer of a synapse.
|
double |
getEpsilon()
Gets epsilon, the threshold to decide if a neuron should be deleted or not.
|
protected double |
getGamma(int aLayer1,
int aNeuron1,
int aLayer2,
int aNeuron2)
Gets gammajj' (is equal to gammaj'j).
|
protected double[] |
getMinCorrelation(int aLayer,
int aNeuron)
Gets the minimum correlation for a certain neuron j.
|
protected double |
getSumAbsoluteWeights(Layer aLayer,
int aNeuron)
Sums up all the absolute values of the output weights of a neuron within a layer.
|
protected double[] |
getSumOutputs(int aLayer,
int aNeuron)
Sums up the (normal and absolute) values of the outputs of a neuron over all patterns.
|
void |
netConverged(ConvergenceEvent anEvent,
ConvergenceObserver anObserver)
This method is called whenever the network has converged according to some
ConvergenceObserver |
void |
netStarted(NeuralNetEvent e) |
void |
netStopped(NeuralNetEvent e) |
void |
netStoppedError(NeuralNetEvent e,
java.lang.String error) |
void |
netValidated(NeuralValidationEvent event) |
boolean |
optimize()
Optimizes the activation functions of the neural network.
|
(package private) void |
patternFinished()
This method is called after every pattern, so we can retrieve information
from the network that is related to the pattern that was just forwarded
through the network.
|
protected void |
removeAllListeners()
Removes all the listeners from the neural network (temporarely).
|
protected void |
removeNeuron(int aLayer,
int aNeuron)
Removes a neuron.
|
protected void |
restoreAllListeners()
Restore all the listeners to the neural network.
|
protected void |
runValidation()
Runs the network with a validator, this way we are able to collect information
related to the different patterns.
|
protected void |
selectNeurons()
Selects neurons to be deleted based on the information calculated by
evaluateNeurons() . |
void |
setEpsilon(double anEpsilon)
Sets epsilon, the threshold to decide if a neuron should be deleted or not.
|
protected void |
weightsUpdateCorrelation(java.util.List aStatuses,
java.util.List aCorrelations,
int aLayer,
int aNeuron)
Updates weights before a neuron is removed (because of its similar correlation).
|
protected void |
weightsUpdateVariance(int aLayer,
int aNeuronOriginal,
int aNeuron)
Updates weights before a neuron is removed (because of its low variance).
|
public Nakayama(NeuralNet aNet)
aNet
- the network to be optimized.public void addLayer(Layer aLayer)
aLayer
- the layer to be added.public void addLayers(NeuralNet aNeuralNet)
aNeuralNet
- the network holding the hidden layers.public boolean optimize()
protected void runValidation()
patternFinished()
.protected void doOptimize()
protected void cleanUp()
protected void selectNeurons()
evaluateNeurons()
. It selects neurons to be deleted and
performs the deletion.protected void weightsUpdateCorrelation(java.util.List aStatuses, java.util.List aCorrelations, int aLayer, int aNeuron)
aStatuses
- the status of the neurons (used to find the correct neuron taking into account
any deletions of neurons).aCorrelations
- a list holding all the correlations (neurons to be removed and the
correlated neuron (which will take over the weights)).aLayer
- the layer of the neuron to be removed.aNeuron
- the neuron to be removed.protected double getGamma(int aLayer1, int aNeuron1, int aLayer2, int aNeuron2)
aLayer1
- the layer of neuron j.aNeuron1
- the neuron j.aLayer2
- the layer of neuron j'.aNeuron2
- the neuron j'.
return gammajj'.protected int findIndex(java.util.List aStatuses, int aLayer, int aNeuron)
aStatuses
- the status of neurons.aLayer
- the layer of the neuron.aNeuron
- the index of the neuron, not considering any deletions.protected int[] findCorrelation(java.util.List aCorrelations, int aLayer, int aNeuron)
aCorrelations
- a list holding correlations.aLayer
- the layer of the neuron to find the correlation for.aNeuron
- the neuron to find the correlation for.(aLayer, aNeuron)
,
return null
in case the correlation is not found.protected void weightsUpdateVariance(int aLayer, int aNeuronOriginal, int aNeuron)
aLayer
- the index of the layer of the neuron.aNeuronOriginal
- the index of the neuron to be removed (NOT taking into account
previous deletions).aNeuron
- the index of the neuron to be removed (taking into account previous
deletions).protected void removeNeuron(int aLayer, int aNeuron)
aLayer
- the index of the layer in which we should remove the neuron.aNeuron
- the index of the neuron to be removed (taking into account previous
deletions).protected Layer findInputLayer(Synapse aSynapse)
aSynapse
- the synapse to find the input layer for.protected Layer findOutputLayer(Synapse aSynapse)
aSynapse
- the synapse to find the output layer for.protected void evaluateNeurons()
protected double[] getMinCorrelation(int aLayer, int aNeuron)
anLayer
- the index of the layer of the neuron aNeuron
.aNeuron
- the neuron within the layer (j).protected double[] getSumOutputs(int aLayer, int aNeuron)
aLayer
- an index of the layer to retrieve the outputs of that layer.aNeuron
- the neuron in the layer.aNeuron
.protected double getSumAbsoluteWeights(Layer aLayer, int aNeuron)
aLayer
- the layer holding neuron aNeuron
.aNeuron
- the neuron in the layer.aNeuron
within layer aLayer
.public void cicleTerminated(NeuralNetEvent e)
cicleTerminated
in interface NeuralNetListener
public void errorChanged(NeuralNetEvent e)
errorChanged
in interface NeuralNetListener
public void netStarted(NeuralNetEvent e)
netStarted
in interface NeuralNetListener
public void netStopped(NeuralNetEvent e)
netStopped
in interface NeuralNetListener
public void netStoppedError(NeuralNetEvent e, java.lang.String error)
netStoppedError
in interface NeuralNetListener
public void netValidated(NeuralValidationEvent event)
netValidated
in interface NeuralValidationListener
protected void removeAllListeners()
protected void restoreAllListeners()
void patternFinished()
public double getEpsilon()
public void setEpsilon(double anEpsilon)
anEpsilon
- the new epsilon.public void netConverged(ConvergenceEvent anEvent, ConvergenceObserver anObserver)
ConvergenceListener
ConvergenceObserver
netConverged
in interface ConvergenceListener
anEvent
- the event that is generated.anObserver
- the observer that generated the event.Submit Feedback to pmarrone@users.sourceforge.net