Hi,
i have a dataset which i have trained with ResilientPropagation mechanism.
Finally i generated the TruePostive,TrueNegative,FalsePostive,FalseNegative matrix.
Now i want to apply some pruning techniques where i can remove weakneurons
or the neurons which doesnt have any significane in the neural network.
In order to achieve this i have selected PruneSelective process where
after creating a neural network once the weight are assigned to the neurons then i sorted the "10 weakest neurons" and deleted from the hidden layers.
Result ::
Time taken to train for the same dataset has improved and True,False Matrix
is showing a 1% improved positive results .
Queries ::
How do i select number of weakest neurons to be deleted from the hidden layer
instead me saying "10" neurons to be deleted from the layer ??
Code Snippet ::
BasicNetwork network = new BasicNetwork();
network.addLayer(inputLayer);
network.addLayer(hiddenLayer1);
network.addLayer(hiddenLayer2);
network.addLayer(outputLayer);
network.getStructure().finalizeStructure();
network.reset();
PruneSelective prune = new PruneSelective(network);
//To find the 10 weakest neurons in Layer[1]
int[] weakestFLNeurons = findWeakestNeurons(network, 1, 10, prune);
//To find the 10 weakest neurons in Layer[2]
int[] weakestSLNeurons = findWeakestNeurons(network, 2, 10, prune);
//Pruning Layer[1]
for (int i=0; i<network.getLayerNeuronCount(1); i++) {
for(int j:weakestFLNeurons) {
if (i == j) {
prune.prune(1, i);
}
}
}
//Pruning Layer[2]
for (int i=0; i<network.getLayerNeuronCount(2); i++) {
for(int j:weakestFLNeurons) {
if (i == j) {
prune.prune(1, i);
}
}
}
/**
* Find the weakest neurons on a layer. Considers both weight and bias.
*
* @param layer
* The layer to search.
* @param count
* The number of neurons to find.
* @return An array of the indexes of the weakest neurons.
*/
private static int[] findWeakestNeurons(BasicNetwork network, final int layer, final int count, PruneSelective prune) {
// create an array to hold the least significant neurons, which will be
// returned
final double[] lostNeuronSignificance = new double[count];
final int[] lostNeuron = new int[count];
// init the potential lost neurons to the first ones, we will find
// better choices if we can
for (int i = 0; i < count; i++) {
lostNeuron[i] = i;
lostNeuronSignificance[i] = prune.determineNeuronSignificance(layer, i);
}
// now loop over the remaining neurons and see if any are better ones to
// remove
for (int i = count; i < network.getLayerNeuronCount(layer); i++) {
final double significance = prune.determineNeuronSignificance(layer, i);
// is this neuron less significant than one already chosen?
for (int j = 0; j < count; j++) {
if (lostNeuronSignificance[j] > significance) {
lostNeuron[j] = i;
lostNeuronSignificance[j] = significance;
break;
}
}
}
return lostNeuron;
}
Regards,
K.Kishore
Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer