# Pruning Query

Hi,

i have a dataset which i have trained with ResilientPropagation mechanism.
Finally i generated the TruePostive,TrueNegative,FalsePostive,FalseNegative matrix.

Now i want to apply some pruning techniques where i can remove weakneurons
or the neurons which doesnt have any significane in the neural network.

In order to achieve this i have selected PruneSelective process where
after creating a neural network once the weight are assigned to the neurons then i sorted the "10 weakest neurons" and deleted from the hidden layers.

Result ::
Time taken to train for the same dataset has improved and True,False Matrix
is showing a 1% improved positive results .

Queries ::
How do i select number of weakest neurons to be deleted from the hidden layer
instead me saying "10" neurons to be deleted from the layer ??

Code Snippet ::

BasicNetwork network = new BasicNetwork();
network.getStructure().finalizeStructure();
network.reset();

PruneSelective prune = new PruneSelective(network);

//To find the 10 weakest neurons in Layer[1]
int[] weakestFLNeurons = findWeakestNeurons(network, 1, 10, prune);

//To find the 10 weakest neurons in Layer[2]
int[] weakestSLNeurons = findWeakestNeurons(network, 2, 10, prune);

//Pruning Layer[1]
for (int i=0; i<network.getLayerNeuronCount(1); i++) {
for(int j:weakestFLNeurons) {
if (i == j) {
prune.prune(1, i);
}
}
}
//Pruning Layer[2]
for (int i=0; i<network.getLayerNeuronCount(2); i++) {
for(int j:weakestFLNeurons) {
if (i == j) {
prune.prune(1, i);
}
}
}

/**
* Find the weakest neurons on a layer. Considers both weight and bias.
*
* @param layer
* The layer to search.
* @param count
* The number of neurons to find.
* @return An array of the indexes of the weakest neurons.
*/
private static int[] findWeakestNeurons(BasicNetwork network, final int layer, final int count, PruneSelective prune) {
// create an array to hold the least significant neurons, which will be
// returned
final double[] lostNeuronSignificance = new double[count];
final int[] lostNeuron = new int[count];

// init the potential lost neurons to the first ones, we will find
// better choices if we can
for (int i = 0; i < count; i++) {
lostNeuron[i] = i;
lostNeuronSignificance[i] = prune.determineNeuronSignificance(layer, i);
}

// now loop over the remaining neurons and see if any are better ones to
// remove
for (int i = count; i < network.getLayerNeuronCount(layer); i++) {
final double significance = prune.determineNeuronSignificance(layer, i);

// is this neuron less significant than one already chosen?
for (int j = 0; j < count; j++) {
if (lostNeuronSignificance[j] > significance) {
lostNeuron[j] = i;
lostNeuronSignificance[j] = significance;
break;
}
}
}

return lostNeuron;
}

Regards,
K.Kishore

Neural Network Forums:

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer