Can you restrict neuron connections in encog?

jimmyjones's picture

Let's say I have 4 input neurons (i1-4) and a hidden layer with 2 neurons (h1,h2)

Is there any way I can connect i1 and i3 to H1 and i2 and i4 to H2 and avoid all other connection between the input and the hidden layer?

jeffheaton's picture

That is a new feature that I have added in Encog 2.4, the newest version which is still in beta test. Its supported both through the work bench and using the core.

I have not done an example yet, but I have tested it in the workbench and it works okay. You start with a fully connected network and then can remove connections between neurons. You can also instruct Encog to remove any connection below a certain value. I will be adding an example in the next few days.

Jeff

SeemaSingh's picture

I created an example where I remove several connections in a neural network. I then train it, and the removed connections do not come back the network is retrained around the missing connections. It is checked into the 2.4 examples. You can see it here as well.


package org.encog.examples.neural.xorpartial;

import org.encog.neural.data.NeuralDataSet;
import org.encog.neural.data.basic.BasicNeuralDataSet;
import org.encog.neural.networks.BasicNetwork;
import org.encog.neural.networks.layers.Layer;
import org.encog.neural.networks.synapse.Synapse;
import org.encog.util.logging.Logging;
import org.encog.util.simple.EncogUtility;

/**
* Partial neural networks. Encog allows you to remove any neuron connection in
* a fully connected neural network. This example creates a 2x10x10x1 neural
* network to learn the XOR. Several connections are removed prior to training.
*/
public class XORPartial {

public static double XOR_INPUT[][] = { { 0.0, 0.0 }, { 1.0, 0.0 },
{ 0.0, 1.0 }, { 1.0, 1.0 } };

public static double XOR_IDEAL[][] = { { 0.0 }, { 1.0 }, { 1.0 }, { 0.0 } };

public static void main(final String args[]) {

Logging.stopConsoleLogging();

BasicNetwork network = EncogUtility.simpleFeedForward(2, 10, 10, 1,
false);
network.reset();

// obtain some of the synapses that we wish to remove connections from
Layer inputLayer = network.getLayer(BasicNetwork.TAG_INPUT);
Synapse inputToHidden1 = inputLayer.getNext().get(0);
Synapse hidden1ToHidden2 = inputToHidden1.getToLayer().getNext().get(0);
Synapse hidden2ToOutput = inputToHidden1.getToLayer().getNext().get(0);

// remove the connection from input neuron 0 to hidden1 neuron 1.
network.enableConnection(inputToHidden1, 0, 1, false);

// remove the connection from hidden1 neuron 2 to hidden2 neuron 3.
network.enableConnection(hidden1ToHidden2, 2, 3, false);

// remove the connection from hidden2 neuron 3 to output neuron 4.
network.enableConnection(hidden2ToOutput, 3, 4, false);

NeuralDataSet trainingSet = new BasicNeuralDataSet(XOR_INPUT, XOR_IDEAL);

EncogUtility.trainToError(network, trainingSet, 0.01);

System.out
.println("Training should leave hidden neuron weights at zero.");
System.out.println("First removed neuron weight:"
+ inputToHidden1.getMatrix().get(0, 1));
System.out.println("First removed neuron weight:"
+ hidden1ToHidden2.getMatrix().get(2, 3));
System.out.println("First removed neuron weight:"
+ hidden2ToOutput.getMatrix().get(3, 4));
System.out.println("Final output:");
EncogUtility.evaluate(network, trainingSet);
}
}

John Merk's picture

Hello Seema,

I believe this example, which appears in 3.0, needs to be modified to work for 3.0 (maybe that is why it comes commented out). In particular, the signature is now enableConnection(int, int, int, boolean), instead of enableConnection(Synapse, int, int, boolean) (from 2.5).

My question is, how are the neurons numbered? The first int is for the layer, the next two are for neurons. Does the numbering start over at each layer, w/ the neurons numbered 0,1,2,...? And I assume the 3rd parameter refers to a neuron in the next layer, correct?

That raises another question. Is there any way to connect neurons from layer 2, to neurons in layer 5 (i.e. skipping one or several layers)?

Thanks,

John


Copyright 2005 - 2012 by Heaton Research, Inc.. Heaton Research™ and Encog™ are trademarks of Heaton Research. Click here for copyright, license and trademark information.