Get the entire book!

We will now examine a simple program that trains a Kohonen neural network. As the network is trained you are shown a graphical display of the weights. The output from this program is shown in Figure 6.4.

Figure 6.4: Training a Kohonen neural network

You will this program contains two input neurons and seven output neurons. Each of the seven output neurons are plotted as white squares. The x-dimension shows their weight to the first input neuron and the y-dimension shows their weight to the second input neuron. You will see the boxes move as training progresses.

You will also see lines from select points on a grid drawn to each of the squares. These identify which output neuron is winning for the x and y coordinates of that point. You will see points with similar x and y coordinates being recognized by the same output neuron.

We will now examine the example program. The program begins by defining certain properties that will be used by the program. This is shown in Listing 6.18.

Listing 6.18: Properties of the Kohonen example (TestKohonen.java)

import java.awt.*;
import javax.swing.*;
import java.text.*;

public class TestKohonen 
  extends JFrame 
  implements NeuralReportable,Runnable {

  public static final int INPUT_COUNT=2;
  public static final int OUTPUT_COUNT=7;
  public static final int SAMPLE_COUNT=100;
  protected int unitLength;
  protected int retry=1;
  protected double totalError=0;
  protected double bestError = 0;
  protected KohonenNetwork net;
  protected TrainingSet ts;
  protected Image offScreen;

You can modify some of the final constants. You can choose s lager training set by setting the SAMPLE_COUNT variable to other values. You can change the OUTPUT_COUNT to values other than 7 as well. You may not change the INPUT_COUNT due to the fact that the x and y coordinates are tied to the inputs. Since you only have two dimensions you may not increase or decrease this value.

A constructor is provided to create the window and center it on the screen. This constructor is shown in Listing 6.19.

Listing 6.19: Construct the example (TestKohonen.java)

  /**
   * The constructor sets up the position and size of
   * the window.
   */
  TestKohonen()
  {
    setTitle("Training a Kohonen Neural Network");
    setSize(400,450);
    Toolkit toolkit = Toolkit.getDefaultToolkit();
    Dimension d = toolkit.getScreenSize();
    setLocation(
               (int)(d.width-this.getSize().getWidth())/2,
               (int)(d.height-this.getSize().getHeight())/2 );
    setDefaultCloseOperation(WindowConstants.DISPOSE_ON_CLOSE);
    setResizable(false);
  }

The update method is provided because we implement the NeuralReportable interface. The update method will be called for each epoch during training. Here we record the status values and requires that the window be redrawn. This is shown in Listing 6.20.

Listing 6.20: Receive status information during training (TestKohonen.java)

  /**
   * Update is called by the neural network as the
   * network is trained.
   * 
   * @param retry What retry number this is.
   * @param totalError The error for this retry.
   * @param bestError The best error so far.
   */
  public void update(int retry,double totalError,double bestError)
  {   
    this.retry = retry;
    this.totalError = totalError;
    this.bestError = bestError;
    this.paint(null);
  }

This example is a multithreaded program. A background thread is used to train the neural network. Multithreading was discussed in much greater detail in Chapter 5. The background thread is shown in Listing 6.21.

Listing 6.21: The background thread (TestKohonen.java)

  /**
   * Called to run the background thread. The background thread
   * sets up the neural network and training data and begins
   * training the network.
   */
  public void run()
  {                
    // build the training set
    ts = new TrainingSet(INPUT_COUNT,OUTPUT_COUNT);
    ts.setTrainingSetCount(SAMPLE_COUNT);

    for ( int i=0;i<SAMPLE_COUNT;i++ ) {
      for ( int j=0;j<INPUT_COUNT;j++ ) {
        ts.setInput(i,j,Math.random());
      }
    }

    // build and train the neural network
    net = new KohonenNetwork(INPUT_COUNT,OUTPUT_COUNT,this);
    net.setTrainingSet(ts);
    net.learn();
  }       

The first thing that the run method does is create random training sets. This will give the neural network a random sampling of training items that it must attempt to classify. Next the learn method of the neural network is called to begin the training process.

As the program runs a graph must be displayed of the neuron weights. Listing 6.22 shows the method that is responsible for drawing this graph.

Listing 6.22: Graph the neuron weights (TestKohonen.java)

  /**
   * Display the progress of the neural network.
   * 
   * @param g A graphics object.
   */
  public void paint(Graphics g)
  {
    if ( net==null )
      return;
    if ( offScreen==null ) {
      offScreen = this.createImage(
        (int)getBounds().getWidth(),
        (int)getBounds().getHeight());
    }
    g = offScreen.getGraphics();
    int width = (int)getContentPane().bounds().getWidth();
    int height = (int)getContentPane().bounds().getHeight();
    unitLength = Math.min(width,height);
    g.setColor(Color.black);
    g.fillRect(0,0,width,height);

    // plot the weights of the output neurons
    g.setColor(Color.white);
    for ( int y=0;y<net.outputWeights.length;y++ ) {

      g.fillRect((int)(net.outputWeights[y][0]*unitLength),
                 (int)(net.outputWeights[y][1]*unitLength),10,10);

    }

    // plot a grid of samples to test the net with
    g.setColor(Color.green);
    for ( int y=0;y<unitLength;y+=50 ) {
      for ( int x=0;x<unitLength;x+=50 ) {
        g.fillOval(x,y,5,5);
        double d[] = new double[2];
        d[0]=x;
        d[1]=y;
        double normfac[] = new double[1];
        double synth[] = new double[1];
        int c = net.winner(d,normfac,synth);


        int x2=(int)(net.outputWeights[c][0]*unitLength);
        int y2=(int)(net.outputWeights[c][1]*unitLength);

        g.drawLine(x,y,x2,y2);
      }

    }

    // display the status info
    g.setColor(Color.white);
    NumberFormat nf = NumberFormat.getInstance();
    nf.setMaximumFractionDigits(2);
    nf.setMinimumFractionDigits(2);
    g.drawString(
                "retry = " 
                + retry 
                + ",current error = " 
                + nf.format(totalError*100)
                +  "%, best error = " 
                + nf.format(bestError*100)
                +"%", 0, 
                (int)getContentPane().getBounds().getHeight());
    getContentPane().getGraphics().drawImage(offScreen,0,0,this);

  }

First an off-screen image is created using the createImage method provided by Java. All of our drawing will be to this off-screen image. Once the graph is complete, the off-screen image will be drawn to the window. This prevents flicker as the program constantly redraws itself.

First the output neurons are drawn as white rectangles. The x and y coordinates for these neurons are obtained from the weight between the output neuron and one of the two input neurons.

Next a grid of samples is drawn onto the screen. This is just a typical grid with each dot being 50 pixels from the other dots. Lines are then drawn from each of these dots to the winning neuron when the network is presented with the x and y coordinates from this grid dot.

To start this program a simple main method is provided as seen in Listing 6.22.

Listing 6.22: Start the program (TestKohonen.java)

  /**
   * Startup the program.
   * 
   * @param args Not used.
   */
  public static void main(String args[])
  {
    TestKohonen app = new TestKohonen();
    app.show();
    Thread t = new Thread(app);
    t.setPriority(Thread.MIN_PRIORITY);
    t.start();
  }
}

This main method begins by displaying the window. Then a background thread is created.