Creating a Java Hopfield Neural Network

    The Hopfield neural network is implemented using two classes. The first class, called HopfieldNetwork is the main class that performs training and pattern recognition. This class relies on the Matrix and MatrixMath classes, introduced in chapter 2, to work with the neural network's weight matrix. The second class, called HopfieldException, is an exception that is raised when an error occurs while processing the Hopfield network. This is usually triggered as a result of bad input.

The HopfieldNetwork Class

    The HopfieldNetwork class is shown in Listing 3.1.

Listing 3.1: The Hopfield Neural Network (HopfieldNetwork.java)

/**
 * Introduction to Neural Networks with Java, 2nd Edition
 * Copyright 2008 by Heaton Research, Inc. 
 * http://www.heatonresearch.com/books/java-neural-2/
 * 
 * ISBN13: 978-1-60439-008-7  	 
 * ISBN:   1-60439-008-5
 *   
 * This class is released under the:
 * GNU Lesser General Public License (LGPL)
 * http://www.gnu.org/copyleft/lesser.html
 */
package com.heatonresearch.book.introneuralnet.neural.hopfield;

import com.heatonresearch.book.introneuralnet.neural.exception.NeuralNetworkError;
import com.heatonresearch.book.introneuralnet.neural.matrix.BiPolarUtil;
import com.heatonresearch.book.introneuralnet.neural.matrix.Matrix;
import com.heatonresearch.book.introneuralnet.neural.matrix.MatrixMath;

/**
 * HopfieldNetwork: This class implements a Hopfield neural network.
 * A Hopfield neural network is fully connected and consists of a 
 * single layer.  Hopfield neural networks are usually used for 
 * pattern recognition.    
 *  
 * @author Jeff Heaton
 * @version 2.1
 */
public class HopfieldNetwork {

	/**
	 * The weight matrix for this neural network. A Hopfield neural network is a
	 * single layer, fully connected neural network.
	 * 
	 * The inputs and outputs to/from a Hopfield neural network are always
	 * boolean values.
	 */
	private Matrix weightMatrix;

	public HopfieldNetwork(final int size) {
		this.weightMatrix = new Matrix(size, size);

	}

	/**
	 * Get the weight matrix for this neural network.
	 * 
	 * @return
	 */
	public Matrix getMatrix() {
		return this.weightMatrix;
	}

	/**
	 * Get the size of this neural network.
	 * 
	 * @return
	 */
	public int getSize() {
		return this.weightMatrix.getRows();
	}

	/**
	 * Present a pattern to the neural network and receive the result.
	 * 
	 * @param pattern
	 *            The pattern to be presented to the neural network.
	 * @return The output from the neural network.
	 * @throws HopfieldException
	 *             The pattern caused a matrix math error.
	 */
	public boolean[] present(final boolean[] pattern) {

		final boolean output[] = new boolean[pattern.length];

		// convert the input pattern into a matrix with a single row.
		// also convert the boolean values to bipolar(-1=false, 1=true)
		final Matrix inputMatrix = Matrix.createRowMatrix(BiPolarUtil
				.bipolar2double(pattern));

		// Process each value in the pattern
		for (int col = 0; col < pattern.length; col++) {
			Matrix columnMatrix = this.weightMatrix.getCol(col);
			columnMatrix = MatrixMath.transpose(columnMatrix);

			// The output for this input element is the dot product of the
			// input matrix and one column from the weight matrix.
			final double dotProduct = MatrixMath.dotProduct(inputMatrix,
					columnMatrix);

			// Convert the dot product to either true or false.
			if (dotProduct > 0) {
				output[col] = true;
			} else {
				output[col] = false;
			}
		}

		return output;
	}

	/**
	 * Train the neural network for the specified pattern. The neural network
	 * can be trained for more than one pattern. To do this simply call the
	 * train method more than once.
	 * 
	 * @param pattern
	 *            The pattern to train on.
	 * @throws HopfieldException
	 *             The pattern size must match the size of this neural network.
	 */
	public void train(final boolean[] pattern) {
		if (pattern.length != this.weightMatrix.getRows()) {
			throw new NeuralNetworkError("Can't train a pattern of size "
					+ pattern.length + " on a hopfield network of size "
					+ this.weightMatrix.getRows());
		}

		// Create a row matrix from the input, convert boolean to bipolar
		final Matrix m2 = Matrix.createRowMatrix(BiPolarUtil
				.bipolar2double(pattern));
		// Transpose the matrix and multiply by the original input matrix
		final Matrix m1 = MatrixMath.transpose(m2);
		final Matrix m3 = MatrixMath.multiply(m1, m2);

		// matrix 3 should be square by now, so create an identity
		// matrix of the same size.
		final Matrix identity = MatrixMath.identity(m3.getRows());

		// subtract the identity matrix
		final Matrix m4 = MatrixMath.subtract(m3, identity);

		// now add the calculated matrix, for this pattern, to the
		// existing weight matrix.
		this.weightMatrix = MatrixMath.add(this.weightMatrix, m4);

	}
}

    To make use of the Hopfield neural network, you should instantiate an instance of this class. The constructor takes one integer parameter that specifies the size of the neural network. Once the HopfieldNetwork class has been instantiated, you can call the provided methods to use the neural network. These methods are summarized in Table 3.3.

Table 3.3: Summary of HopfieldNetwork Methods

    The getMatrix and getSize methods are both very simple accessor methods and will not be covered further. Most of the work is done with the present and train methods. These methods will be covered in the next two sections.

Recalling Patterns with the Java Hopfield Network

    To recall a pattern with the HopfieldNetwork class, the present method should be used. The signature for this method is shown here:

public boolean[] present(final boolean[] pattern) throws HopfieldException

    The procedure for training a Hopfield neural network was already discussed earlier in this chapter. The earlier discussion explained how to recall a pattern mathematically. Now we will see how to implement Hopfield pattern recollection in Java.

    First, an array is created to hold the output from the neural network. This array is the same length as the input array.

final boolean output[] = new boolean[pattern.length];

    Next, a Matrix is created to hold the bipolar form of the input array. The bipolar2double method of the BiPolarUtil class is used to convert the boolean array. The one-dimensional array is converted into a “row matrix.” A “row matrix” is a Matrix that consists of a single row.

final Matrix inputMatrix = Matrix.createRowMatrix(BiPolarUtil
				.bipolar2double(pattern));

    We must now loop through each item in the input array. Each item in the input array will be applied to the weight matrix to produce an output.

for (int col = 0; col < pattern.length; col++) {

    Each column in the weight matrix represents the weights associated with the connections between the neuron and the other neurons. Therefore, we must extract the column that corresponds to each of the input array values.

  Matrix columnMatrix = this.weightMatrix.getCol(col);

    We must now determine the dot product of that column and the input array. However, before that can be done we must transpose the column from the weight matrix. This will properly orient the values in the column so the dot product computation can be performed.

  columnMatrix = MatrixMath.transpose(columnMatrix);

    Next, the dot product is calculated.

  final double dotProduct = 
    MatrixMath.dotProduct(inputMatrix, columnMatrix);

    If the dot product is above zero, then the output will be true; otherwise the output will be false.

    if (dotProduct > 0) {
      output[col] = true;
    } else {
      output[col] = false;
    }
  }

    Zero is the threshold. A value above the threshold will cause the output neuron to fire; at or below the threshold and the output neuron will not fire. It is important to note that since the Hopfield network has a single layer, the input and output neurons are the same. Thresholds will be expanded upon in the next chapter when we deal with the feedforward neural network.

    Finally, the output array is returned.

return output;

    The next section will explain how the Java Hopfield network is trained.

Training the Hopfield Network

    The train method is used to train instances of the HopfieldNetwork class. The signature for this method is shown here:

public void train(final boolean[] pattern) throws HopfieldException 

    The length of the pattern must be the same as the size of the neural network. If it is not, then an exception is thrown.

if (pattern.length != this.weightMatrix.getRows()) {
  throw new HopfieldException("Can't train a pattern of size "
  + pattern.length + " on a Hopfield network of size "
  + this.weightMatrix.getRows());
}

    If the pattern is the proper length, a matrix is created that contains a single row. This row will contain the bipolar representation of the input pattern.

final Matrix m2 = Matrix.createRowMatrix(BiPolarUtil
					.bipolar2double(pattern));

    The row matrix is then transposed into a column matrix.

final Matrix m1 = MatrixMath.transpose(m2);

    Finally, the row is multiplied by the column.

final Matrix m3 = MatrixMath.multiply(m1, m2);

    Multiplying the row by the column results in a square matrix. This matrix will have a diagonal of ones running from its northwest corner to its southwest corner. The ones must be converted to zeros. To do this, an identity matrix of the same size is created.

final Matrix identity = MatrixMath.identity(m3.getRows());

    The identity matrix is nothing more than a matrix containing a diagonal of ones, which can be subtracted from the pervious matrix to set the diagonal to zero.

final Matrix m4 = MatrixMath.subtract(m3, identity);

    The new matrix is now added to the old weight matrix.

this.weightMatrix = MatrixMath.add(this.weightMatrix, m4);

    This produces a weight matrix that will likely recognize the new pattern, as well as the old patterns.