Get the entire book!

Before we explain how to implement the feed forward back propagation neural network in Java it is important to understand how threads work in Java. Many programmers are have not worked with threads a great deal. The JOONE neural network is designed to make extensive use of threading. This allows JOONE to achieve great speed advantages on a multi-processor computer, as the task is already divided for the CPU's.

Threads can enhance performance and functionality in Java, by allowing a program to perform multiple tasks simultaneously. We will now take a look at the implementation of threads using Java, and offer a step-by-step overview of the fundamentals needed to incorporate threads into a Java program.

First we must examine what exactly a thread is. Basically, a thread is a path of execution through the program. Most programs written today run as a single thread, and thus have just one single path of execution. This can cause problems when multiple events or actions need to occur at the same time. For example, single threaded program is not capable downloading data from the network while responding to mouse or keyboard input from the user. The single threaded program must give its full attention to either the keyboard input or the network download. The ideal solution to this problem is the seamless execution of two or more sections of a program at the same time. This is the problem that threads were designed to solve.

Multithreaded applications are capable of running many threads concurrently within a single program. From a logical point of view, multithreading means multiple lines of a single program can be executed at the same time. This is not the same as starting a program twice and saying that there are multiple lines of a program being executed at the same time. In the case of simply running a program twice, the operating system is treating the programs as two separate and distinct processes. Take for example, the Unix forking process. UNIX creates a child process with a different address space for both code and data when the fork() function is evoked. However, fork() creates a lot of overhead for the operating system, making it a very CPU-intensive operation. By starting a thread instead, an efficient path of execution is created while still sharing the original data area from the parent. The idea of sharing the data area is very beneficial, but brings up some areas of concern that we'll discuss later in this chapter.

Creating threads

Java was designed to allow threads to be created in two ways. To create a Java thread you must either implement the Runnable interface or extend the Thread class. When a Java class is extended, the new class inherits methods and variables from a parent class. Java only allows a class to extend from one parent. To overcome this limitation, interfaces are used. This brings us to the second, more common way, that threads are created. By implementing the Runnable interface any class can act as a thread.

Interfaces provide a template that provides a way for programmers to lay the groundwork of a class. They are used to design the requirements for a set of classes to implement. The interface sets everything up, and the class or classes that implement the interface do all the work. The different set of classes that implement the interface has to follow the same rules.

There are several differences between a class and an interface. An interface can only contain abstract methods and/or static final variables (constants). Classes, on the other hand, can implement methods and contain variables that are not constants. An interface may extend from other interfaces, but not other classes. An interface may (unlike classes) extend from multiple interfaces. Finally an interface cannot be instantiated with the new operator; for example, Runnable a=new Runnable(); is not allowed.

The first method of creating a thread is to simply extend from the Thread class. This method should only be used if this new class does not need to extend from another class. Because Java classes can only extend from one class the thread class is the ONLY class you will be extending from when using this method. The Thread class is defined in the java.lang package so it does not need to be imported (java.lang is always automatically imported by Java).

public class MyThread extends Thread 
{ 
  public void run() 
  { 
    // perform actual thread work here
  } 
} 

The above example creates a new class named MyThread that extends the Thread class and overrides the Thread.run() method for its own implementation. The run() method is where all the work of the Counter class thread is done. This same functionality could also be achieved by implementing the Runnable interface.

public class MyRunnable implements Runnable 
{ 
  Thread T; 
  public void run() 
  { 
    // perform actual thread work here
  } 
} 

In the above example, the abstract run() method is defined in the Runnable interface and is being implemented. We have an instance of the Thread class as a variable of the MyRunnable class. The only difference between the two methods is that by implementing Runnable, there is greater flexibility in the creation of the class Counter. In the above example, the opportunity still exists to extend the Counter class, if needed. The majority of classes created that need to be run as a thread will implement Runnable since they probably are extending some other functionality from another class.

The Runnable interface is not doing any real work when the thread is being executed. It is merely a class to hold the code to be executed by the thread. The Runnable interface is very small and contains only one abstract method. The following is the definition of the Runnable interface directly from the Java source:

public interface Runnable 
{ 
  public abstract void run(); 
} 

These few lines of code are all that there is to the Runnable interface. An interface provides only a foundation upon which classes should be implemented. In the case of the Runnable interface, it forces the definition of only the run() method.

Controlling the Thread’s Execution

Now that we have examined the different ways to create an instance of a thread, we now will discuss the ways available to start and stop a Thread. We will also examine a short example program that uses several threads that remain synchronized. To cause a thread to begin execution the start method must be called. To cause a thread to cease execution, the top method must be called.

public static void main(String args[])
{
  MyRunnable run = new MyRunnable();
  Thread t = new Thread(run);
  t.start();
}

Once the above code is called a new thread is completed that will begin by executing the run method of the MyRunnable class. Once this method is running it is important to sleep somewhere in a thread. If not, the thread may consume all CPU time for the process and may not allow any other threads to be executed. Another way to cease the execution of a thread is to call the stop() method.

It is not generally recommended to stop threads. It is better to let the thread stop itself by allowing the run method to return.

Suspending and Resuming Threads

Once a thread’s stop() method has been called, it cannot be restarted with the start() command. Instead you can pause the execution of a thread with the sleep() method. The thread will sleep for a certain period of time and then begin executing when the time limit is reached. But, this is not ideal if the thread needs to be started when a certain event occurs. In this case, the suspend() method allows a thread to temporarily cease executing and the resume() method allows the suspended thread to start again.

To keep track of the current state of the applet, the boolean variable suspended is used. Distinguishing the different states of an applet is important because some methods will throw exceptions if they are called while in the wrong state. For example, if the application has been started and stopped, executing the start() method will throw an IllegalThreadStateException exception.

Thread Scheduling

The Java Thread Scheduler monitors all running threads in all programs and decides which threads should be running and which are in line to be executed. There are two characteristics of a thread that the scheduler identifies in its decision process. The first, and most important, is the priority of the thread. The other, and less used, is the daemon flag. The scheduler's basic rule is if there are only daemon threads running, the Java Virtual Machine (JVM) will exit. New threads inherit the priority and daemon flag from the thread that created it. The scheduler determines which thread should be executed by analyzing the priorities of all threads. Those with the highest priority are allowed execution before any lower priority threads.

There are two types of thread scheduling algorithms commonly used, preemptive or non-preemptive. Preemptive schedulers give a certain time-slice to all threads running on the system. The scheduler decides which thread is next to run and resume() that thread for some constant period of time. When the thread has executed for that time period it will be suspended() and the next thread scheduled will be resumed(). Non-preemptive schedulers decide which thread should run and run it until the thread is complete. The thread has full control of the system for as long as it likes. The yield() method is a way for a thread to force the scheduler to start executing another waiting thread. Unfortunately Java uses both algorithms, depending on the system Java is running on.

Thread priority determines what percent of CPU time one thread should get relative to another thread. This is based on a priority number assigned to each thread. The range of priorities is from 1 to 10. The default priority of a thread is Thread.NORM_PRIORITY, which is assigned the value of 5. Two other static variables are made available; they are Thread.MIN_PRIORITY, which is set to 1, and Thread.MAX_PRIORITY, which is set to 10. The getPriority() method can be used to find the current value of the priority of a thread.

Daemon threads run at a low priority in the background. Daemon threads are sometimes called “service” threads, as they provide a basic service to a program or programs when activity on a machine is reduced. An example of a daemon thread that is continuously running is the garbage collector thread. This thread, provided by the JVM, will scan programs for variables that will never be accessed again and free up their resources back to the system. A thread can set the daemon flag by passing a true boolean value to the setDaemon() method. If a false boolean value is passed, the thread will become a user thread. However, this must occur before the thread has been started.

Synchronizing Threads

Up to this point we have only talked about independent, asynchronous threads. That is, each thread contained all of the data and methods required for its execution. Threads of this type don’t require any outside resources or methods. Threads of this nature may run at their own pace without concern over the state or activities of any other threads that may be running in the background.

This is not usually the case. Threads must usually work with other related threads. In this case these concurrent threads must share data and must consider the state and activities of other threads. The threads used to create a the neural network are of this type. The work of looking many web pages is broken down into smaller sub tasks that will be given to individual threads. These threads must communicate with each other to ensure that new work is obtained and no new work duplicates work already completed.

Java provides several mechanisms to facilitate this thread synchronization. Most Java synchronization centers around object locking. Every object in Java that descends from the object named Object has an individual lock. And since every object in Java must descend from object, every object in Java has its own lock.

The code segments within a program that access the same object from separate, concurrent threads are called critical sections. In the Java language, a critical section can be a block or a method and are identified using the synchronized keyword. Java associates a lock with every object that contains code that uses the synchronized keyword.

Generally the put and get methods of an object are ideal candidates for the synchronized keyword. This is because the get method reads the internal state of any object and the set method changes the internal state of an object. You do not want the state changing right in the middle of a get operation. Likewise, you do not want the state to change while another thread is changing the state with the set operation. The following code shows just such an example.

public class MySynchronizedObject 
{
  int myInt;

  public synchronized int getMyInt() 
  {
    return myInt;
  }

  public synchronized void putMyInt(int value) 
  {
    myInt = value;
  }
}

The method declarations for both putMyInt() and getMyInt() make use of the synchronized keyword. As a result, the system creates a unique lock with every instantiation of MySynchronizedObject. Whenever control enters a synchronized method, the thread that called the method locks the object whose method has been called. Other threads cannot call a synchronized method on the same object until the object is unlocked.

Once a thread calls either putMyInt() or getMyInt(), that thread now owns the lock of that instance of MySynchronizedObject, until the putMyInt() or getMyInt() method exits. This acquisition and release of a lock is done automatically and atomically by the Java. This ensures that race conditions cannot occur in the underlying implementation of the threads, thus ensuring data integrity. Synchronization isn't the whole story. The two threads must also be able to notify one another when they've done their job.
Each object only has one lock that is shared by all synchronized areas of that program. A common misconception is that each synchronized area contains its own lock.

Using the notifyAll and wait Methods

The MySynchronizedObject stores its variable called contents. A boolean, named available, is also declared. The available variable has a value of true when the value has just been put but not yet gotten and is false when the value has been gotten but not yet put. We will first consider a simple implementation of synchronized get and put methods.

public synchronized int get() 
{    
  if (available == true) 
  {
    available = false;
    return contents;
  }
}
public synchronized void put(int value) 
{    
  if (available == false) 
  {
    available = true;
    contents = value;
  }
}

These two methods will not work. Consider first the get method. What happens if nothing has yet been put in the MySynchronizedObject and available isn't true? In this case get does nothing. Likewise, if the something is put into the object before the get method was called the put does nothing.

What is needed is for the caller of the get method to wait until the there is something to read. Likewise, the caller of the put method should wait until there is no data and it is safe to store its value. The two threads must coordinate more fully and can use Object's wait and notifyAll() methods to do so.

Consider this new implementation of both get and put that wait on and notify each other of their activities:

public synchronized int get() 
{
  while (available == false) 
  {
    try 
    {
      // wait for a thread to put a value
      wait();
    } 
    catch (InterruptedException e) 
    {
    }
  }
  available = false;
  // notify that value has been retrieved
  notifyAll();
  return contents;
}

public synchronized void put(int value) 
{
  while (available == true) 
  {
    try 
    {
      // wait for to get value
      wait();
    } 
    catch (InterruptedException e) 
    {
    }
  }
  contents = value;
  available = true;
  // notify that value has been set
  notifyAll();
} 

The get method loops until the put method has been called and there is data to read. The wait method is called each time through this loop. Calling the wait method relinquishes the lock held on the MySynchronizedObject (thereby allowing other threads to get the lock and update the MySynchronizedObject) as it waits for a notify method to be called. Once something is put in the MySynchronizedObject, it notifies any waiting threads by calling notifyAll(). These waiting threads will then come out of the wait state. Available will be set to true, causing the loop to exit. This causes the get method to return the value in the MySynchronizedObject. The put method works in a similar fashion, waiting for the a thread to consume the current value before allowing the other threads to add more values.

The notifyAll() method wakes up all threads waiting on the object in question (in this case, the MySynchronizedObject). The awakened threads compete for the lock. One thread gets it, and the others go back to waiting. The Object class also defines the notify method, which arbitrarily wakes up one of the threads waiting on this object.

You should now be familiar with the basics of thread handling in Java. You are now ready to learn how a feed forward back propagation neural network can be implemented. This will be the topic of the next two sections of this chapter.