Introduction

    Computers can perform many operations considerably faster than a human being. However, faster is not always better for problem solving. There are many tasks for which the computer falls considerably short of its human counterpart. There are numerous examples of this. For instance, given two pictures, a preschool child can easily tell the difference between a cat and a dog. Yet, this same simple task is extremely difficult for today’s computers.

    The goal of this book is to teach the reader how to construct neural networks using the Java programming language. As with any technology, it is just as important to know when not to use neural networks as it is to understand when they should be used. This chapter provides an introduction to the appropriate uses of neural networks, and explains which programming requirements are conducive to their use.

    This chapter begins with a brief introduction to the structure of neural networks. This discussion provides an overview of neural network architecture, and explains how a typical neural network is constructed. Following this introduction is a discussion of how a neural network is trained. Finally, an overview is provided of the ultimate task, validating a trained neural network.

How is a Biological Neural Network Constructed?

    The term neural network, as it is normally used, is actually a misnomer. Computers attempt to simulate biological neural networks by implementing artificial neural networks. However, most publications use the term “neural network,” rather than “artificial neural network” (ANN). This book follows suit. Unless the term “neural network” is explicitly prefixed with the terms “biological” or “artificial” you can assume that the term “artificial neural network” is intended. To explore this distinction, you will first be shown the structure of a biological neural network.

    To construct a computer capable of “human-like thought,” researchers have used the only working model they have available—the human brain. However, the human brain as a whole is far too complex to model. Rather, the individual cells that make up the human brain are studied. At the most basic level, the human brain is composed primarily of neuron cells. They are the basic building blocks of the human brain. Artificial neural networks attempt to simulate the behavior of these cells.

    A neuron cell, as seen in Figure 1.1, accepts signals from dendrites. When a neuron accepts a signal, that neuron may fire. When a neuron fires, a signal is transmitted over the neuron’s axon. Ultimately, the signal will leave the neuron as it travels to the axon terminals. The signal is then transmitted to other neurons or nerves.

Figure 1.1: A neuron cell.

    This signal, transmitted by the neuron, is an analog signal. Most modern computers are digital machines, and thus, require a digital signal. A digital computer processes information as either off or on, using the binary digits zero and one, respectively. The presence of an electric signal is indicated with a value of one, whereas the absence of an electrical signal is indicated with a value of zero. Figure 1.2 shows a digital signal.

Figure 1.2: A digital signal.

    Some of the early computers were analog, rather than digital. An analog computer uses a much wider range of values than zero and one. This wider range is achieved by increasing or decreasing the voltage of the signal. Figure 1.3 shows an analog signal. Though analog computers are useful for certain simulation activities, they are not suited to processing the large volumes of data that digital computers are typically required to process. Thus, nearly every computer in use today is digital.

Figure 1.3: Sound recorder showing an analog file.

    Biological neural networks are analog. As you will see in the next section, simulating analog neural networks on a digital computer can present some challenges. Neurons accept an analog signal through their dendrites, as seen in Figure 1.1. Because this signal is analog, the voltage of each signal will vary. If the voltage is within a certain range, the neuron will fire. When a neuron fires, a new analog signal is transmitted from the firing neuron to other neurons. This signal is conducted over the firing neuron’s axon. The regions of input and output are called synapses. Later, in chapter 5, The Feedforward Backpropagation Neural Network example will demonstrate that the synapses are the interface between a program and a neural network.

    A neuron makes a decision by firing or not firing. The decisions being made are extremely low-level decisions. It requires a large number of decisions to be made by many neurons just to read this sentence. Higher-level decisions are the result of the collective input and output of many neurons.

    Decisions can be represented graphically by charting the input and output of neurons. Figure 1.4 illustrates the input and output of a particular neuron. As you will be shown in chapter 5, there are different types of neurons, all of which have differently shaped output graphs. Looking at the graph shown in Figure 1.4, it can be seen that the neuron in this example will fire at any input greater than 0.5 volts.

Figure 1.4: Activation levels of a neuron.

    A biological neuron is capable of making basic decisions. Artificial neural networks are based on this model. Following is an explanation of how this model is simulated using a digital computer.