Using a Simple Neural Network

    Following is an example of a very simple neural network. Though the network is simple, it includes nearly all of the elements of the more complex neural networks that will be covered later in this book.

    First, consider an artificial neuron, as shown in Figure 1.6.

Figure 1.6: Artificial neuron.

    There are two attributes associated with this neuron: the threshold and the weight. The weight is 1.5 and the threshold is 2.5. An incoming signal will be amplified, or de-amplified, by the weight as it crosses the incoming synapse. If the weighted input exceeds the threshold, then the neuron will fire.

    Consider a value of one (true) presented as the input to the neuron. The value of one will be multiplied by the weight value of 1.5. This results in a value of 1.5. The value of 1.5 is below the threshold of 2.5, so the neuron will not fire. This neuron will never fire with Boolean input values. Not all neurons accept only boolean values. However, the neurons in this section only accept the boolean values of one (true) and zero (false).

A Neural Network for the And Operator

    The neuron shown in Figure 1.6 is not terribly useful. However, most neurons are not terribly useful—at least not independently. Neurons are used with other neurons to form networks. We will now look at a neural network that acts as an AND gate. Table 1.1 shows the truth table for the AND logical operation.

Table 1.1: The AND Logical Operation

    A simple neural network can be created that recognizes the AND logical operation. There will be three neurons in total. This network will contain two inputs and one output. A neural network that recognizes the AND logical operation is shown in Figure 1.7.

Figure 1.7: A neural network that recognizes the AND logical operation.

    There are two inputs to the network shown in Figure 1.7. Each neuron has a weight of one. The threshold is 1.5. Therefore, a neuron will only fire if both inputs are true. If either input is false, the sum of the two inputs will not exceed the threshold of 1.5.

    Consider inputs of true and false. The true input will send a value of one to the output neuron. This is below the threshold of 1.5. Likewise, consider inputs of true and true. Each input neuron will send a value of one. These two inputs are summed by the output neuron, resulting in two. The value of two is greater than 1.5, therefore, the neuron will fire.

A Neural Network for the Or Operation

    Neural networks can be created to recognize other logical operations as well. Consider the OR logical operation. The truth table for the OR logical operation is shown in Table 1.2. The OR logical operation is true if either input is true.

Table 1.2: The OR Logical Operation

    The neural network that will recognize the OR operation is shown in Figure 1.8.

Figure 1.8: A neural network that recognizes the OR logical operation.

    The OR neural network looks very similar to the AND neural network. The biggest difference is the threshold value. Because the threshold is lower, only one of the inputs needs to have a value of true for the output neuron to fire.

A Neural Network for the XOR Operation

    Next we will consider a neural network for the exclusive or (XOR) logical operation. The XOR truth table is shown in Table 1.3.

Table 1.3: The XOR Logical Operation

    The XOR logical operation requires a slightly more complex neural network than the AND and OR operators. The neural networks presented so far have had only two layers— an input layer and an output layer. More complex neural networks also include one or more hidden layers. The XOR operator requires a hidden layer. As a result, the XOR neural network often becomes a sort of “Hello World” application for neural networks. You will see the XOR operator again in this book as different types of neural network are introduced and trained.

    Figure 1.9 shows a three-layer neural network that can be used to recognize the XOR operator.

Figure 1.9: A neural network that recognizes the XOR logical operation.

    Consider the case in which the values of true and true are presented to this neural network. Both neurons in the hidden layer receive the value of two. This is above the thresholds of both of the hidden layer neurons, so they will both fire. However, the first hidden neuron has a weight of -1, so its contribution to the output neuron is -1. The second neuron has a weight of 1, so its contribution to the output neuron is 1. The sum of 1 and -1 is zero. Zero is below the threshold of the output neuron, so the output neuron does not fire. This is consistent with the XOR operation, because it will produce false if both inputs are true.

    Now consider if the values of false and true are presented to the neural network. The input to the first hidden layer neuron will be 1, from the second input neuron. This is lower than the threshold of 1.5, so it will not fire. The input to the second hidden layer neuron will also be 1, from the second input neuron. This is over the 0.5 threshold, so it will fire. The input to the output neuron will be zero from the left hidden neuron and 1 from the right hidden neuron. This is greater than 0.5, so the output neuron will fire. This is consistent with the XOR operation, because it will produce true if one of the input neurons is true and the other false.

    Of course, the neural networks shown in the preceding sections are very simple. However, they illustrate all of the key points for more complex neural networks. Future chapters will introduce additional types of neural networks; however, neural networks will almost always feature weights and thresholds.