Introduction to Neural Networks for Java, Session 1
| Course Name | Introduction to Neural Networks for Java |
| Instructor | jeffheaton |
| Session Title | What is a Neural Network? Weights & Thresholds |
| Session Number | 1 |
Session Material
It is just as important to know when to use a neural network as it is to know when not to use a neural network. This class session emphasizes some of the cases where it is important to use or not use a neural network.Don't look for "excuses" to use a neural network. If you are already solving the problem just fine with non-neural network code, then a neural network is probably not necessary. Some of the cases where a neural network is not necessary are listed here:
- Problems where the exact solution must be documented/understood
- Problems that has a solution that can easily be represented as a flowchart
- Problems that have a clear, well understood, solution
Though neural networks are not the ideal choice in many situations, there are situations where a neural network is the ideal choice. They are listed here:
- Problems where the solution must be adapted over time
- Problems where brute force would take too long
- Problems where traditional programming techniques fail
What Are the Uses of Neural Networks
Neural networks can be used in many ways. Some of the more common are listed here.- Classification
- Prediction
- Pattern recognition
- Optimization
Classification - The neural network breaks the incoming data into groups.
Prediction - The neural predicts future events by searching for patterns in the past.
Pattern Recognition - The neural network recognizes patterns in the incoming data.
Classification - The neural network is used to optimize systems such as electronic circuits.
A Simple Neural Network
We will examine a very simple neural network that can can lean logical operators.
First, consider an artificial neuron, as shown here.
There are two attributes associated with this neuron: the threshold and the weight. The weight is 1.5 and the threshold is 2.5. An incoming signal will be amplified, or de-amplified, by the weight as it crosses the incoming synapse. If the weighted input exceeds the threshold, then the neuron will fire.
Consider a value of one (true) presented as the input to the neuron. The value of one will be multiplied by the weight value of 1.5. This results in a value of 1.5. The value of 1.5 is below the threshold of 2.5, so the neuron will not fire. This neuron will never fire with Boolean input values. Not all neurons accept only boolean values. However, the neurons in this section only accept the boolean values of one (true) and zero (false).
A Neural Network for the And Operator
The neuron shown in Figure 1.6 is not terribly useful. However, most neurons are not terribly useful?at least not independently. Neurons are used with other neurons to form networks. We will now look at a neural network that acts as an AND gate. The following table shows this.
| A | B | A AND B |
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 1 | 0 | 0 |
| 1 | 1 | 1 |
A simple neural network can be created that recognizes the AND logical operation. There will be three neurons in total. This network will contain two inputs and one output. A neural network that recognizes the AND logical operation is shown in following figure.
There are two inputs to the network shown in Figure 1.7. Each neuron has a weight of one. The threshold is 1.5. Therefore, a neuron will only fire if both inputs are true. If either input is false, the sum of the two inputs will not exceed the threshold of 1.5.
Consider inputs of true and false. The true input will send a value of one to the output neuron. This is below the threshold of 1.5. Likewise, consider inputs of true and true. Each input neuron will send a value of one. These two inputs are summed by the output neuron, resulting in two. The value of two is greater than 1.5, therefore, the neuron will fire.
A Neural Network for the Or Operation
Neural networks can be created to recognize other logical operations as well. Consider the OR logical operation. The truth table for the OR logical operation is shown in following table. The OR logical operation is true if either input is true.
| A | B | A OR B |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
The neural network that will recognize the OR operation is shown in the following figure.
The OR neural network looks very similar to the AND neural network. The biggest difference is the threshold value. Because the threshold is lower, only one of the inputs needs to have a value of true for the output neuron to fire.
A Neural Network for the XOR Operation
Next we will consider a neural network for the exclusive or (XOR) logical operation. The XOR truth table is shown in following table.
| A | B | A XOR B |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
The XOR logical operation requires a slightly more complex neural network than the AND and OR operators. The neural networks presented so far have had only two layers? an input layer and an output layer. More complex neural networks also include one or more hidden layers. The XOR operator requires a hidden layer. As a result, the XOR neural network often becomes a sort of “Hello World” application for neural networks. You will see the XOR operator again in this book as different types of neural network are introduced and trained.
The following figure shows a three-layer neural network that can be used to recognize the XOR operator.
Figure 1.9: A neural network that recognizes the XOR logical operation.
Consider the case in which the values of true and true are presented to this neural network. Both neurons in the hidden layer receive the value of two. This is above the thresholds of both of the hidden layer neurons, so they will both fire. However, the first hidden neuron has a weight of -1, so its contribution to the output neuron is -1. The second neuron has a weight of 1, so its contribution to the output neuron is 1. The sum of 1 and -1 is zero. Zero is below the threshold of the output neuron, so the output neuron does not fire. This is consistent with the XOR operation, because it will produce false if both inputs are true.
Now consider if the values of false and true are presented to the neural network. The input to the first hidden layer neuron will be 1, from the second input neuron. This is lower than the threshold of 1.5, so it will not fire. The input to the second hidden layer neuron will also be 1, from the second input neuron. This is over the 0.5 threshold, so it will fire. The input to the output neuron will be zero from the left hidden neuron and 1 from the right hidden neuron. This is greater than 0.5, so the output neuron will fire. This is consistent with the XOR operation, because it will produce true if one of the input neurons is true and the other false.
These are very simple neural networks that were handcrafted. Handcrafting a neural network is not feasable for large neural networks. These neural networks are trained. We will see how to train a neural network in later class sessions.
RSS
YouTube
Twitter
Facebook