Introduction

    In this chapter we shall examine one of the most common neural network architectures, the feedforword backpropagation neural network. This neural network architecture is very popular, because it can be applied to many different tasks. To understand this neural network architecture, we must examine how it is trained and how it processes a pattern.

    The first term, “feedforward” describes how this neural network processes and recalls patterns. In a feedforward neural network, neurons are only connected foreword. Each layer of the neural network contains connections to the next layer (for example, from the input to the hidden layer), but there are no connections back. This differs from the Hopfield neural network that was examined in chapter 3. The Hopfield neural network was fully connected, and its connections are both forward and backward. Exactly how a feedforward neural network recalls a pattern will be explored later in this chapter.

    The term “backpropagation” describes how this type of neural network is trained. Backpropagation is a form of supervised training. When using a supervised training method, the network must be provided with both sample inputs and anticipated outputs. The anticipated outputs are compared against the actual outputs for given input. Using the anticipated outputs, the backpropagation training algorithm then takes a calculated error and adjusts the weights of the various layers backwards from the output layer to the input layer. The exact process by which backpropagation occurs will also be discussed later in this chapter.

    The backpropagation and feedforward algorithms are often used together; however, this is by no means a requirement. It would be quite permissible to create a neural network that uses the feedforward algorithm to determine its output and does not use the backpropagation training algorithm. Similarly, if you choose to create a neural network that uses backpropagation training methods, you are not necessarily limited to a feedforward algorithm to determine the output of the neural network. Though such cases are less common than the feedforward backpropagation neural network, examples can be found. In this book, we will examine only the case in which the feedforward and backpropagation algorithms are used together. We will begin this discussion by examining how a feedforward neural network functions.