Introduction

Get the entire book!

In this chapter we shall examine one of the most common neural network architectures-- the feed foreword back propagation neural network. This neural network architecture is very popular because it can be applied to many different tasks. To understand this neural network architecture we must examine how it is trained and how it processes the pattern. The name "feed forward back propagation neural network" gives some clue as to both how this network is trained and how it processes the pattern.

The first term, "feed foreword" describes how this neural network processes the pattern and recalls patterns. When using a "feed forward neural network" neurons are only connected foreword. Each layer of the neural network contains connections to the next layer (for example from the input to the hidden layer), but there are no connections back. This differs from the Hopfield neural network that was examined in Chapter 2. The Hopfield neural network was fully connected, that is each neuron was connected to every other neuron in the network—even to itself. Exactly how a "feed forward neural network" recalls the pattern will be explored in this chapter.

The term back propagation describes how this type of neural network is trained. Back propagation is a form of supervised training. When using a supervised training method the network must be provided with sample inputs and anticipated outputs. These anticipated outputs will be compared against the anticipated output from the neural network. Using these anticipated outputs the "back propagation" training algorithm then takes a calculated error and adjusts the weights of the various layers backwards from the output layer all the way back to the input layer. The exact process by which back propagation occurs will be discussed later in this chapter.

The algorithms "back propagation" and "feed foreword" are often used together. This by no means needs to be the case. It would be quite permissible to create a neural network that uses the feed forward algorithms to determine its output, and yet does not use the "back propagation training algorithm". Similarly if you choose to create a neural network that uses "back propagation training methods" you are not necessarily limited to a "feed foreword" algorithm to determine the output of the neural network.. Though such cases are less common than the "feed forward back propagation neural network", examples can be found. In this book will examine only the case of using the "feed foreword" and "back propagation" algorithms together. We will begin this discussion by examining how a feed forward neural network functions.