Get the entire book!

Neural network program is deeply rooted in mathematics. One of the most common forms of mathematics used in Neural Network study is the Matrix. In this appendix I will present a quick overview of how some of the matrix operations in this book are performed.

Why Matrixes?

Why are matrixes so important to Nerual Networks? It is because a matrix is generally used to store the Neuron weights. Consider a 2-layer nural network with two neurons in each layer. A weight exists between each neuron, in each layer. The neurons within a single layer are not connected. This results in two connections. The two neurons in the first layer each have a connection to the two in the other layer.

But wait! There are more than just the two neurons. Each connection is two way. This results in four total connections. This can be written as the following weight matrix.

Matrix Operations

There are many operations that can be performed on Matrixes. In AI the most common are:

• Matrix Multiplication
• Matrix Inverse
• Matrix Dot Product

These topics are covered, in detail, at the following site. This site is operated by the makers of MathCAD. I refer to it often.

http://mathworld.wolfram.com/topics/MatrixOperations.html

Sigma Notation

I have seen many programmers, without a mathematical background, intimidated by sigma notation. Its actually VERY simple, in programming, it relates very much to a for loop. Consider the following.

Sigma is used to sum something, in a loop. Think of the above sigma-equation as a for loop that counts from 1 to 10, using y as the index. Each time through the (y+1) part is summed and the result is placed into x. The following Java program does the same thing.

Int x = 0;
for( int y = 0; y<=10; y++ )
{
	x+= (y+1);
}