Introduction to Neural Networks for C#, Session 7

Session Material

Simulated annealing is another method of training neural networks. Simulated annealing can often perform faster than backpropagation or genetic algorithms. Further, simulated annealing can sometimes be used in conjunction with backpropagation. When backpropagation is no longer successfully training a neural network you can switch to simulated annealing to train further, or perhaps get the backpropagation algorithm past a "flat spot", or local minimum. A local minimum is an area where the backpropagatin can get stuck on a local minimum in the error rate. It is better to find the "global minimum" for the error rate.

Simulated annealing is based on a real-life metallurgical process called annealing. Annealing, in metallurgy and materials science, is a heat treatment wherein a material is altered, causing changes in its properties such as strength and hardness. It is a process that produces conditions by heating to above the re-crystallization temperature and maintaining a suitable temperature, and then cooling. Annealing is used to induce ductility, soften material, relieve internal stresses, refine the structure by making it homogeneous, and improve cold working properties.

The following flow chart describes how simulated annealing works.

Basically, computer simulated annealing starts the neural network at a high temperature. At a a high temperature the molecules of a metal would be in an excited state. Similarly, the weight matrixes of the neural network will be randomized from their original values to greater degrees when the temperature is higher. The neural network is not allowed to increase its error rate. If randomization causes an increase in the error rate, then that cycle is discarded. As the temperature cools, the weights fall into more optimal positions, and the error should decrease.

Mid-Term Assignment

For this assignment, you will create a neural network that attempts to identify the type of forest a certain area of land will be covered with. To do this you will use data from the USA Forestry Service. You can find this data here. This data is stored in a CSV file.

[Forest Cover Data]

At the above link you can find information about the layout of the file. Choose some(or all) of the data to use for inputs. You will have to decide how to represent the inputs as input neurons. One option is to establish min/max data for each input, for example, elevation. Then calculate what percent of the maximum each input is. You will also need to decide some way to express the different forest cover types as output neurons. Will you have one output neuron, or one for each cover types? Train your neural network and then present some of the data that you did not train with and see how accurate it was. I was able to train to about 10% error rate with a few hours training.

Videos for this Session

Video	Title
	Introduction to Neural Networks for C#(Class 7/16, Part 1/3)
	Introduction to Neural Networks for C#(Class 7/16, Part 2/3)
	Introduction to Neural Networks for C#(Class 7/16, Part 3/3)