Introduction to Neural Networks for C#, Session 5
| Course Name | Introduction to Neural Networks for C# |
| Instructor | jeffheaton |
| Session Title | Genetic Algorithms, Traveling Salesman, TicTacToe |
| Session Number | 5 |
Session Material
Backpropagation is not the only way to train a feedforward neural network. Neural networks. There are many training algorithms for this purpose. This course will focus on backpropagation, genetic training and simulated annealing. This class session will focus on genetic training. Genetic training uses a genetic algorithm.
You might wonder why you would need more than one training algorithm. Sometimes a neural network will not train effectively with a particular training algorithm. Some problems will require multiple training algorithms. Such hybrid solutions can often find a solution faster than a single solution.
How a Genetic Algorithm Works
Genetic algorithms attempt to solve computer problems by simulating the way that organisms evolve into new organifimsms that are better suited to their environment. A genetic algorithm evolves potential solutions.
To use a genetic algorithm the solution must be broken down into an array of numbers. Further, this array of numbers must always be the same length. This string of numbers corresponds to a DNA double helix. As the organisms mate with each other, this DNA strand will be split and recombined. This allows offspring to inherit desirable traits from their parents.
Different texts will define the parts of a genetic algorithm differently. This course uses the following terms.
- Gene - One number in the DNA sequence
- Chromosome - One DNA sequence
- Life-form - One or more chromosomes
This is a very simplified version of what is really going on in biology. In this course the life-forms will always have a single chromosome.
Mutation is also used. Sometimes the elements to an optimal solution may not be present in the population pool. Mutation allows the genes to be randomized a bit to introduce new traits. Mutation introduces new traits in biological systems as well.
The Traveling Salesman Problem
The traveling salesman problem has long been a mainstay for computer science. A salesman must visit an itinerary of cities. But, what is the optimal route through these cities? Traditional brute force programming can solve this problem for a small number of cities. However, larger number of cities require less traditional programming, for example, a genetic algorithm.
To adapt the traveling salesman problem (TSP) to a genetic algorithm the optimal solutions must be converted to chromosomes. This is relatively easy for the TSP. Each potential solution is an itinerary. The itinerary can be converted to a chromosome by simply creating an array that contains the cities to be visited, in the order to visit them.
Training a Neural Network
Adapting a neural network to a genetic algorithm is relativly easy as well. The weight and threshold matrix is simply converted to a long string of floating point values.
Programming Assignment #1
The following programming assignment is due by the next class session. We will review this programming assignment in the next class session.
Learning Objectives:
- Learn to read data from CSV files
- Train a neural network for multiple patterns
- Create a feedforward neural network
The XOR example that we saw earlier simply hard coded all of the XOR training data into the program. This is okay for small data sets that will never change, however, the other programming assignments in this course will require larger training sets than you would like to hard code into a program. Program 1 shows you how to read CSV files.
Shown here is a CSV file for the XOR operator:
Listing: A CSV File for the XOR Operator
0,0,0 0,1,1 1,0,1 1,1,0
The following program demonstrates how to read a CSV file using the ReadCSV class. This is an enhanced version of the ReadCSV class that was provided by the book.
Listing: Reading a CSV File
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using HeatonResearchNeural.Util;
namespace TestCSV
{
class Program
{
static void Main(string[] args)
{
ReadCSV csv = new ReadCSV("c:\\data\\xor.csv", false);
while (csv.Next())
{
double in1 = Double.Parse(csv.Get(0));
double in2 = Double.Parse(csv.Get(1));
double out1 = Double.Parse(csv.Get(2));
Console.WriteLine("Input 1:" + in1);
Console.WriteLine("Input 2:" + in2);
Console.WriteLine("Output 1:" + out1);
Console.WriteLine("-------------");
}
}
}
}
This program produces the following output.
Listing: Program Output
Input 1:0.0 Input 2:0.0 Output 1:0.0 ------------- Input 1:0.0 Input 2:1.0 Output 1:1.0 ------------- Input 1:1.0 Input 2:0.0 Output 1:1.0 ------------- Input 1:1.0 Input 2:1.0 Output 1:0.0 -------------
The ReadCSV class provided by the book always assumes that the first line of the CSV file contains the column header names. This is not always the case. As a result, the enhanced version of the ReadCSV class provides a boolean field on the constructor that specifies if the header line is present. You can see the enhanced ReadCSV here.
Listing: Enhanced ReadCSV.cs
// Introduction to Neural Networks for C#, 2nd Edition
// Copyright 2008 by Heaton Research, Inc.
// http://www.heatonresearch.com/online/introduction-neural-networks-cs-edition-2
//
// ISBN13: 978-1-60439-009-4
// ISBN: 1-60439-009-3
//
// This class is released under the:
// GNU Lesser General Public License (LGPL)
// http://www.gnu.org/copyleft/lesser.html
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
namespace HeatonResearchNeural.Util
{
public class ReadCSV
{
/// <summary>
/// The format that dates are expected to be in.
/// </summary>
public const String dateFormat = "yyyy-MM-dd";
/// <summary>
/// The file to read.
/// </summary>
private TextReader reader;
/// <summary>
/// The names of all of the columns, read from the first line of the file.
/// </summary>
private IDictionary<String, int> columns = new Dictionary<String, int>();
/// <summary>
/// The data for the current line.
/// </summary>
private String[] data;
/// <summary>
/// Format a date/time object to the same format that we parse in.
/// </summary>
/// <param name="date">The date to format.</param>
/// <returns>A formatted date and time.</returns>
public static String DisplayDate(DateTime date)
{
return date.ToString(dateFormat);
}
/// <summary>
/// Parse a date using the specified format.
/// </summary>
/// <param name="when">A string that contains a date in the specified format.</param>
/// <returns>A DateTime that was parsed.</returns>
public static DateTime ParseDate(String when)
{
try
{
return DateTime.Parse(when);
}
catch (FormatException)
{
return default(DateTime);
}
}
/// <summary>
/// Construct an object to read the specified CSV file.
/// </summary>
/// <param name="filename">The filename to read.</param>
public ReadCSV(String filename,bool headers)
{
this.reader = new StreamReader(filename);
if (headers)
{
// read the column heads
String line = this.reader.ReadLine();
string[] tok = line.Split(',');
for (int index = 0; index < tok.Length; index++)
{
String header = tok[index];
this.columns.Add(header.ToLower(), index);
}
this.data = new String[tok.Length];
}
}
/// <summary>
/// Close the file.
/// </summary>
public void Close()
{
this.reader.Close();
}
/// <summary>
/// Get the specified column using an index.
/// </summary>
/// <param name="i">The zero based index of the column to read.</param>
/// <returns>The specified column as a string.</returns>
public String Get(int i)
{
return this.data[i];
}
/// <summary>
/// Get the specified column as a string.
/// </summary>
/// <param name="column">The specified column.</param>
/// <returns>The specified column as a string.</returns>
public String Get(String column)
{
if (!columns.ContainsKey(column.ToLower()))
{
return null;
}
int i = this.columns[column.ToLower()];
return this.data[i];
}
/// <summary>
/// Read the specified column as a date.
/// </summary>
/// <param name="column">The specified column.</param>
/// <returns>The specified column as a DateTime.</returns>
public DateTime GetDate(String column)
{
String str = Get(column);
return DateTime.Parse(str);
}
/// <summary>
/// Get the specified column as a double.
/// </summary>
/// <param name="column">The column to read.</param>
/// <returns>The specified column as a double.</returns>
public double GetDouble(String column)
{
String str = Get(column);
return double.Parse(str);
}
/// <summary>
/// Get an integer that has the specified name.
/// </summary>
/// <param name="col">The column name to read.</param>
/// <returns>The specified column as an int.</returns>
public int GetInt(String col)
{
String str = Get(col);
try
{
return int.Parse(str);
}
catch (FormatException)
{
return 0;
}
}
/// <summary>
/// Read the next line.
/// </summary>
/// <returns>Return false if there are no more lines in the file.</returns>
public bool Next()
{
String line = this.reader.ReadLine();
if (line == null)
{
return false;
}
string[] tok = line.Split(',');
if (this.data == null)
{
this.data = new String[tok.Length];
}
for (int i = 0; i < tok.Length; i++)
{
String str = tok[i];
if (i < this.data.Length)
{
this.data[i] = str;
}
}
return true;
}
}
}
For this programming assignment you must create a CSV file. Then read it to the input and ideal arrays for training. You should construct a CSV file that will train the network for the XOR, AND and OR operators. Because these are three different operators you will need some way of communicating to the neural network WHICH operator it is attempting to process. To do this, it is suggested that the neural network be constructed as follows:
- Input 1: The left side of the operation
- Input 2: The right side of the operation
- Input 3: Is this an XOR operation? (1=yes, 0=otherwise)
- Input 4: Is this an AND operation? (1=yes, 0=otherwise)
- Input 5: Is this an OR operation? (1=yes, 0=otherwise)
- Output 1: The output from this operation
Find a suitable number of hidden layer neurons.
Example. The input for "true and false" would be:
1,0,0,1,0
To submit this program send a ZIP with both the CSV and source file(s) to the "support" email address at the website heatonresearch.com.
I have provided a starting point Visual Studio project file for this assignment. You can download all of the book files and a simple XOR example. This includes the extended ReadCSV class. Simply create a new project in this workspace to create your assignment. You need only submit your source file(s) and CSV.
[Download Class Starting Project]
RSS
YouTube
Twitter
Facebook
