org.encog.neural.data.buffer
Class BufferedNeuralDataSet

java.lang.Object
  org.encog.neural.data.buffer.BufferedNeuralDataSet

All Implemented Interfaces:

java.lang.Iterable<NeuralDataPair>, Indexable, NeuralDataSet

public class BufferedNeuralDataSet
extends java.lang.Object
implements NeuralDataSet, Indexable

This class is not memory based, so very long files can be used, without running out of memory. This dataset uses a binary file as a buffer. When used with a slower access dataset, such as CSV, XML or SQL, where parsing must occur, this dataset can be used to load from the slower dataset and train at much higher speeds. If you are going to create a binary file, by using the add methods, you must call beginLoad to cause Encog to open an output file. Once the data has been loaded, call endLoad. The floating point numbers stored to the binary file may not be cross platform.

Nested Class Summary

class

BufferedNeuralDataSet.BufferedNeuralDataSetIterator An iterator to move through the buffered data set.

Field Summary

static java.lang.String	ERROR_ADD Error message for ADD.
static java.lang.String	ERROR_REMOVE Error message for REMOVE.

Constructor Summary

BufferedNeuralDataSet(java.io.File bufferFile)
Construct a buffered dataset using the specified file.

Method Summary

void	add(NeuralData data1) Add only input data, for an unsupervised dataset.
void	add(NeuralData inputData, NeuralData idealData) Add both the input and ideal data.
void	add(NeuralDataPair inputData) Add a data pair of both input and ideal data.
void	beginLoad(int inputSize, int idealSize) Begin loading to the binary file.
void	close() Close all iterators.
void	endLoad() This method should be called once all the data has been loaded.
int	getIdealSize()
int	getInputSize()
void	getRecord(long index, NeuralDataPair pair) Get a record by index and copy it into the specified pair.
long	getRecordCount() Determine the total number of records in the set.
BufferedNeuralDataSet.BufferedNeuralDataSetIterator	iterator()
void	load(NeuralDataSet source) Load from the specified data source into the binary file.
Indexable	openAdditional() Open a second buffered data set, useful for multithreading.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ERROR_ADD

public static final java.lang.String ERROR_ADD

Error message for ADD.

See Also:

Constant Field Values

ERROR_REMOVE

public static final java.lang.String ERROR_REMOVE

Error message for REMOVE.

See Also:

Constant Field Values

Constructor Detail

BufferedNeuralDataSet

public BufferedNeuralDataSet(java.io.File bufferFile)

Construct a buffered dataset using the specified file.

Parameters:

bufferFile - The file to read/write binary data to/from.

Method Detail

add

public void add(NeuralData data1)

Add only input data, for an unsupervised dataset.

Specified by:

add in interface NeuralDataSet

Parameters:

data1 - The data to be added.

add

public void add(NeuralData inputData,
                NeuralData idealData)

Add both the input and ideal data.

Specified by:

add in interface NeuralDataSet

Parameters:

inputData - The input data.

idealData - The ideal data.

add

public void add(NeuralDataPair inputData)

Add a data pair of both input and ideal data.

Specified by:

add in interface NeuralDataSet

Parameters:

inputData - The pair to add.

beginLoad

public void beginLoad(int inputSize,
                      int idealSize)

Begin loading to the binary file. After calling this method the add methods may be called.

Parameters:

inputSize - The input size.

idealSize - The ideal size.

close

public void close()

Close all iterators.

Specified by:

close in interface NeuralDataSet

endLoad

public void endLoad()

This method should be called once all the data has been loaded. The underlying file will be closed.

getIdealSize

public int getIdealSize()

Specified by:

getIdealSize in interface NeuralDataSet

Returns:

Get the ideal data size.

getInputSize

public int getInputSize()

Specified by:

getInputSize in interface NeuralDataSet

Returns:

Get the input data size.

getRecord

public void getRecord(long index,
                      NeuralDataPair pair)

Get a record by index and copy it into the specified pair.

Specified by:

getRecord in interface Indexable

Parameters:

index - The index to load.

pair - THe pair to copy into.

getRecordCount

public long getRecordCount()

Description copied from interface: Indexable

Determine the total number of records in the set.

Specified by:

getRecordCount in interface Indexable

Returns:

The number of records in the set.

iterator

public BufferedNeuralDataSet.BufferedNeuralDataSetIterator iterator()

Specified by:

iterator in interface java.lang.Iterable<NeuralDataPair>

Returns:

Create an iterator.

load

public void load(NeuralDataSet source)

Load from the specified data source into the binary file. Do not call beginLoad before calling this method, as this is handled internally.

Parameters:

source - The source.

openAdditional

public Indexable openAdditional()

Open a second buffered data set, useful for multithreading.

Specified by:

openAdditional in interface Indexable

Returns:

The additional buffered data set.