FATAL: no PostgreSQL user name specified in startup packet

Submitted by nnuser9897 on Sat, 09/26/2009 - 14:15

Using Encog in Java

Hello,

what this exception means and how to repair it?

Exception in thread "main" org.encog.neural.NeuralNetworkError: org.postgresql.util.PSQLException: FATAL: no PostgreSQL user name specified in startup packet

Part of my source code:

NeuralDataSet nds = new SQLNeuralDataSet(sql.toString(), 6, 2, "org.postgresql.Driver", "jdbc:postgresql://localhost:5432/data", "postgres", "postgres");

this.network = new BasicNetwork();
this.network.setName("tseries-6-5-2-100000");
this.network.addLayer(new BasicLayer(6));
this.network.addLayer(new BasicLayer(5));
this.network.addLayer(new BasicLayer(2));
this.network.getStructure().finalizeStructure();
this.network.reset();

final Train train = new ResilientPropagation(this.network, nds);

Display.showMessageWithDate("Start training to max. " + maxError + " error rate ...");

int epoch = 1;

do {
train.iteration();
if (epoch % 50000 == 0) {
System.out.println("Epoch #" + epoch + " Error:" + train.getError());
}
epoch++;
} while (train.getError() > maxError);

Display.showMessageWithDate("End training...");

saveNetwork();

Thank you for reply.

Submitted by nnuser9897 on Sun, 09/27/2009 - 14:14.

Does someone know the answer to this problem?

Submitted by jeffheaton on Sun, 09/27/2009 - 16:46.

I will start with.. I have no idea off hand. It sounds like a password is not being specified to Postgre. I have no experience with Postgre. But I am not sure its a setup thing with Postgre, it might be. I guess for sure, make sure your connection information is correct.

However, the SQL dataset is somewhat untested. I have really never used it for a project. It has a simple unit test that tests that it can read data from SQL. If there is a bug in it, I would like to resolve it. Especially with Encog 2.2 just weeks away.

Encog needs a SQL dataset example, there currently is not one. Its an important feature, especially with very large training sets. I am going to create an Encog example for SQL training, in the next day or so, and test it. This will give me more information as to what might be causing this.

Hope to have more information soon. This is a much needed Encog example anyhow.

Submitted by jeffheaton on Sun, 09/27/2009 - 18:09.

Yes, this appears to be an Encog bug. I created an example myself, and I get this:

Exception in thread "main" org.encog.neural.NeuralNetworkError: java.sql.SQLException: Access denied for user ''@'localhost' (using password: NO)
at org.encog.neural.data.sql.SQLNeuralDataSet$SQLNeuralIterator.(Unknown Source)
at org.encog.neural.data.sql.SQLNeuralDataSet.iterator(Unknown Source)
at org.encog.neural.networks.training.propagation.Propagation.iteration(Unknown Source)
at org.encog.examples.neural.xorsql.XORSQL.main(XORSQL.java:97)
Caused by: java.sql.SQLException: Access denied for user ''@'localhost' (using password: NO)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3558)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3490)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:919)
at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3996)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1284)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2142)
at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:781)
at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:352)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:284)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
... 4 more

Which is basically the MySQL version of the error you were getting. Encog is somehow dropping the userid/password by the time it gets around to creating the iterator for training.

I will fix this bug. Thanks for finding it, I should have had an example for this feature, and it would have been caught much sooner.

This should not be too difficult, I will post an update when it is resolved, probably sometime tomorrow.

Submitted by jeffheaton on Sun, 09/27/2009 - 19:05.

I updated the mainline source code. You can download the latest Encog from:

http://build.heatonresearch.com/cruisecontrol/buildresults/java-encog-core

It is under "build artifacts".

Submitted by nnuser9897 on Mon, 09/28/2009 - 02:46.

Hello, thanks for the bug fix, it seems to be ok!

I have another issue, or question rather. My table, where data for training reside, contains something over 16.000.000 records. When I start training at all this dataset, Encog throws an exception:

28.9.2009 10:33:53 org.encog.neural.networks.training.propagation.Propagation iteration
INFO: Beginning propagation iteration
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

These si probably caused there is too much data for training. Do you have some advice how to solve this situation? Split it into more datasets? How is recommended maximum dataset size?

Submitted by nnuser9897 on Mon, 09/28/2009 - 03:28.

I have idea how to solve the heap problem.

I plan to split all rows into separated parts e.g. about 100-thousands rows. This should remove problem with heap size. For all these reduced data sets I will execute training phase of my neural net.

One question: is it difference to train neural net on one dataset with 1 million rows or on one thousand datasets with one thousand rows per dataset? Is it necessary to change somehow training technique?

P.S. I use ResilientPropagation

Submitted by SeemaSingh on Mon, 09/28/2009 - 06:50.

I think we we allowed the "fetch size" to be set, that might take care of this. Then you can set the fetch size to say 1000 or so, and the JDBC driver SHOULD divide the query up and not overwhelm the memory. It is a simple change, I will implement it today.

Another idea, if for some reason fetch size does NOT do it. Depends on how the JDBC driver they are using was implemented. Then the UnionDataSet, that we previously discussed, would do the job. This is not a class currently in Encog, but it is planned. It takes several training sets and makes them one. So you could use it make a union of say 10 partial queries.

The UnionDataSet would not be had to implement, I could bump it up in the schedule, if the fetch size does not work.

Submitted by SeemaSingh on Mon, 09/28/2009 - 08:37.

I think this will even increase performance to some degree since the connection can now stay open. You will use the SQL dataset the same way, just now you can add this line.

sqlDataSet.getStatement().setFetchSize(1000);

I THINK this might fix the issue, but I am not sure. At an rate you now have getStatement and getConnection, so you can set many of the properties directly now and have much greater control over how the JDBC driver is being used. You can download the JAR from the same URL Jeff posted earlier.

Submitted by jeffheaton on Mon, 09/28/2009 - 08:50.

Encog 2.2 goes out in approx 3 weeks. If you think you can get the UnionDataSet in and stable before then... I say... go for it!

Submitted by nnuser9897 on Mon, 09/28/2009 - 11:28.

I'll try it, thank you.