Setting Up Microsoft Access

    Microsoft Access is a commercial database provided by Microsoft. To use Microsoft Access you can either use the DDL script shown in Listing F.4, or use the spider.mdb database that was provided with the companion download.

Listing F.3: Microsoft Access DDL Script

CREATE TABLE [spider_host] (
[host_id] counter NOT NULL,
[host] varchar(255) NOT NULL,
[status] varchar(1) NOT NULL,
[urls_done] int NOT NULL,
[urls_error] int NOT NULL,
PRIMARY KEY ([host_id]),
CONSTRAINT `host` UNIQUE (`host`)
);

CREATE TABLE [spider_workload] (
[workload_id] counter NOT NULL,
[host] integer NOT NULL,
[url] varchar(255) NOT NULL,
[status] varchar(1) NOT NULL,
[depth] integer NOT NULL,
[url_hash] integer NOT NULL,
[source_id] integer NOT NULL,
PRIMARY KEY ([workload_id])
);

create index idx_status on spider_workload (status);
create index idx_url_hash on spider_workload (url_hash);

    Java contains a built in JDBC driver that bridges to ODBC. ODBC is a database connectivity standard commonly used on Microsoft Windows. To access Microsoft Access, we will use the built in ODBC bridge. This class is named:

    sun.jdbc.odbc.JdbcOdbcDriver

    To use the ODBC driver to access Microsoft Access, use a URL as follows:

jdbc:odbc:Driver={Microsoft Access Driver (*.mdb)};DBQ=c:\spider.mdb;DriverID=22;READONLY=false}

    You will need to modify the above URL to contain a path to your Microsoft Access database. Listing F.4 shows a sample spider configuration file for Microsoft Access

Listing F.4: Sample Spider Configuration for Microsoft Access

timeout:		60000
maxDepth:		-1
userAgent:
corePoolSize:	100 
maximumPoolSize:100
keepAliveTime:	60
dbURL: jdbc:odbc:Driver={Microsoft Access Driver (*.mdb)};DBQ=c:\spider.mdb;DriverID=22;READONLY=false}
dbClass:		sun.jdbc.odbc.JdbcOdbcDriver
workloadManager:com.heatonresearch.httprecipes.spider.workload.sql.SQLWorkloadManager
startup:		clear
filter:			com.heatonresearch.httprecipes.spider.filter.RobotsFilter

    Of course, you will have to modify the above listing to include the filename and path of your database.