Download XML data

To make life easier for you at data.openaustralia.org we’ve put up all the source data (in XML form) that’s required to get an instance of openaustralia.org up and running.

All you need to do is download the files (keeping the same directory structure) and put them in a directory called openaustralia/pwdata. So, you should end up with a directory tree that looks something like this:

pwdata/
|-- members
|   |-- divisions.xml
|   |-- links-abc-qanda.xml
|   |-- ministers.xml
|   |-- people.xml
|   |-- representatives.xml
|   |-- senators.xml
|   |-- twitter.xml
|   `-- websites.xml
`-- scrapedxml
    |-- representatives_debates
    |   `-- 2007-09-20.xml
    `-- senate_debates
        `-- 2007-09-20.xml

Note that there are a lot of files under scrapedxml. There is one file for each day that each house sat since the beginning of 2006. You probably don’t want to download them all at once. To test things out just start with a couple.

If you have the commandline tool wget installed you can easily download a sample of XML files:

$ wget -r http://data.openaustralia.org -I members -A *.xml
$ wget -r http://data.openaustralia.org -I scrapedxml -A 2007-09-20.xml
$ mv data.openaustralia.org openaustralia/pwdata

Load data into the database

Move to the directory with the database loading scripts:

$ cd openaustralia/twfy/scripts

To load the data in members/ministers.xml, members/people.xml, members/representatives.xml, members/senators.xml and scrapedxml:

$ ./xml2db.pl --debates --lordsdebates --members --all

To load the data in members/links-abc-qanda.xml, members/twitter.xml and members/websites.xml:

$ ./mpinfoin.pl links