Journey to Work (JTW) 2016


#1

This dataset contains a snapshot of commuting patterns for Census day 2016 (Tuesday 9th August). It is a complete enumeration via a self-completion Census form.


This is a companion discussion topic for the original entry at https://opendata.transport.nsw.gov.au/dataset/journey-work-jtw-2016

#2

Hi.

I have downloaded the dataset JTW2016_csv.rar and unarchived it. However, two of the files failed. See messages below. Wondering if anyone else has experienced similar problems.

Thanks.

K.

Table 12 OSA2 by DSA2 by Mode15.csv (-1985867255 B)… Failed! (Attempted to read more data than was available)
Table 13 OTZ by DTZ by Mode15.csv (300135148 B)… Failed! (Wrong checksum)


#3

Hi @k4werri, yea looks like there’s a problem with those two files. Looking into it and hopefully will get it resolved soon. Thanks


#4

Thanks …

K.


#5

Hi @k4werri, the problem is to do with the large file size of those two tables. We’re looking at spliting the files into more manageable sizes. Apologies about this, we’ll try to get it resolved as soon as possible. Thanks


#6

Hi Alejandro
I too am having trouble with Table 13.

R has hit a memory limit and terminated part way through importing. It seems to me like the problem is that every field is coded which is what totally bloats the file size. The csv is a ridiculous 26GB!! R handles Table 12 without difficulty.

I’d be happy with uncoded data at the simplest level possible, with code frames and equivalences to aggregate from, say mode xx or from TZ up to SA2, SA3, SA4 etc.

Also, the single zipped download is huge for those of us not yet on NBN or corporate download speeds. I’m sure others would really appreciate the option of taking only the tables they wanted, or downloading them individually over time.
cheers
Gillian


#7

More comments:

Table 12 won’t import into MS Access, too big.

Why does Table 12 have the additional geographic codes (SA3/4) for the destination SA2, but not for the origin SA2? And no O_SA2 code, just the name, which makes equivalencing to other sources of SA3/4 a bit painful.


#8

@GPAK thanks for the feedback. We are currently in the process of pulling out from Table 13 non-essential text fields like SA Names. This will reduce the file size of Table 13 considerably.

Also, we be uploading individual .zip files for each table shortly.

Regarding Table 12, we recommend first processing the data in R (or other programming language) to extract only the required data for analysis.

We have attached SA2 codes to Table 12 and will be uploading in the coming days.