Realtime gtfs schedules download inconsistent?


#1

So i’m authenticating quite fine, HOWEVER the download from gtfs schedule is really odd - see the curls below, usually I get an ~11MB file that is not a valid zip. Every now and then I get an ~6MB file that contains the csv files I’m expecting. All these are spaced within a minute of each other.

what’s happening behind the scenes for this to happen? and any chance you could provide an expected crc for the download as a header?

feels like a :monorail:

~  $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > text.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.3M    0 11.3M    0     0   593k      0 --:--:--  0:00:19 --:--:--  624k
################## <---- this was a working download..
~  $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 6396k    0 6396k    0     0   537k      0 --:--:--  0:00:11 --:--:--  601k
##################
~  $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.3M    0 11.3M    0     0   598k      0 --:--:--  0:00:19 --:--:--  636k
~  $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.3M    0 11.3M    0     0   539k      0 --:--:--  0:00:21 --:--:--  630k
~  $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.3M    0 11.3M    0     0   598k      0 --:--:--  0:00:19 --:--:--  633k
~  $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.3M    0 11.3M    0     0   597k      0 --:--:--  0:00:19 --:--:--  629k

and here is a verbose one for you too:

~  $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" -vvv > text.zip
Note: Unnecessary use of -X or --request, GET is already inferred.
* timeout on name lookup is not supported
*   Trying 52.63.188.104...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to api.transport.nsw.gov.au (52.63.188.104) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [89 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4149 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*        subject: C=AU; ST=New South Wales; L=Chippendale; O=Transport for NSW; OU=Group IT; CN=api.transport.nsw.gov.au
*        start date: Mar 16 00:00:00 2016 GMT
*        expire date: Mar 17 23:59:59 2017 GMT
*        subjectAltName: api.transport.nsw.gov.au matched
*        issuer: C=US; O=Symantec Corporation; OU=Symantec Trust Network; CN=Symantec Class 3 Secure Server CA - G4
*        SSL certificate verify ok.
} [5 bytes data]
> GET /v1/gtfs/schedule/sydneytrains HTTP/1.1
> Host: api.transport.nsw.gov.au
> User-Agent: curl/7.46.0
> Accept: */*
> Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04
>
{ [5 bytes data]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0< HTTP/1.1 200 OK
< Cache-control: no-cache="set-cookie"
< Content-Type: application/zip
< Date: Wed, 20 Apr 2016 01:24:29 GMT
< Server: Apache-Coyote/1.1
< Set-Cookie: AWSELB=777BE349184966AA129955C33D987E6DD8D703B5EF4A7519C0F239B0DEF5976D23C251E17BD29C40F168F479A3C3324706C07F18B9FF538671E0CB5671F32FDB3B65D0A6F9;PATH=/
< transfer-encoding: chunked
< Connection: keep-alive
<
{ [8852 bytes data]
100 11.3M    0 11.3M    0     0   573k      0 --:--:--  0:00:20 --:--:--  629k
* Connection #0 to host api.transport.nsw.gov.au left intact


TfNSW real-time map
#2

This seems to have been reported in this thread too.


#3

yeah well 1/8 attempts to get the right data can’t be too bad?? :expressionless:

we’re users of the TDX and that’s working fine for the same data…


#4

Can you advise if this is still an issue for you all?


#5

whoops - yes this was fixed AND has been consistent ever since.

thanks!