So i’m authenticating quite fine, HOWEVER the download from gtfs schedule is really odd - see the curls below, usually I get an ~11MB file that is not a valid zip. Every now and then I get an ~6MB file that contains the csv files I’m expecting. All these are spaced within a minute of each other.
what’s happening behind the scenes for this to happen? and any chance you could provide an expected crc for the download as a header?
feels like a
~ $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > text.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11.3M 0 11.3M 0 0 593k 0 --:--:-- 0:00:19 --:--:-- 624k
################## <---- this was a working download..
~ $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6396k 0 6396k 0 0 537k 0 --:--:-- 0:00:11 --:--:-- 601k
##################
~ $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11.3M 0 11.3M 0 0 598k 0 --:--:-- 0:00:19 --:--:-- 636k
~ $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11.3M 0 11.3M 0 0 539k 0 --:--:-- 0:00:21 --:--:-- 630k
~ $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11.3M 0 11.3M 0 0 598k 0 --:--:-- 0:00:19 --:--:-- 633k
~ $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" > test1.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11.3M 0 11.3M 0 0 597k 0 --:--:-- 0:00:19 --:--:-- 629k
and here is a verbose one for you too:
~ $ curl -X GET https://api.transport.nsw.gov.au/v1/gtfs/schedule/sydneytrains -H "Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04" -vvv > text.zip
Note: Unnecessary use of -X or --request, GET is already inferred.
* timeout on name lookup is not supported
* Trying 52.63.188.104...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to api.transport.nsw.gov.au (52.63.188.104) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [89 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4149 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=AU; ST=New South Wales; L=Chippendale; O=Transport for NSW; OU=Group IT; CN=api.transport.nsw.gov.au
* start date: Mar 16 00:00:00 2016 GMT
* expire date: Mar 17 23:59:59 2017 GMT
* subjectAltName: api.transport.nsw.gov.au matched
* issuer: C=US; O=Symantec Corporation; OU=Symantec Trust Network; CN=Symantec Class 3 Secure Server CA - G4
* SSL certificate verify ok.
} [5 bytes data]
> GET /v1/gtfs/schedule/sydneytrains HTTP/1.1
> Host: api.transport.nsw.gov.au
> User-Agent: curl/7.46.0
> Accept: */*
> Authorization: Bearer 02da0592-86b8-4d90-8862-b9e88590bf04
>
{ [5 bytes data]
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0< HTTP/1.1 200 OK
< Cache-control: no-cache="set-cookie"
< Content-Type: application/zip
< Date: Wed, 20 Apr 2016 01:24:29 GMT
< Server: Apache-Coyote/1.1
< Set-Cookie: AWSELB=777BE349184966AA129955C33D987E6DD8D703B5EF4A7519C0F239B0DEF5976D23C251E17BD29C40F168F479A3C3324706C07F18B9FF538671E0CB5671F32FDB3B65D0A6F9;PATH=/
< transfer-encoding: chunked
< Connection: keep-alive
<
{ [8852 bytes data]
100 11.3M 0 11.3M 0 0 573k 0 --:--:-- 0:00:20 --:--:-- 629k
* Connection #0 to host api.transport.nsw.gov.au left intact