# Are we hitting your API right?
**URL:** https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447
**Category:** Real-time Public Transport
**Created:** [November 9, 2018, 6:23am UTC](https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447 "2018-11-09T06:23:03Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@simonGIS](https://opendataforum.transport.nsw.gov.au/u/simonGIS)
#### Post date: [November 9, 2018, 6:23am UTC](https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447/1 "2018-11-09T06:23:03Z")
We have built [a simple front-end application](https://www.bgtdevhub.com/TfNSW/index.html) where you can see the data on a map.
We also have a [back-end AWS Lamda function](https://github.com/bgtdevhub/tfnsw-ago-lambda) that is connecting to the Real-Time Locations API and upserting into a Hosted Feature Layer in ArcGIS Online.
Our first observation is that it looks like **we have too many vehicles** to what we would expect.
Is this something we have done wrong on our side, or at the API end?
- Poll every 30 seconds the API
- Using Vehicle\_ID as unique field
- [Checking for duplicates](https://github.com/bgtdevhub/tfnsw-ago-lambda/blob/master/routes/routes.js#L99) on the API responses
- Upserting into the Esri layer
- Every minute we check for any records older than 1 minute and delete
1. We initially spotted that _vehicle\_id_ is not always unique. In one request, often getting a number of duplicates with different timestamps, but same _vehicle\_id._
2. We also spotted that sometimes the _vehicle\_id_ sends through a really long string. Is this expected?
---
### Author:  [@jxeeno](https://opendataforum.transport.nsw.gov.au/u/jxeeno)
#### Post date: [November 12, 2018, 9:12am UTC](https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447/2 "2018-11-12T09:12:44Z")
> [@simonGIS](#):
>
> Our first observation is that it looks like **we have too many vehicles** to what we would expect.
Probably the first thing to look out for is that the bus feed reports vehicle locations for trips which are starting up to 30 minutes in advance. This is useful for users who want to see where their bus is prior to the trip starting, but less so when you’re building a network view. You can filter those out by checking the `tfnsw_vehicle_descriptor` \> `performing_prior_trip` = `true` field.
> [@simonGIS](#):
>
> We initially spotted that _vehicle\_id_ is not always unique. In one request, often getting a number of duplicates with different timestamps, but same _vehicle\_id._
Yeah, I’ve noticed some instances (for unscheduled bus trips) where there are duplicate `vehicle_id` values. Your solution of checking for a duplicate ID is probably sufficient since the number of affected trips is generally small, and is usually for trips which don’t correspond to a published timetable. You could also append a number at the end of the ID to ensure its uniqueness, but you’ll likely lose identifier consistency if you’re planning on building a time-series out of the data.
> [@simonGIS](#):
>
> We also spotted that sometimes the _vehicle\_id_ sends through a really long string. Is this expected?
Yes, that’s an Indian Pacific service which has 29 train carriages (note `R.29` in the `trip_id`). For the Sydney Trains feed, the vehicle ID is formed by joining a series of numbers - one representing each a masked carriage number. Hence, why it’s that long for that particular train service.
---
### Author:  [@simonGIS](https://opendataforum.transport.nsw.gov.au/u/simonGIS)
#### Post date: [November 12, 2018, 11:52am UTC](https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447/3 "2018-11-12T11:52:04Z")
Huge thanks for taking the time to respond.
> [@jxeeno](#):
>
> `tfnsw_vehicle_descriptor`
Missed that entirely. We will update our code accordingly. And also catch for the longer vehicle ID, didn’t actually realise that it’s a concatenation of carriage IDs.
Will report back once we make some changes.
Thanks again!
---
### Author:  [@jxeeno](https://opendataforum.transport.nsw.gov.au/u/jxeeno)
#### Post date: [November 13, 2018, 12:07pm UTC](https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447/4 "2018-11-13T12:07:24Z")
> [@simonGIS](#):
>
> And also catch for the longer vehicle ID, didn’t actually realise that it’s a concatenation of carriage IDs.
They aren’t strictly carriage IDs (see this conversation: [Sydney Trains Realtime Carriage IDs](https://opendataforum.transport.nsw.gov.au/t/sydney-trains-realtime-carriage-ids/655)), but its length and concatenation is linked with the number of cars in the scheduled train consist.
---
### Author:  [@simonGIS](https://opendataforum.transport.nsw.gov.au/u/simonGIS)
#### Post date: [November 14, 2018, 2:59am UTC](https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447/5 "2018-11-14T02:59:44Z")
> [@jxeeno](#):
>
> You can filter those out by checking the `tfnsw_vehicle_descriptor` \> `performing_prior_trip` = `true` field.
False= actual position of buses
True = position of future buses? And we dont want these.
Am I [reading that](https://opendata.transport.nsw.gov.au/sites/default/files/tfnsw-gtfs-realtime.proto_.txt) right?
We made the changes to our Github repo
---
### Author:  [@jxeeno](https://opendataforum.transport.nsw.gov.au/u/jxeeno)
#### Post date: [November 14, 2018, 4:28am UTC](https://opendataforum.transport.nsw.gov.au/t/are-we-hitting-your-api-right/1447/6 "2018-11-14T04:28:39Z")
I think you’ve got it. Both are actual positions, but associated with a current and future trip – i.e.
`performing_prior_trip = false`: actual position of bus, associated with current trip
`performing_prior_trip = true`: actual position of bus, associated with next trip – can be disregarded for network view purposes.
A note on the vehicle ID for trains and your work around there. If you have a 40 character limit for your primary key, it’s generally safe to simply truncate to the first 40 characters for the train vehicle ID. That way, you don’t have to hard code an alternative vehicle ID.