Sydneytrains vehiclepos v2 feed intermittent

Real-time Public Transport
1

Since Saturday Aug 20th, the sydneytrains v2 GTFS-R vehicle position feed has stopped updating for up to 12 hours each day.

Is this a bug or some form of scheduled maintenance?

2

HI @Tugzrida,

thank you for your message. Sydney Trains are aware of intermittent latency issues with their V2 GTFS-R vehicle position feeds and are investigating. We will provide an update once we receive one.

1 Like
3

Hi @Tugzrida ,

just to follow up - a ticket was raised on Tuesday to monitor this issue with Sydney Trains.

Please note that the feed has gone stale today since 11:26am. There are no updates from Sydney Trains at this stage.

Cheers,
Marcela

4

Thanks @Marcie!

I’m managing a fallback to the V1 feed when the V2 feed is stale, though obviously getting the V2 feed back reliably would be preferable as the V1 feed doesn’t contain data for vehicles outside ATRICS territory.

1 Like
5

Just an update from Sydney Trains team on this issue:

There is a scheduled release on Sep 10 that the team are hoping will alleviate this issue. We do not yet understand the root cause.

It was reported as stale again today.

6

Thanks @Marcie, I’ll have my fingers crossed for Sep 10

1 Like
7

Fingers crossed the release tomorrow will fix this issue :crossed_fingers:

8

The issue appears to be a little improved but not completely resolved.

The v2 feed was patchy or down completely for around 8 hours on the 11th, and then 15-30 mins every day since.

9

Hi @Tugzrida thanks for updating us. Ill let the team know and get back to you with an update.

Cheers,
Marcela

1 Like
10

Just updating here: the v2 feed is currently becoming unstable or going down completely for 1-2 hours every weekday morning during peak.

This is different behaviour to when the issue first surfaced in late August, so it seems the September 10 update did improve things somewhat.

I’ve been logging data associated with the feed, viewable in the attached screenshot and Grafana snapshot.

While the morning peak is generally larger in terms of number of active vehicles, I can’t 100% link the number of active vehicles to the feed going down, as it often happens before the crest of the peak, so doesn’t quite seem to be related to something being overloaded by too many vehicles.

Screen Shot 2022-09-27 at 10.52.30
Screen Shot 2022-09-27 at 10.52.302738×1348 307 KB

https://snapshots.raintank.io/dashboard/snapshot/ZvKWItkUHQbIE0Xd4x4bgm6Mk1yfYHVk

11

Hi @Tugzrida ,

a heads up that Sydney Trains have a release Sat 12 Nov, which includes a patch to address issues seen in V2 vehiclepos reatime feed. There is no outage for users.

We’ll be monitoring the feed throughout next week :slight_smile:

1 Like
12

Very happy to report that since the update last Saturday, there have been almost no interruptions to the v2 feed, and the ones that did occur lasted only 1-2 minutes.

Thanks to everyone involved :grin:

1 Like
13

Thanks for the message @Tugzrida, great news!

1 Like