New API: Notifications

• Date: October 27, 2016

We have a new API!

Getting from A to B is a lot more than just planning a journey to your destination, or following a schedule when using public transport. Travel is impacted by all sorts of things: that pothole is finally getting fixed, the stop you always use has moved, the ABs are trying to win again, or Slipknot is calling metal fans out of their basements - something is always happening in this city of ours.

Auckland Transport is now publishing the data they have through their api portal. It is a first “stab” at this, and clearly not particularly polished. We want to hear from you what you’d like to do with the data - what form should it take to be useful for you?

We added some details about this API to ATLabs, please see our getting started section for everything (?) worth knowing.

Leave us a note on Gitter; we’ll collect the feedback, and approach Auckland Transport to make this API the best it can be. Sounds good?

Civic Hacking

• Date: October 17, 2016

Earlier this year I was asked to speak to the IPENZ Transportation Group’s conference and show transport engineers some of the ways in which technology is changing their world. I introduced them to the concept of civic hacking and explained how civic hackers were playing a significant role in the future of transport. In an effort to help translate between the cultures of hackers and transport engineers, I explained it like this:

Hacking defined

I have spent a lot of time explaining the concept of ‘hacking’ to people who have only heard the term used in relation to dubious practices on the internet. I pondered what might be a suitable analogy for describing ‘hacking’ to people who spend a large amount of their time looking at vehicles, and came up with this:

Maybe it was because my dad was a hot rodder when he was young, but I am a total fan of grungy functional things that exude personality, rather than the more civilised precision things that look too pretty to be messed with! Which is an interesting insight in itself — things often need to be a little worn out or broken before people have the urge to mess with them.

The 3 forms of Civic Hacking

In the civic hacking scene, there are basically three forms of hacking — software hacks, hardware hacks, and non-tech hacks. Software hackers are kind of like custom car painters — they totally transform the look and feel of the vehicle, making people want to spend time cruising the streets showing off!

Hardware hackers are the equivalent of the grease monkeys that get down and dirty to turn their machines into white-knuckled rides to adrenaline heaven.

Then there are the non-tech folks. They are the ‘makers’ who can re-imagine and transform ordinary household objects into manifestations of problem-solving genius. Want to turn a bicycle into a sailboat? These are the people to do it.

Traffic engineering hacks

Hacking is actually something that has been done by traffic engineers for decades — it’s just not a term that they use. Long before the concrete for new roads, kerbs, and traffic islands is poured, temporary traffic management features are put in place to test driver behaviour. It also helps to know whether those large emergency service vehicles can negotiate the carriageway before the barriers are installed. In the same way that engineers experiment with new layouts, hacking is all about doing things lean.

Inside the hacker ethos

As I looked at the types of things that underpin the hackathon scene, there are a few things that jump out at me that could be described as a hacker’s ethos.

The people who participate in hackathons tend to be those that love to solve problems from a first person perspective, with a focus on getting shit done. These tend to be similar to those in the service design and ‘startup’ scene as well, which all tend to adopt a user-centred design (lean) approach to solving problems.

This ethos translates into 5 principles typically held by the civic hacking community:

Collaboration | Communication | Openness | Interoperability | User-Centered

What they look like in practice

The software hacking scene tends to consist of hackathons and online communities that create new tools, optimise systems, minimise waste, and use data through APIs. Hardware hacks use sensors to create data and create interventions through hardware devices that contribute to the internet of things, or through the use of 3D printing technology. In the civic context, non-tech hackers are often found in maker spaces, or on the street in the form of placemakers or tactical urbanists that set up urban labs or refineries to conduct real world tests to measure community impacts.

Enablers of hacking

Hacking is not new. In fact at its most elemental form, finding creative solutions to tricky problems is something that humans have done for ever. But there are some things that have enabled hacking as a ‘movement’ to gain momentum over recent years.

Technology | Communication | Collaboration | Empowerment

Each of these factors is probably more accurately described as ‘states’ or ways of looking at and interacting with the world in which we live. Because they are not ‘things’ that can be taken away, it is safe to say that ‘hacking’ is here to stay.

Why it’s effective

Hacking is very effective way of solving problems. Why? Because it’s fast, it’s affordable, and it creates tangible solutions. In the same way that hot rods still attract stares and sideways glances from passers-by, hacking has a tendency to show up the establishment by drawing attention to things that would otherwise be taken for granted. There is nothing like a bit of public shaming to make an impact!

You won’t beat them

My take home message for the transport engineers is simple. Hacking is a thing, and they are setting about changing the world whether you like it or not. The best thing to do in this situation is of course to join them. Who’s up for a custom paint job?

New: Notifications API

• Date: October 12, 2016

The Notifications API publishes event data that has an impact on travel in general - public transport, and Auckland’s road network.

Format

The format for notifications is currently derived directly from the backend systems - and as such not (yet) normalised. We don’t know yet how these APIs are going to be used, therefore making up a structure and format seemed premature. Please let us know your suggestions.

Notifications API

The Notifications API provides the aggregated results of all categories of the Notifications by Category API.

Notifications by Category

Notifications are issued in several categories:

• moved_stop (Stops not serviced)
• road (Roadworks)
• events (Public events like sports or concerts)
• real_time
• highway (Highway maintenance)

All notifications returned as part of this API (The Notifications by Stop API has a different format) follow a base format:

{
   "time": "...",
   "data": [
      {
         "id": ...,
         "orderingDate": ...,
         "type": ... (category),
         "title": ...,
         "description": ...,
         ...
      },
      ...
   ]
}

Depending on the category, other fields are returned; for example, a road-type event will include subtype, startDate and endDate, a moved stop-event will include stopId, etc.

The id field is unique within the dataset returned. However, it is not guaranteed to be unique over time. Don’t use it to synchronise data.

Special note: The highway-type event returns expectedResolution, which is a string value - not a date. The value can be something like “Until further notice”, or a date like “26/10/2016 06:00”; note that the date is not in ISO 8601/RFC 3339. Care should be taken when attempting to parse the date.

Notifications by Stop

The Notifications by Stop API returns details about new stops to be used if a particular public transport stop is currently not serviced/used. Similar data us published under the “moved_stops” category of the Notifications by Category API; this API adds the alternative stops to be used.

The origin of the data published is different to the origin of the Notifications by Category API. The data usually aligns, but it doesn’t always have to be the case.

The format of the data returned looks like this:

[
   {
      "attributes": {
         "OBJECTID": ...,
         "AFFECTEDSTOPID": ...,
         "AFFECTEDROUTE": ...,
         "NEWSTOPID": ...,
         "NEWSTOPNAME": ...,
         "NEWSTOPEXISTINGROUTE": ...,
         "NEWSTOPSTATUS": ...,
         "NEWSTOPNOTE": ...
      }
   },
   ...
]

The API may return multiple “new” stops for an affected existing stop. Multiple stops indicate that some services for the stop have been re-routed to one “new” stop, and other services are routed to the other “new” stop(s).

The AFFECTEDROUTE field contains a comma-separated list of, well, affected bus services. Note that the affected services always includes all routes from the affected stop, not the services routed through the new stop. In other words, the list of affected routes is always the same, even if multiple new stops are returned.

The GTFS Routes By Stop API can be used to determine which routes exactly are serviced by which “new” stop.

Engaging civic-minded developers

• Date: October 12, 2016

No-one wants to work for free.

Only a handful are willing to take the time, energy and focus they normally reserve for work, family, friends and apply them to ideas that are seemingly unformed, goals that are best described as aspirational or projects that may never see the light of day.

Even these folks don’t want to work for free.

So why do these people be they experienced developers, UX designers, project managers, customer representatives, tech students or even “people with an interest in computers” put aside time to bring their considerable skills to “civic tech”?

Ok, a definition is needed for “civic tech”, and here’s mine:

The application of technology to open up opportunities to solve local¹ issues by the community directly affected

Here’s one example of such a problem/solution - there are too many taxis with one person driving from the airport thus creating traffic flow issues. Maybe we could create and disseminate an app to passengers that guides those going to the same destination into one cab - 4 cabs with 1 person is now 1 cab with 4 people.

This was an actual app, produced during a hackathon at which app developers, infrastructure experts, city planners and members of the community came together to use the Internet and open systems to sort out their problem.

So that’s what civic-minded developers do, they give their skills, their time and their passion to help communities resolve local issues. But why? Why would a highly paid developer forgo a weekend or a few days salary to do such a thing?

It’s not for free; that’s for sure.

At Hack Miramar, we believe there are 3 main drivers deep within developers that have them volunteering to help.

It’s a funny old world, engineering eh. It’s fast moving with today’s greatest development paradigm quickly becoming tomorrows “old world”. It’s also a slow moving world, with the problems of yesteryear still being the focus for many “IT departments” today.

In this fast yet slow world of the professional developers’ world, there is a freedom in “civic tech”. Rarely are traditional problems presented, these are for the Council, the Government agency, “them” to sort out, but civic tech attracts difficult problems or even wicked problems calling on a multitudes of skills, with data from here mashed with systems from there and hung together with a language at the very cutting edge of tech.

Developers love to show you what they can do, and how fast and how beautifully. And civic developer will gladly “do their thing”, stretch their technical wings as long as peer recognition is forthcoming because they love people.

Forget the stereotype you may have of developers living in the basement, coding up a storm and never wanting to see anyone. Developers love people … they may be very picky about who those people are, but gaining peer recognition is one of their joys of life. For some that may be on a grand scale winning prizes but for others is may be a quiet, “Wow, that’s a fantastic solution, top work”, from someone she has long looked up to.

Developers, like everyone but certainly civic developers, also love to know that their work will be used, that it will make a difference, that it will help. And the best will wait, as the UX designers nut out the intricacies of the flow, as the hustlers wrangle as much data as they can, they will wait, listen and learn, “These are good people doing their bit, my time will come.”

And then they get into it, huddled together, sharing code, Slacking out those grrr moments and constantly making sure they’re all working together. It may look like quiet, solitary work, but it’s anything but.

With the problem defined, the goal whiteboard’ed and the sprints are on … but what’s the ‘Playdo’, what will they be using, what data will they have, what APIs are available?

Many civic developers come from a ‘closed shop’ at work where they have a limited and more importantly known set of resources to work with. In the civic world, it is much more open with open data, open source, free flowing of approaches and agility of mind the order of the day. With access to new resources (particularly data but also domain experts), these civic developers can stretch their wings, explore new patterns and gain deeper skill sets and learn innovative techniques.

So how do you engage with developers, how do we grow the pool of ‘civic developers’, by ensuring we let them show off (on their own terms), bring them interesting and like-minded people to play with and supply new, even exclusive, data and resources.

No-one wants to work for free, but everyone wants to grow, make a difference and solve problems.

Good times at API Meetup

• Date: October 05, 2016

Good turnout for last night’s Auckland’s API and Microservices meetup, the first time we presented the ATLabs concepts to the outside world (well, around 50 …). We received some good feedback, with ideas already flowing how we can improve transport in Auckland - keep them coming through our gitter chat.

Personally speaking, I’m quite relieved that it went well. Quite a few evenings in addition to my “regular” job went into thinking about how to present ATLabs to a group of (mostly) software developers - particularly since this wasn’t a talk about technical details, but to explain the idea of a community that forms a partnership with public entities like Auckland Transport.

Thank you to all who attended:

The pdf of the presentation can be found here.

Registration

• Date: August 15, 2016

Just like accessing almost any other API, the first step is the registration. We do that so we can plan infrastructure and resources better - and also to stop those few trying to abuse our API, and make life worse for all of us.

Thankfully, registration is simple (and free): Head to https://dev-portal.at.govt.nz/, click on the “sign up” button, and fill in the details. Afterwards, you’ll receive a confirmation email; click on the link - and you are good to go.

To use an API, you will need to subscribe to it: Go to “products”, and select the product “Developers” (at the moment there is only this one), and give it the name of your application. That product grants you access to 600 calls/minute, and 35,000 calls/week - should be plenty to start? Afterwards, you’ll receive a primary and secondary key.

You can use either of these keys for your application - they are there if you want to switch to a new key for an upgrade (rolling upgrades), or to grant temporary access. See the Microsoft Azure Team Blog for details.

Once you have an app production-ready, you may need more calls than the “developers” product allows. Please get in contact with us then (preferably a bit earlier), so we can check that we have enough resources for your app.

Known Issues

• Date: August 15, 2016

Well, we’d like to be perfect, everything is working as designed all the time … but, you know: stuff happens … occasionally … it is complicated … it was the dog.

There are two “categories” of known issues: API related, and ATLabs related. API related issues (e.g. slow, wrong data, error messages) are managed by Auckland Transport, while ATLabs related issues are our domain. We do, however, have good connections into Auckland Transport, and part of the team that is running ATLabs is also supporting Auckland Transport’s API. So there you go: any issues, contact us, easy as.

Here are the things we know about:

API Issues

Labs Issues

Good news, we don’t know of any.

Authorisation

• Date: August 15, 2016

As explained in the registration post, once you register for an API you’ll receive two keys: A primary and a secondary key. You can use either, they are there to allow rolling updates and temporary access.

The key must be supplied in an HTTP header called Ocp-Apim-Subscription-Key.

For example:

Groovy

def key = 'xxx'

def gtfs = new RESTClient('https://api.at.govt.nz/v2/gtfs/')
gtfs.defaultRequestHeaders.'Ocp-Apim-Subscription-Key' = key

gtfs.get(path: '...')

…

Response and error handling

• Date: August 15, 2016

The response format for all API calls is unified, and - silently brushing over potential bugs - all behave the same way.

The response contains a status field, which can either be “OK” or “Error”. Successful calls will have the response field populated, and the error field set to null. For errors, it’s the other way around. Makes sense?

The status codes for successful operations are the HTTP status codes in the 2xx range , errors are in the 5xx range. We got - at least - that one right.

The format for a successful call looks like this:

{
  "status": "OK",
  "response": ...,
  "error": null
}

All API calls also support the JSONP format, by adding the callback parameter to the URL. The response looks like this (assuming ‘cb’ as the callback value, and formatted for readability):

/**/
typeof cb === 'function' && cb({
   "status":"OK",
   "response": ...,
   "error":null
});

If an error should occur in the backend, then the response will look like this:

{
  "status": "Error",
  "response": null,
  "error": {
    "name": "...",
    "message": "..."
  }
}

API access errors, like insufficient permissions, are formatted differently, and look like this:

{
  "statusCode": ...,
  "message": "..."
}

The statusCode field matches the HTTP response status code.

Should be fairly self-explanatory.

Tools API

• Date: August 15, 2016

The tools API covers miscellaneous “tools” - at the moment, the journey planner and the geo coder.

Journey Planner

The API for the geo coder is currently “public restricted”, i.e. only available to partners. Please contact us to either make it public, or get a partner access key.

The journey planner API, as the name suggests, connects to an Auckland Transport system that allows users to plan their journeys. Given a start and end point, time, and some preferences, it calculates a few suggestions for journeys, including walkways, and bus/train/ferry journeys, and also the expected fares. It is the API behind the journey planner on Auckland Transport’s website.

The journey planner engine “knows” about Auckland’s road and walkway network, the routes for buses, trains and ferries, all stops, and the public transport schedule. Routes, stops and schedule are all made public as part of the GTFS API - in fact, the same published GTFS feed is regularly imported into the journey planner engine.

The fares are added to the journey suggestions by Auckland Transport fare calculator (application).

The response from the journey planner API contains a few structures:

Itinerary

An itinerary is the suggested trip end-to-end from the given start point to the given end point. It contains the expected start and end times, the duration, the expected fare, and one or more legs (see below).

The (shortened) structure looks like this:

{
   "duration": ...,
   "startTime": "...",
   "endTime": "...",
   ...
   "fareAdult": ...,
   "fareChild": ...,
   "fareHopAdult": ...,
   "fareHopChild": ...,
   "fareHopTertiary": ...,
   "legs": [...]
}

Leg

A leg is a segment or section of a journey on a particular service (bus/train/ferry), or a walkway. All modes of legs, like the itinerary, contain start and end times, the duration, the fare, and also the geographical locations of start and end.

The (shortened) structure looks like this:

{
   "startTime": "...",
   "endTime": "...",
   "duration": ...,
   "from": "...",
   "to": "...",
   ...
   "fareAdult": ...,
   "fareChild": ...,
   "fareHopAdult": ...,
   "fareHopChild": ...,
   "fareHopTertiary": ...,
   "mode": "WALK|BUS|TRAIN|FERRY",
   ...
   "legGeometry": {
      "points": "...",
      "length": ...
   }
}

The legGeometry is using Google’s polyline encoding algorithm to encode the leg’s travel path. Popular JavaScript map libraries like Leaflet support this format either directly, or through plugins.

Further, depending on the mode (walk/bus/train/ferry) of a leg, the API returns additional data:

For walk mode:

{
   ...
   "distance": "...",
   "distanceExact": ...,
   "mode": "WALK",
   ...
}  

For bus, train and ferry modes:

{
   ...
   "routeCode": "...",
   "routeName": "...",
   "tripId": "...",
   "mode": "BUS",
   ...
   "stops": [
      {
         "code": "...",
         "geometry": {
            "type": "point",
            "data": [...],
            "encoded": true|false
         }
      },
      ...
   ],
   ...
}

The tripId field refers to the GTFS API Trip entity, and the code field within the stops list refers to the GTFS Stop entity.

Geo Coder

The API for the geo coder is currently “public restricted”, i.e. only available to partners. Please contact us to either make it public, or get a partner access key.

The geo coder API connects to an Auckland Transport geo coder service, which translates geographical locations (latitude, longitude) into addresses - called reverse encoding -, and vice versa - called forward encoding.

Please note that the translation process of the geo coder is quite resource-intensive. Frequent calls to the API can cause the process to slow down - for all users. The backend service is not easy to scale - so use it wisely, and in moderation.

Reverse Encoding

The reverse encoding API, as noted above, translates a given geographical location into addresses. The geocoder uses an internal algorithm to return not only the exact address, but also surrounding addresses, and public transport stops.

Forward Encoding

The forward encoding API, indirectly, has two functions: First, as noted above, to translate addresses into geographical locations. Secondly, as it uses a given search string, it also works to distinguish between streets that share the same name, e.g. Drake Street in Auckland Central, and Drake Street in Howick.

The geocoder also has (limited) understanding of intersections and public transport stops: Search queries that start with or contain “corner”, “cnr”, “and”, “&” or “&&” are interpreted as an intersection. The geo coder will also try to interpret search queries that only consist of numbers as public transport stops.

Realtime API

• Date: August 15, 2016

The purpose of the Realtime API is to provide real-time data for each of Auckland’s public transport modes: buses, trains and ferries.

At the time of writing, only real-time data for buses is available. Integration of real-time train data is in progress, and ferries are planned.

Most of the public transport data available follow the model of Google’s Realtime Transit Specification (GTFS Realtime). A good place to start is the GTFS Realtime Reference. This will tell you a lot about the domain: entities, trip updates, vehicle positions and alerts (to name the top-level structural elements).

Format

The GTFS Realtime specification technically uses Google’s Protocol Buffers as its serialisation format. The Realtime API offers the same in JSON, by default, but can also return protobuf if the Accept header is set to “application/x-protobuf”.

Naming Convention

The API uses the convention described in the GTFS Realtime Specification (snake case) as its naming convention. For example

required FeedHeader header = 1;
repeated FeedEntity entity = 2;

becomes

{
   "header": {
      ...
   },
   "entity": [
      ...
   ]
   ...
}

The same applies to the GTFS API: You may not like the underscores, but if you want to re-use their documentation it is best to follow their convention. Please don’t ask for camel/pascal/random case.

Realtime

At the moment, the feed is updated at least every 30 seconds. The GTFS Realtime specification provisions for, but does not currently allow incremental/differential updates. i.e. The GTFS Realtime API behaves to spec.

Realtime API

The Realtime API provides the aggregated results of the Trip Update API (vehicle progress along a trip), the Vehicle Position API (positioning information for a vehicle), and the Alert API (notification of incidents) - please refer to the subsequent sections for further details.

The API returns all updates by default, but query parameters can be specified to filter by vehicles or trips.

Trip Update API

The Trip Update API provides progress updates for all vehicles along a trip within Auckland Transport’s network.

The API returns all updates by default, but query parameters can be specified to filter by vehicles or trips.

Trip Update

Real-time update on vehicle progress along a trip.

According to the GTFS Realtime specification, the Stop Time Update has a cardinality of “repeated”. The JSON feed only allows a single update - which is incorrect. Future versions of this API should fix this.

The fields/structures trip, vehicle and stop_time_update, as well as timestamp, are provided. The optional and experimental delay field is omitted.

Trip Descriptor

A descriptor that identifies an instance of a GTFS trip, or all instances of a trip along a route.

The trip_id and route_id refer to the trips and routes published through the GTFS API. Please note the particular management of routes, explained in the GTFS API.

The fields direction_id, as well as start_time and start_date are not set. They can be retrieved from the GTFS API if required.

The schedule_relationship is always set to 0 (SCHEDULED). The backend system does not process added, unscheduled or cancelled trips.

Vehicle Descriptor

Identification information for the vehicle performing the trip.

Currently, only the id is set. The id is provided by AVL (automatic vehicle location) devices installed on buses, trains and ferries. It is a string, as per specs, although it currently looks numerical for buses. For trains, it looks very different, and even includes spaces.

The id is just that: a vehicle identifier. As vehicles can be used for different routes, or even swapped (in the case of a vehicle break down, for example), the id should not be used to deduce routes or similar. Secondly, the id (& format) may change in future, so please don’t try to read anything else into it.

Stop Time Event

Timing information for a single predicted event (either arrival or departure).

Both delay and time fields are set. Note that the delay can be negative reasonably often, particularly for buses.

The uncertainty field is currently omitted. According to the specs it probably should be set, given the API is returning the delay.

Stop Time Update

Realtime update for arrival and departure events for a given stop on a trip. Updates can be supplied for both past and future events.

The fields stop_sequence and stop_id are both set. The arrival and departure stop time events are populated when the vehicle enters a stop location (arrival) or exits a stop location (departure).

The schedule_relationship is always set to 0 (SCHEDULED) - the backend system does not process skipped stops.

Vehicle Position API

The Vehicle Position API provided real-time updates for all vehicles in the Auckland Transport network.

Vehicle Position

Real-time positioning information for a given vehicle.

The fields/structures trip, vehicle and position, as well as timestamp, are provided. All other fields, like stop_id, are omitted.

Trip Descriptor

A descriptor that identifies an instance of a GTFS trip, or all instances of a trip along a route.

Please see the Trip Descriptor section, part of the Trip Update API, above.

For trip descriptors in the context of the vehicle position messages, the field start_time is included.

Vehicle Descriptor

Identification information for the vehicle performing the trip.

Please see the Vehicle Descriptor section, part of the Trip Update API, above.

Position

The geographic position of a vehicle.

Both latitude and longitude, as per the GTFS Realtime specification, are included. The bearing is only provided if the AVL device on the bus provides it.

The odometer and speed fields are currently not provided.

Alert API

The Alert API provides notifications for incidents across Auckland Transport’s public transport network.

Alert

A notification of an incident in the public transit network.

All fields for the alert entity are populated - including optional fields. English is the only translation available for the url, header_text and description_text fields.

The active_period currently sets the start or end to “null”, and doesn’t omit them as per the GTFS Realtime specification. This may change in future.

The alert entity also provides two extension elements:

1. is_show_header: Indicates if the header_text should be displayed
2. level: This field indicates the severity of the incident - similar to log entries. The value can be either of “Info”, “Warning” or “Critical”.

These extensions are used internally and don’t form part of the API contract. While they can be used, care should be taken as future versions of the API may refactor or remove them.

Locations API

• Date: August 15, 2016

The purpose of the Locations API is to publish geographical locations relevant to Auckland Transport.

Format

All API responses contain geo-location information in a standard format:

{
   "id": "...",
   ...
   "latitude": ...,
   "longitude": ...
}

Depending on the type of location, further information is provided.

Customer Service Centres

The customer service centre API, pretty much as the name suggests, lists available Auckland Transport service centres across Auckland, with contact details, opening hours, if HOP services are provided, etc.

Parking

The parking API lists all Auckland Transport managed (i.e. not the ones operated by Wilson Parking and others) parking buildings and spaces, including facilities for mobility parking.

Note the response of this API call is rather large. Please contact us if you are after something particular, like a search function.

Points of Interest

This endpoint is out of order at the moment, because of changes to Auckland Transport’s GIS system. Please contact us if you would like to use it.

The points of interest API lists well-known public places.

Scheduled Works

The scheduled works API, again as the name suggests, lists all scheduled works across the Auckland Transport managed road network (which btw excludes the motorways; these are NZTA-managed). The API returns additional geographical details, like if a particular area is affected, as well as planned start and end dates, contractor details, etc.

Note the response of this API call is rather large (the contractors are busy). Please contact us if you are after something particular, like a search function.

GTFS API

• Date: August 15, 2016

The purpose of the GTFS API is to make public transport routes and schedules available. At the moment, the API supports routes, trips, stops, calendars and agencies (see below for further details). Fares and transfers are not provided (yet?).

Most of the public transport data available follow the model of Google’s Transit Feed Specification (GTFS). A good place to start is Google Transit Reference - this will tell you a lot about the domain, for example, routes, trips, or calendars.

Naming Convention

The API uses the GTFS convention - snake case - as its naming convention. You may not like the underscores, but this is the easiest if you want to re-use their documentation. Please don’t ask for camel/pascal/random case.

Versions & Identifiers

The transport network is regularly changing: routes are changed, stops are created, removed or relocated, times change, etc. The data “underneath” therefore is versioned. GTFS allows for changes - which is what calendars are designed to do. However, Auckland Transport’s internal planning systems are a little more restrictive, which forces us to add version indicators to many identifiers.

For example, the service 334 has a route “Otahuhu To Britomart Via Mangere Town Centre” with the ID “33401-20160803113210_v43.25”, but also a route with the ID “33401-20160729161628_v43.23”. These are the same route, but different versions for different date ranges, and potentially different stop sequences. See below for further details around routes and trips.

And, like most identifier, don’t try to read anything into them. Sure, an ID like “71302-20160803113210_v43.25” looks like bus 713, and version 43.25 created on the August 3^rd - but that might change, so please treat it as a “string thingy”.

The following sections are about GTFS - in a nutshell -, with a few Auckland specific details worth noting:

Agency

An Agency is an operator of a public transit network, often a public authority.

For Auckland Transport, these are the different bus, train, and ferry operators, like Sea Link (ferry), Sky Bus (bus to airport), Ritchies (bus), or AT Metro (trains).

Route

GTFS Routes are equivalent to “Lines” in public transportation systems.

A route, in Auckland terms, is, for example, bus 334 (route_short_name), “Otahuhu To Britomart Via Mangere Town Centre” (route_long_name). The record itself is mostly for display purposes - the trip (see below) ties it to the schedule.

But: Usually GTFS defines one route (for example 334), and several trips reference it to identify inbound and outbound directions, different schedules for weekdays and weekends, etc. At Auckland Transport, because of internal processing, this works a bit differently. Multiple route records usually exist for one “service” (334), one for each direction, and sometimes also for partial routes and different stop sequences. So, compared to how GTFS should be, there are routes that should be one route, and routes that have the same service number, but should perhaps have different numbers.

For example, the service 334, as mentioned above, is eight routes, all with the same route_short_name (334):

1. Otahuhu To Britomart Via Mangere Town Centre (33401)
2. Britomart To Otahuhu Via Mangere Town Centre (33402)
3. Otahuhu To Britomart Via Massey Rd (33403)
4. Britomart To Otahuhu Via Massey Rd (33404)
5. Otahuhu To Onehunga Via Mangere Town Centre (33405)
6. Onehunga To Otahuhu Via Mangere Town Centre (33406)
7. Otahuhu To Onehunga Via Massey Rd (33407)
8. Onehunga To Otahuhu Via Massey Rd (33408)

1&2, in this example, illustrate different directions, 1&3 different stop sequences, and 1&5 illustrate partial routes.

On this topic, it is also worth pointing again to the Versions & Identifiers section above, which describes multiple versions of the same route at the same time.

Trip

A Trip represents a journey taken by a vehicle through Stops.

By example, a trip is bus 334, “Otahuhu To Britomart Via Mangere Town Centre”, leaving “Platform 1a Otahuhu” at 14:00:00, and arriving at “71 Customs St East” at 15:25:00, Monday to Friday.

From a model perspective, the trip ties together the route (Bus 334, “Otahuhu To Britomart Via Mangere Town Centre”), a series of stop times (14:00:00 - Stop 8098; [63 other stops]; 15:25:00 - Stop 7021) - these can then be resolved to the stops (8098=”Platform 1a Otahuhu”; 7021=”71 Customs St East”) - and a calendar (Monday-Friday).

The identifiers for trips - just like routes - are versioned. See section Versions & Identifiers above for details.

Stop Time

A StopTime defines when a vehicle arrives at a location, how long it stays there, and when it departs.

This one should be obvious: It defines the scheduled arrival and departure time at each stop for a trip.

Stop

A stop is a location where vehicles stop to pick up or drop off passengers.

This one should also be fairly obvious: A bus stop, train station or ferry pier.

Most bus stops have “parent” stops, which usually indicates a bus stop on each side of the road. For example, stop 11522 (“Mountain Rd and Morrow”) is the parent for 8600 and 8601. It is worth noting here that the child stops can have different geographical locations, and also names. For example, 8600 is called “Gillies Ave near Morrow St”, and 8601 is “5 Gillies Ave”. Some bus stops are also the “single child” of a parent stop. Though, don’t rely on all bus stops having a parent.

Train stations, at the moment, are single stops without parents. The reason for that is that platforms in stations are currently allocated dynamically; linking platform stops to trips would mean the GTFS constantly needs to be regenerated. So, from a GTFS point of view, all trains passing through a station pass by a single stop.

Note that the value for stop_id happens to be the same as for stop_code. Don’t bank on it, and don’t try to use the ID to look up stops by code.

Calendar

Services define a range of dates between which a Trip is available, the days of the week when it is available (such as Monday through Friday)

GTFS separates calendars out to allow re-use between trips. At Auckland Transport, due to how the schedules are generated, a trip and a calendar form a 1:1 relation - one calendar for each trip. This relationship may change in future, though, so please don’t base your code on this assumption.

The identifiers for calendars - just like routes - are versioned. See section Versions & Identifiers above for details.

Calendar Dates/Calendar Exceptions

You can use this to define exceptional days when the Trip is operated, as well as when it is not operated.

Nothing particular to note. It is defined, though, and must be used to determine if a particular service runs for a given date.

Shapes

Shapes describe the physical path that a vehicle takes

A call to shapes, for a given trip, returns a sequence of points (geographical locations).

The attribute shape_dist_traveled is not set.

Frequencies

Frequencies are not used at the moment. Auckland’s Link buses, to which this could apply, run on a schedule like all other buses, trains and ferries.

Transfers

Transfers are not used at the moment.

Fare Rules/Attributes

Not in use, at the moment. Until recently, the fares structure for Auckland was too difficult to be accurately reflected in the GTFS fares structures. With the recent introduction of zone-based fares, this may now be possible. Please contact us if this would be useful for your project.

Please also see the Farecalculator API to get fares for trips.

Feed Info

The file contains information about the feed itself, rather than the services that the feed describes.

This API is probably less of interest if you consume the API directly, but is useful if you want to download the GTFS feed directly: It contains the version number, as well as start and end dates for that version.

Note that a new version can be published before the stated end date. However, the data up to that end date will (should) stay the same.

Please refer to AT Google Transit Feed to download the GTFS feed directly.

Coming Soon: Display API

• Date: August 15, 2016

Coming soon.

AT is not going to change?

• Date: August 15, 2016

We are just getting started - through ATLabs, we envision ideas to emerge from all sides, either under the umbrella of Auckland Transport or as new businesses, to advance the future of Auckland

It’s one of those favourite past time activities of Aucklanders: Complaining about the inefficiencies of Auckland Council, and, in this context, Auckland Transport (yes, we know, we live here too). Everybody, when asked, has their points of contention, things they would do better/different, if only they were in charge.

While there are ways to engage, in the past these were only for the most determined of us, and the result often didn’t look anything like the proposal. Too many cooks, as the saying goes.

With the new ATLabs, we are aiming to improve that - starting with the digital space.

This is just the start

Auckland Transport’s digital channels, admittedly, are rather basic: A website covering information about what AT does, its projects, congestion & parking, cycle ways, manage and top up your HOP card, some tools to plan your journey on public transport. And a couple of mobile apps (Metro app, Track My Bus) that are trying to make your public transport experience easier while you are on the go.

We realise this is just the start.

A lot more can be done. How often do you think: “If only we had …”? With ATLabs, we’d like to help you make these ideas become a reality, by connecting people with ideas to people who can make these ideas come alive - and with people who have some expertise in this: us. We start with what most ideas require: data. We can provide data about roads, traffic, schedules, geographical positions, routes. Let us know if you need anything else that we might have. And even if we don’t have the data: Ask anyway, and we’ll try to get it.

Auckland Transport is changing

Auckland Transport’s “mission” is to provide the transport infrastructure to make Auckland the most liveable city. Since its inception, AT has been working on this, somewhat in isolation. Without trying to make excuses, this has been because tools and processes to integrate ideas and feedback have not been available, or usable. Coming up with, and implementing new ideas has always been a lengthy, and expensive process.

APIs, one of the cornerstones of ATLabs, are changing this, together with advances in technology. Digital products are always a combination of infrastructure, logic and data. Finally, infrastructure has become flexible and inexpensive enough to just “try”. APIs can provide the data required to power ideas - so we are really only missing the logic/ideas.

Auckland Transport, with ATLabs, is changing to make the most out of these opportunities.

Though, it’s not only Auckland Transport that is changing …

Transport is changing

We firmly believe that massive change is around the corner: The likes of Uber/Lyft are just the start. These companies have already shown us that, through smart optimisation, the cost of taxi/limousine service can be brought into an area where car ownership, for some people, is a more expensive option.

Some companies are already experimenting with ride sharing, or smaller buses and dynamic routes, which should be even cheaper - and blurs the line between rather rigid public transport and the flexibility of owning a car.

And, of course, self-driving cars are appearing on the horizon, which, in my mind, is going to bring the biggest disruption of all. From a significantly reduced need to own a car, while still maintaining much of the flexibility, to not needing central car parks (but perhaps better pick-up zones), to dynamic ride sharing - and these are just changes for the user. The insurance industry, petrol stations, and roads themselves are all going to change, as driving becomes safer, cheaper, and more efficient.

Though, rather than leaving all the excitement, and rewards, to our friends in Silicon Valley: Why not start here, in New Zealand?

What next?

By sharing Auckland Transport’s data and systems, we’d like to make innovation easier, and for new transport ideas and initiatives to develop inside and, especially, outside of AT. We envision ideas to emerge from all sides, either under the umbrella of Auckland Transport or as new businesses - which in turn can change how Auckland Transport goes about doing its part for the future of Auckland.

Let us know your thoughts to improve Auckland Transport - or how we can support your business ideas. We could really use your help.

Welcome to ATLabs

• Date: August 11, 2016

Auckland Transport is looking after Auckland’s transport network - roads & cycleways, traffic, parking, public transport. Until recently, though, it “did its thing” in a bit of an isolated way. This is where the ATLabs comes in: we’d like to open up the digital side of transport in Auckland.

Why are we doing this?

Honestly, we could do with some help - from Aucklanders, who want to get around their city easier, quicker, and safer. We believe that by inviting people to contribute we get better ideas, and faster execution - both which benefits all Aucklanders.

We like the idea of experiments: Better than committing to a year-long software development project; we’d like to develop an idea, test it against the market (Auckland) quickly (in weeks, not months or years), and continuously improve it. And this is something a community of software developers and ideas people excels at.

Why contribute?

We know there are plenty of ideas “out there”, plenty of people who would like to leave their mark on their city - and a whole lot of software developers who want to do more than customise reporting software, integrate financial transaction systems, or churn out one website after another.

ATLabs is not for profit; it is to make you get to where you want to go. You aren’t making somebody else rich - you are improving life in your city, and, who knows, perhaps even the life of people in cities around the world.

How to contribute

For now, we decided to make this about people with ideas for software applications, and software developers (and a special welcome to developers with ideas).

Idea contributions

Do you have an idea that will revolutionise transport? Or a seemingly small improvement that will make our lives so much better? Great, we want to hear them all - this is what our ideas forum is about. We are here to connect the idea with people with expertise in this area, and people who can implement it. If the idea is a good one, then there should be no problem finding many helping hands - and we’ll chip in too, wherever we can.

We understand that your idea is important to you - especially if it is an idea for a new product or service. We want to keep you engaged throughout the implementation cycle, and credit you for your idea. However, we also want to make your idea the best it can be, so we take the opinions of our experts strongly into consideration. This collaboration means your idea might not turn out exactly as you envisioned, but hopefully better.

Please note that this is about ideas for software applications. While we all would like to get that pothole fixed, and an in-bus relaxation massage service, for now, we are looking to make Auckland better through software. We aren’t saying that your idea can’t have a physical component to it. The most interesting ideas are often a combination of hardware and software.

Code contributions

At heart, we are coders too. We understand that this strange combination of working out software challenges, perhaps while learning a new language or technology, and seeing something coming out that people use, and like to use, is very fulfilling. This is why we go to work, and to bed way too late at night.

Do you have an idea, but don’t know how to go about it? Leave us a message in the ideas forum, and we’ll be in contact. And if you are looking for ideas: Join us at the same forum, there ought to be something for you.

Useful next steps are to look in the getting started section, and have a look through the Auckland Transport API to see what data is available. We put some sample code up, so you can get some insights on how to use the API. If you have a question, need some help - or perhaps missing data, let us know through our gitter chat.

We haven’t (yet) worked out exactly how getting your code shipped will work. Generally speaking, we’d like the code to stay your code. Just create some project repository with your repo provider of choice (we like github) - or fork our ones. If you need help with your code, continuous integration, testing, infrastructure, or getting it into an app store, get in contact with us, and we should be able to help you out.

We are looking to work out a better process for this - but would like to take your inputs on how you think it should work. Stay tuned.