Hello, I'm Alan

A little ride in the country

Thursday, October 5 2017

One of the things I enjoy doing away from the computer is riding my bike, and exploring the countryside. A while back I was perusing my cycling club’s list of routes and was frustrated that it only had start/finish locations, which didn’t lead to the easy discoverability of routes. Luckily, the published list of routes mainly used services such as RideWithGPS, which has the ability to export a GPX route file (intended for GPS navigation), which lead me to the question of… how handy would a map be?

One weekend I sat down, wrote a scraper to scrape the club’s site, and extract out the name of the route, and the link to the site it was hosted on. I downloaded the GPX files and generated a JSON file which had the needed metadata about the ride, as well as the filename of the GPX route. For a while, I’d been seeing that GeoJSON seems to be the best format for open geographical information exchange on the internet, so I converted the downloaded GPX to GeoJSON.

Taking the metadata and GeoJSON that was generated, I used leaflet.js to display the routes on an interactive map, using map tiles from OpenStreetMap. This had a working prototype, but as the average GeoJSON file in use was about 150kb and there are around 400 routes, that left a web page that needed to download about 60mb of data to operate, which was a little excessive for an approximate map, not a detailed track which would be used for navigation while on the ride.

Researching some more I learnt about another format called TopoJSON. Simply TopoJSON uses a number of techniques to simplify a route without losing detail. Just with this change, it reduced an average route down to around 40kb which is a decent improvement. It’s still beefy, but right with caching it becomes OK on the average Australian internet connection which is my target audience.

I initially wrote this for myself but shared it with the club and a few months later I got an email asking why it wasn’t working, turns out some other members were also getting utility out of having it available. Around this time I got asked if I wanted to host it on the club’s website, but the problem was it was a dodgy ruby script written by me which I knew how to finesse into working. After some discussion, it turned out there was a project underway that would end up putting these routes into a database that could be made available from an API, which would mean I could remove all of the scraping code.

Given that the only library that converts GeoJSON to TopoJSON was written in Javascript, it seemed if I was to implement this correctlyâ„¢ then writing it in Javascript seemed like the way to go. Up until now I’d written client side JS, but never really had to deal with asynchronous processing, that needed to wait for that async process to finish before continuing. I used the opportunity to learn about how promises work, and wrote a fetcher that would fetch the routes, and then convert the GPX to GeoJSON, and then on to TopoJSON.

After the reimplementation from Ruby to Javascript it cleaned up a lot of the crud I’d written quite well, but the last remaining thing I don’t like is I still think it downloads too much data for the job it’s doing. The thought I’ve had so far is packing all the routes into a TopoJSON file and see how well it takes advantage of the shared sections of rides.

Also, the code is here https://github.com/aussiegeek/audax-tracks