Endo2Atom – An Endomondo ATOM/RSS feed to post your activities via tools like IFTTT #OpenData

I was very disappointed to realise that Endomondo has no API. The best they can do for pushing your activities elsewhere is built-in Facebook/Twitter and some widgets. When will companies learn that lock-in is not how you keep customers?

I’d like to be able to push my activities to sites like App.net, Tumblr or anywhere really. The best way to do that is with ATOM/RSS and great tools like IFTTT and dlvr.it.

A bit of poking around last night led me to good old screen-scraping. Their widgets are publicly accessible so it wasn’t too hard to scrape one of them and generate an ATOM feed. I used it as an opportunity to try out Bottle. If you need to build a simple web-app fast, with minimal overhead and you use Python, I think you’ll like it. For kicks and giggles I decided to put it up on Heroku – my first time to do so properly. That’s a very nice deployment flow indeed.

marathon

So here you go. If you want the ATOM feed of your public Endomondo activities, just grab your numeric user ID from your profile page and tack it onto http://endo2atom.conoroneill.com

For example, my Endomondo profile is http://www.endomondo.com/profile/8922951 so my ATOM feed is http://endo2atom.conoroneill.com/8922951

Code, as ever, is on GitHub.

I have also created a simple IFTTT Recipe for auto-posting those activities to App.Net. I’ll try some other targets over the weekend.

It has only been tested on treadmill running, treadmill walking and road running. There is no error checking so it may fall over on non-distance-based activities. Let me know if it does and I’ll improve. The code is also limited to the first page of activities and doesn’t crawl backwards on the other pages. That should be easy enough to add.

Hope you find it useful.

A simple tool to bulk import your RunKeeper data into Endomondo

TL;DR – Download the Windows RunKeeper to Endomondo tool.

I’ve been using RunKeeper as my main mobile running app since January of last year. I liked it so much, I moved to the paid “Elite” tier pretty quickly. For almost a year, I really couldn’t fault it and liked the social interactions with other Irish users along with its rock solid stability.

Then in December, it all went to hell. Whatever they changed in the Android App around the end of the year made a complete mess of the GPS tracking. Every run looked like I was a drunk Mr Incredible, staggering from one side of the road to the other in a single step.

I ran a bunch of tests and compared the data to other tracking apps like Google My Tracks. They confirmed that the problem wasn’t the phone/OS/location/weather/time/mood or even the moon, the problem was RunKeeper. Even worse, my wife’s RK on iPhone was showing similar rubbish data on the same route! Given the terrible mobile signal in Old Chapel, is it possible that RK’s inability to live upload location actually trashes the location data itself?

After bitching and moaning for a few weeks and submitting a problem report, the RunKeeper people told me the all-new V3.0 should be better. And overall the accuracy was much improved, but now the altitude data is rubbish. And for some bizarre reason, they have removed functionality like the Map view in Android.

I expressed my annoyance in a status update on RK and was interested to find two other Irish runners who are equally annoyed by the problems of the past few months and RK’s seeming lack of urgency with fixing the core functionality of their product. And we are all paid users, not freeloaders.

I decided to check out alternatives and quickly landed on Endomondo. Initial impressions are that the mobile app is nice but very laggy on a HTC Sensation, the web-app is dog-ugly but very functional and there are a _lot_ more of my Facebook friends on it than on RunKeeper. It also seems to be more of a generic activity app and I like the challenges they set.

But then I ran into a problem. I’m not moving Apps unless I can keep my running history. I keep a record of all my activity on a Google Doc spreadsheet but that obviously doesn’t have the GPS data. RunKeeper have an excellent API and also provide a full dump of all your GPS traces as a zipped GPX file. The problem is Endomondo, it can only import one GPX file at a time. And life is too short for that nonsense.

So I sat down and decided to create a simple importer which takes all of your unzipped GPX files and concatenates them into one large GPX file which Endomondo can handle. Once imported into Endomondo, the activities all appear separately as you’d expect.

I did my usual programming-by-Google-Search and found some simple generic Python GPX concatenation code (on GitHub, natch) which I had to modify to work with RK data. Of course I realised that the average runner is not going to be able to use a Python script with installation dependencies so I used Py2Exe to build an executable. Then I realised that most people nowadays won’t be comfortable with a command line app and I had a quick poke around PySide. A bit of messing with some example code and I had a nice GUI. Thennnnn I found PyInstaller which gives me a single windows EXE without needing an installer etc.

And so finally I get to the point. A simple Windows App which you point to a directory containing all your unzipped GPX files from RunKeeper and which generates a single endomondo.gpx file which you can import into Endomondo.

Converting from RunKeeper to Endomondo

  1. You export from RunKeeper as follows:
    1.  Click on your name in the top right of the main screen
    2. Select My Settings
    3. Scroll down and you’ll see a small blue link on the bottom left to “Export Data”
    4. Click that and follow the instructions
  2. Download runkeeper2endomondogui.exe
  3. This is a Windows Executable. You don’t know me from Adam. Please run a virus checker on it after you download.
  4. run runkeeper2endomondogui.exe
  5. Use the File menu to select the directory with all of your RK GPX files
  6. Currently the import only does those GPX files. So it is missing all of my manually entered treadmill runs (and there are a lot of those).
  7. My code is very rough and may show “not responding” as it grinds through larger files. It tells you what it is doing in the main window so just be a bit patient, particularly if you have an older PC.
  8. When it is done, you’ll have a new endomondo.gpx file in the same directory as the exisiting files
  9. I’ve read that Endomodo craps-out on files over 10MB, so if the endomondo.gpx file is bigger than that, you’ll have to do this process in batches with sub-sets of the RK files. As a guide, my 47 GPX files including several half marathons and a full marathon come in at 4MB.
  10. You import into Endomondo as follows:
    1. Once my tool is done, go to Endomondo and select “New Workout”
    2. Select the “Import from File” option
    3. Point it to the endomondo.gpx file
    4. Endomondo will appear to hang when you do the import. Leave it running and then open another browser tab and check your activity history. When it is fully populated with the imported activities, you can then safely close the original browser tab.
    5. That’s it, you’re done

The code has a few path dependencies on Windows but it should be trivial to change it to work on OS X and Linux.

All of the source code is, as always, up on GitHub, for you to change as you see fit

Questions or Problems or Expressions of Horror at the awfulness of the code? Leave a comment here. Not on Facebook or Twitter.

Open Data Ireland’s first meetup tonight in Engine Yard

Great to see this Meetup happen and big props to Eamon and the crew for hosting. If you are interested in getting involved in OpenData in Ireland please attend. Registration is here but the event is free.

Ireland needs to get beyond talking about OpenData and start increasing the number of people actually Opening Up The Data in a big way. The Irish tech community has to lead by example and that means just doing it.

(Apologies to Code For America for the visual knock-off)

There is so much data out there that anyone with even a modicum of technical ability could unleash. The most complicated thing I had to do in geo-coding the Property Price Register was to merge two columns in Excel!

I’m hopefully (the Eircom DSL gods permitting) going to give a short talk remotely from Bandon, long known as “The Silicon Valley on the N71″ :-) I’ve a few bits n bobs to describe, some thoughts to share and some ideas I hope others can run with.

Ideally we’ll do the whole thing as a Google+ Live Hangout so anyone can connect in. Sign up for the Hangout here. I’ll post the Live URL if we have success with it. If that falls apart then I’m afraid it’ll just be point-to-point Skype.

 

From tiny CSV acorns, many #OpenData oaks can grow. Now where is Ireland’s CIO/CTO?

This has been a superb week in Ireland for showing what motivated geeks can do with even the most clunky of Government data. After the Oireachtas XML debacle, it was a joy to see the new Residential Property Price Register site make its data available. Sure, it was poxy old CSV files per year and it was riddled with errors but I really am pleased with what it triggered.

Within hours of the data going up, we had:

  1. Properties by Price Range
  2. A merged Excel spreadsheet
  3. YellowSchedule with a great search interface
  4. A Heroku-based App with Search
  5. My import into Google Fusion Tables and the (patchy) Geo-Coding of the data

And by the end of the week we also had:

  1. Another searchable site with maps
  2. Mapquest-based Geo-Coding
  3. Others?

In fact the only thing we seem to be missing is someone sticking a JSON API in the front of the data. UPDATE: See comment below from Brendan. Nice one!

I hope those working on data in the Public Sector take encouragement and ideas from this. Most of what was done took very little work. I’m sure many of the people involved would be only too happy to share their technical approaches.

If we all keep leading by example, I expect the quality and quantity of Public Sector Web Services and Data to improve in leaps and bounds over the next 12 months.

Three years ago next week, I suggested a regular BarCamp called  IRLCamp where public sector and private sector get together and present interesting projects, ideas and technologies and learn from each other. I still think we should do this.

Three years later we still need a CIO/CTO for Ireland too.

I’m speaking at the OFFSITE Conference next week about AWS, APIs, SaaS etc

I mentioned the OFFSITE Conference over on Web2Ireland recently. It’s the frst time a major technology event is being held in West Cork and everyone in business in Munster and beyond should consider attending. It’s all happening next week, 27th September in Rosscarbery.

The title of my talk is “Building on The Shoulders of Giants: How Irish businesses can quickly deliver Enterprise-quality products through the smart use of software services

The bulk of the talk will be about all the lessons we have learned over the past few years in running the LouderVoice service on a cloud infrastructure and how you could do similar.

Some of the topics I’ll be covering include:

  • Building a SaaS business
  • Amazon Web Services (EC2, EBS, EIP, RDS, S3, CloudFront, CloudWatch)
  • IaaS vs PaaS (AWS vs Google vs Cloud Foundry vs Azure etc)
  • Tying it all together with APIs
  • Scaling
  • Pricing and Cost Monitoring
  • Robustness Cost-Benefit
  • Pitfalls
  • What’s Next?
  • Big Data Opportunities
  • Open Data Opportunities

This won’t be a geek-out session, it will be a business-centric talk where I hope those who are looking at the cloud get a lot of useful tips and cautions.

Let me know if there is anything in particular you would like to hear about.

 

My mind is blown – Met Eireann’s radar image converted to #OpenData

Brendan Cunningham has just left the most exciting comment this blog has ever received. He has thrown some serious smarts at the Met Eireann radar image of rain and converted in to actual numbers you can use!

That vector data with rainfall values is now up on Google Fusion Tables and is updated every 15 minutes so now it can be integrated with GIS and other location based services.

As he points out, with national coverage like this, you can watch for rainfall trends. Heck, you could even do predictions. He mentions that previous flood data may be available from the OPW  and a “Benefitting Lands” layer from the OSi.

I pointed out to Brendan that this data may be perfect for Open Weather Map which has a radar layer for the US and Canada already. Hopefully there are no rights issues with the data (which I assume the Irish taxpayer has paid for). His follow-up comment seems to indicate that all is ok as long as it’s not for profit.

It’s projects like this that give me hope for Ireland. Where the State is unwilling, unable or simply lacking in knowledge to release #OpenData, we can step in. Motivated, knowledgeable people with a community spirit can make the data we have already paid for, do more for the country.

I’m working on a small project in this area at the moment and have just received all of the hardware. Once it’s ready, I’ll publish the PoC here. I hope it will be of benefit to a lot of people.