COMMENTARY

Peter Burgess

WHAT IS NESTORIA?

Welcome to the Nestoria blog. Nestoria is a property search engine.

We offer the Nestoria property search experience in Australia, Brazil, France, Germany, India, Italy, Spain, and the UK.

Why (and how) we've switched away from Google Maps

Fellow Nestoria fans,

this week we went live with a significant change to our service - in most countries we've moved away from Google maps and are now relying exclusively on OpenStreetMap maps served by MapQuest.

Before I dive into the why (and the how for all our neogeo mapping freaks amongst our readership), let me say that Google maps remains a phenomenal service that is continually adding amazingly innovative new functionalities. The boom in online cartography witnessed over the last years was kicked off by the launch of Google maps, and I can still remember the light bulb going on in my head when in early 2005 I saw housingmaps.com, the first Google maps mashup. It was clear I was looking at the future. A little over a year later, in June 2006, we launched Nestoria. So I am the first to recognize the unequaled contribution Google has made and continues to make in unlocking the potential of cartography for the world (and technology in general).

In 2006 I had another 'ah-ha' moment as well, though. I met Steve Coast, the founder of OpenStreetMap. He explained the idea - a free and editable map of the world made by user contribution. A map that would not just allow me to get the final rendered output, but also the actual data underneath. At the time, looking at Steve as he showed me the GPS device he was using to map the details of our meeting (which took place in a little cafe in Soho across the street from the John Snow pub), the idea of creating a viable map via volunteer submissions seemed preposterous in the extreme. But I watched as OSM grew and grew from those humble beginnings. Now, less than six years later, that map powers Nestoria thanks to millions of man hours of contribution by individuals and organizations that recognize the power of open data.

So, why have we switched? There are four main reasons

1. The maps are equal or better

OpenStreetMap's great strength is that anyone can contibute. Since the project started over 500,000 people around the world have signed up to do just that, often going into insane levels of detail. Fixes can be added and reflected in the maps very quickly. It is a fundamentally different model than the traditional 'only an expert from the government can come make the map' model. People can map whichever features are important to them (paths, pubs, buildings, etc) and escape the car centric focus of many mapping services. All of this data is then made freely available for all to use. Increasingly government agencies are realising that it makes more sense to cooperate with and benefit from this new approach to data gathering and maintenance.

Thanks to the hard work of all of these volunteers, in many places of the world, particularly the European countries we were focused on, OSM maps are of equal or better quality than any other widely available mapping service.

2. It's another visible way for us to support open data

Our service does nothing more (and nothing less!) than aggregate data from many different sources and present it in an easy to use format. We benefit greatly from open data, and as such we want to do our part (within the limited resources of a start-up) to help the open data movement. This is why we sponsor OpenStreetMap conferences and recently donated to OpenStreetMap's humanitarian efforts. This is why we feature the work of open data advocates on our blog, and also why we make our own data available via our API and other tools. We are a company that believes in open data.

3. Google introduced charging for map usage

Earlier this year Google announced that they would begin introducing limits to the use of Google maps by commercial websites. The good news is that Nestoria has grown nicely since our start in 2006. The bad news was that our size meant that we were well over the free usage limits Google announced.

In November I was contacted by a sales person from the Google Enterprise team. I had suspected we might be over the limit. Obviously no one looks forward to a new cost for their business, but I approached the talk with an open mind. Google Maps is a great service, and we had benefited greatly from it. As a businessperson, I know there's no such thing as a free lunch, and so I was open to paying Google a reasonable fee for their continued service.

Unfortunately Google's sales process was not good. Having agreed to a time for a call, the sales rep missed the appointment with no warning, instead calling me 45 minutes late. It was quickly obvious he had done no research whatsoever about our service, what we do, or even where (in which countries) we do it. He was unable to explain the basics of the new charging regime - for example, what exactly is a 'map-view', telling me instead to 'ask your developers'. Finally he quoted a price to continue using Google Maps (just on nestoria.co.uk, one of eight countries we operate in) that would have bankrupted our company.

Google's excellence in technical services was definitely not matched by its salesmanship. The experience was disappointing, and I say this as the founder of a site that has often been featured by Google in promotional literature for our innovative use of Google maps. Having always envisioned that we would someday move to OSM, this was the nudge that pushed us over the cliff.

In the interest of fairness I should also mention that Google has been a great supporter of OSM in the past, donating funds for hardware for instance, and hopefully they will continue to be far into the future. Google remains a great supporter of opensource software with initiatives like their excellent Summer of Code initiative.

We're very thankful to Google for all the innovation they encourage and for allowing us to use their maps service for free for years. The decision to introduce charging is theirs to make and we can't argue with it. Despite this though, I have to question some of the logic they presented regarding their reasons for introducing charging now. Google claims charging is needed to ensure the long term commercial viability of the service, but is belt tightening really needed at the same time as Google announces record revenues and profits?

More importantly though I wonder if the decision really achieves the desired outcome. While us moving away from Google Maps will reduce some fractional amount of bandwidth costs for Google it also means our team of engineers will be spending our time working with, and innovating on, other geo technologies. While on the one hand Google spends a lot of effort trying to court developers, decisions like this turn them away. Especially combined with the subpar sales implementation I experienced, this seems to go completely against the ecosystem model that has enabled Google Maps to flourish, which is disappointing.

4. The tools are ready.

Despite all of this, we would not have been technically able to make the switch unless there was a solid set of tools and services around OSM that made the switch possible. I'll go into these in more detail in the technical part of this post, but let me here once again publicly thank all the developers around the world who have worked hard over the last few years to create the modern neogeo tool chain from scratch. Also let me explicitly thank the companies like AOL's Mapquest and Microsoft's Bing who are actively supporting OpenStreetMap.

Before I dive into the technical minutiae, let me say that all of this isn't to imply that OSM maps are perfect. No map ever is. But it all added up to a compelling sense that now was the time to switch. Nevertheless if you see problems, please tell us (via twitter or the feedback link on the page you are using). Or, better, please get involved with OSM and start contributing.

Now, in the interest of encouraging innovation (and thanking those who made this move possible) let's move on to the technical fun of HOW we actually switched map providers?

When we realized it was time for us to make the move we faced one big decision - should we use someone else's OSM tiles or should we render and serve our own? We called in an expert to advise us. OSM expert, and former Nestoria blog interviewee, Andy Allan runs OpenCycleMap, a rendering of OSM data designed specifically with the interests of cyclists in mind. He was kind enough to come to Nestoria HQ and spend some time taking us through the pros and cons of rendering our own tiles. Rendering has the advantage that you can make the map look exactly the way you want. When done well this can produce phenomenal results, a good example of this is Michal Migurski's recently announced terrain layer, but unfortunately it's no small technical undertaking, especially when we've also got a property search engine to run.

We concluded the only viable path was for us to leave the rendering and serving to experts and use someone else's OSM tileset. At this point the more astute of you may be asking why we dont just use the tiles from openstreetmap.org directly. That's unfortunately not an option due to OSM's tile usage policy. As a volunteer run organization, OSM doesn't have the technical or financial resources to serve tiles for us and the whole world. Luckily however several companies have stepped in to fill this gap - CloudMade has for several years offered an OSM tile layer for all to use. In 2010 MapQuest released a similar service. While we are longtime fans of CloudMade (we use their tiles on our Where Can I Live? service), for their global infrastrucutre and speed we decided we'd prefer to use MapQuest's OSM tiles. But now the question was how to get the OSM tiles on to our pages.

First of all, if your website is using a map I strongly advise you to consider using Mapstraction, which, as the name implies, is a javascript mapping abstraction layer. You write your code using Mapstraction methods and can then switch between anyone of 10 or more supported mapping services. Even if you plan to stay with one mapping provider this can make sense as they create new versions (as Google did several years ago when they released version 3 of their service requiring a different syntax than version 2). The good news is we've been using Mapstraction since the very beginning of Nestoria, in fact we funded the initial development - this was the topic of my 2006 meeting with Steve Coast and others. Five years later Mapstraction continues to flourish, with an active community of developers. At this point in the project (mid-Nov) Mapstraction offered two different services for loading OSM based tiles: CloudMade and OpenLayers. Cloudmade would mean using Cloudmade tiles. OpenLayers was an option, but not a technology we had much experience with. Meanwhile over the last few months I had been hearing a lot of buzz abouta new mapping library called Leaflet. Leaflet is also from the folks in the CloudMade team, but is newer (thus benefiting from all the lessons learned in building the original CloudMade map library) and is opensource. Leaflet allows the user to request any tileset, configuring it to query MapQuest was trivial. Everyone can contribute and in the six months or so since Leaflet launched almost 150 developers have forked the code. Lots of people are submitting patches, the pace of development is rapid, and the documentation is excellent.

Unfortunately, there was no Mapstraction plugin for Leaflet. And this is where the magic of an engaged and vibrant open source community once again enters our tale. The very day I pondered whether to dive in an write a Leaflet plugin, Ben Welsh submitted exactly what we needed to github for all to use (and modify). It worked almost perfectly. A few minor tweaks (and submitted patches) later we had an OSM map on Nestoria. Many thanks Ben!

So, there you have it - OSM to MapQuest to Leaflet to Mapstraction. If anyone out there can top that neogeo chain I'll be impressed. Please let us know what you think or if you have any questions.

BTW - if all this has raised your interest in all things geo (and you're in London), please do come to the next #geomob event on 16 February where I'll be giving a talk about our move away from Google Maps.

UPDATE: in the comments below paulmaunders points to this very good post where he lays out the options his firm, Fubra, considered under similar circumstances.

Posted 4 days ago by Ed Freyfogle

25 comments

Dec 26, 2011 phph guy said... Yeah its sad they started charging. Its the real difference between a for-profit corproation and some kind of foundation. You cant really blame them. The do no evil is marketing pr.

Dec 26, 2011 wodim said... 'Finally he quoted a price to continue using Google Maps (just on nestoria.co.uk) that would have bankrupted our company.' So you should be honest and brave and just say: 'we left Google Maps because we can't afford it, now we use an alternative which is worse but that, at least, doesn't force us to close our start-up.'

Dec 26, 2011 paulmaunders said... Hey Ed, We chose a similar route when Google wrote to us in November with the same proposition. We've gone for a slightly shorter chain using MapQuest OSM tiles directly with Leaflet - and we've blogged about it here: http://www.fubra.com/blog/2011/11/24/google-maps-free-alternatives/ It's interesting to see we came up with pretty much the same conclusion :) Paul

Dec 26, 2011 Ben Welsh said... You can find the Mapstraction fork with Leaflet support here. It is current a pull request with the trunk. https://github.com/mapstraction/mxn/pull/101

Dec 26, 2011 Ed Freyfogle said... Hi wodim, thanks for commenting, but I have to disagree on two points a. OSM isn't worse. In many places it's better. b. Our start-up is 6 years old and nicely profitable. But I'm not ashamed to admit that doesn't mean we can pay the same rates as massive corporate multinationals, which was what was suggested. enjoy the holidays, Ed

Dec 26, 2011 Gabe da Silveira said... @wodim - What is dishonest about what he said? Why should he frame things the way you want them to be framed? Dec 26, 2011 Ed Freyfogle said... @paulmaunders great post, thanks for making me aware. Will tweet about it. Hope all is well otherwise, let me know if you're ever up in London and want to meet up.

Dec 26, 2011 gregorymarler said... It's not every day my blog breaks news before the public announcement. :) I've linked to this post from my blog. http://www.livingwithdragons.com/2011/12/christmas-presents-for-mappers My reaction to Google Maps charging didn't bother me much. Understandably they should make some money from the bigger users, but the sales rep story you gave is just shock after shock. If I got that treatment from them I'd be glad I couldn't afford their charges.

Dec 26, 2011 Michael said... don't be so apologetic towards Google. they have shown over and over again they do not care about their customers. to them it's all about algorithms and efficiency. the cost of Google Maps was merely a calculated one, if you can't afford it, okay bye.

Dec 26, 2011 Sheridan S. said... Thanks for the insight and all the best to your company in 2012. As a developer with over 20 years experience, I've also noticed the lack of customer service with many of Google's SAAS & other products for years now. Unfortunately, this seems to happen to many large companies and as the first comment in the list indicated, the 'do no evil' is just marketing PR. It would be awesome if you or maybe Paul Maunders over at fubra could write up a how-to for some of us using PHP, Python, JS? Never dipped into geo stuff but would find it interesting to learn the toolchain you all described from those who have succeeded at enterprise-scale.

Dec 26, 2011 A Nony Mouse said... In Google's defense, it is quite common to lower an initial quote from a company by a large amount. 'We will charge X per month' 'Sorry that would bankrupt us' 'Hmm, how about X/20 instead' These conversations happen every single day.

Dec 26, 2011 sam said... I just got on mapquest, more ads per square inch than I've ever seen before. Hideous site in my opinion.

Dec 26, 2011 Shoaib Burq said... Now to iron out OSM routing and bring it up to speed with Google Maps

Dec 26, 2011 Tomas Cot said... I checked the places near where I live and I found out that OSM is much more accurated and have a lots of little town that Google doesn't have. There are a lot of naming error in Maps that OSM got right.

Dec 26, 2011 smick said... So how does one use open street map to create or get directions? I'm trying to grok this thing as a non-cartographer. It's a very nice and responsive map, but I'm torn on how I can use it to embed. I do like I can export maps to vector and the like.

Dec 26, 2011 wyclif said... There is a minor typo under '1. The maps are equal or better': 'Since the project started over 500,000 people waround the world hase signed up to do just that, often going into insane levels of detail.' Around, not 'waround.' Thanks for the great article. Land surveyor here.

Dec 26, 2011 wyclif said... Also, have not 'hase.' Heads up.

Dec 26, 2011 Ed Freyfogle said... @wyclif thanks for telling me. Corrected. Glad you enjoyed the article. Would be interesting to learn a land surveyor's perspective on OSM.

Dec 26, 2011 niket said... I think it's awesome that you laid out why you made the switch and was gracious to your previous provider. Google has not been a company most people interface with outside of pure Ad Sales, so I'm sure that is getting ramped up from the poor iteration you encountered. It's good to know competitors exist so overall mapping technology and abilities will remain top notch and being pushed forward.

Dec 27, 2011 Jan Debonnet liked this post.

Dec 27, 2011 Anonymous said... We had a similar call from Google, and got a similar quote that was about 10x what we would have gladly paid. Their pricing model, for those wondering, starts at $10 per 1000 page views. But no one pays that, of course. If you don't get a contract, then above the first free 25,000 views, you pay $4 per 1000. And we got them to quote us to about $2, maybe $1.80. In our case, that meant paying $200,000-$400,000 a year for maps. Versus spending two weeks implementing our own maps, based on OSM data, with our own custom tiles, and tilemill. Sure, we'll have to pay for bandwidth for the tiles, and spend more time making sure it looks good. But it was worth it, both in savings, and now having more control of our own maps. Google got their pricing off by at least an order of magnitude. I don't know many companies making more than a couple of dollars per thousand views. Had they charged us $0.20 per 1,000, we would have gladly paid.

Dec 27, 2011 Anthony Ferla liked this post.

Dec 27, 2011 rantsfromron said... As a casual user of Google Maps mainly within the UK, I've always found it to be inaccurate and unpredictable (i.e. errors are inconsistent). Today, looking up a location, I was surprised to see that GM had renamed the building I was looking for, and erroneously applied the name of my target to a location several miles to the south. Had I been a stranger looking up that location, I would have been seriously misled.

Dec 28, 2011 nickw said... wodim: Research OSM a bit and you will find that in Europe certainly, you will find that OSM is certainly not 'worse'. Pick a typical European city, compare Google Maps and OSM.

Dec 29, 2011 Terry Stigers said... A few points - I'm a little unclear on how charging for a service Google pours resources into constitutes 'doing evil'. I think businesses like this one are actually in the minority. If a business is generating traffic that amounts to more than 25,000 map views per day, it seems pretty reasonable for Google to assume said business is able to generate some form of income and therefore can pay for such a solid mapping service. And chances are, the majority of them can. There is a fundamental difference between OSM and Google Maps that most people seem to be missing. OSM is a data provider. Google is a map provider. To the average user, this difference may seem insignificant, but to those who make maps the difference is enormous. When I look at OSM, I see a map and I have access to all the data used to draw it. Google Maps shows me the data THEY have chosen to show me. Call me crazy, but my idea of important features on a landscape may differ from those of the developers at Google. Lastly, only someone who knows nothing about maps, GIS and/or data would claim that Google Maps are 'better' or more accurate than OSM. This simply isn't the case. OSM data is consistently more accurate and more detailed than Google's data (I've written about this on a few occasions). Most importantly, OSM data has a feature no other data out there possesses: it is self-correcting. The value of this cannot be overstated.

BLOGROLL Cloud Made data.gov.uk Blog Datablog Digital Urban fuck yeah cartography! GeoNames Google Geo Developers Guardian DigitalMedia Homes24:blog housingdabble Humanitarian OSM Team Ito World Kelsey Group Londonist Map Scripting Mashable mySociety dev blog nearby.org.uk Nubricks OpenGeoData PaidContent ParkatmyHouse Perl Buzz Programmable Web Property Owl Property Portal Watch Read/WriteWeb Strange Maps TechCrunch UK The Negotiator The Rat and Mouse