Ever since the news broke that ESRI recieved a Whitehouse contract to:

merge a federal website that publishes geospatial information with Data.gov, the government’s depot for downloadable data sets, the company’s president said on Thursday.

California-based ESRI began last summer tying Data.gov to Geodata.gov, the geospatial information gateway, said company President Jack Dangermond in an interview with Nextgov….

Anyone will be able to create mashups on the free website ArcGIS.com, which ESRI launched on Saturday.

the community has been pretty upset. I’ve seen the same angry sentiment repeated over Twitter, IRC, Skype and beers.

This made me begin to wonder why were folks – myself included getting so riled up over this. ArcGIS.com looks like a great site that makes geospatial data more accessible to the public. Data.gov is making more government data available – marriage made in heaven.

So, what is the problem? At the core the deal and the platform violate the two guiding principles of Gov 2.0. – transparency and open data. When the Obama administration announced their “Transparency and Open Government” initiative there was a hugely positive response from the community.

The deal outlined by the NextGov article violates the first premise – transparency. The ESRI deal was a sole sourced contract that was not competed, but instead an extension to a six year old contract for GeoData.gov from 2004. What was the the process for selecting ESRI for this critical service? What were the requirements? What are the benefits of their solution? Are open standards being supported? No clue on any of it because there has been zero transparency on the contract.

No only was the process not transparent ESRI gets access to government data for ArcGIS.com that is not being made available to anyone else. Worse yet ESRI had access and was working on integration for a year according to the article. No other citizen, project, NGO or company had access to the data to integrate into other existing projects. Keep in mind the current incarnation of Geodata.gov is a data catalog of metadata not a data repository. So, there is no easy way for anyone to systematically pull all the geodata cataloged in Geodata.gov.

A second critical aspect of open government is open data. Open data means the raw data available in open standard formats. So, ESRI got a sole sourced contract and a head start integrating the data into their new Web portal. Some might find it a bit sketchy, but big deal we can get over it. At least the data will be available to the public to repurpose and innovate on top of…right?

Sadly this does not seem to be the case at all. To quote the articles:

He (Dangermond) said he expects Geodata.gov’s map services, which enable Web-based applications from different sources to communicate with each other, to be available on Data.gov within two months.

This is an important nuance. Data won’t be available – “map services” will be available. So what is a “map service” you might ask? A “map service” is a proprietary data stream from an ArcGIS server. A service that can only be produced by ESRI technology. You can overlay the “mapping service” on say a Google or Bing basemap, but there is no way to repurpose the data or open it unless you have ESRI technology. Building anything with the data requires reliance on ESRI technology. So, where will citizens be able to consume this data – ArcGIS.com of course. What does ArcGIS.com support? Let’s look at the screenshots:

arcgis_upload

ArcGIS.com supports ArcGIS server “mapping services”, ESRI mapping applications, and ESRI mobile applications. All proprietary and none of which can be utilized by anything other than an ESRI product. The “mapping services” coming out of the Geodata.gov work – all the same. No standards and no support for anything other than ESRI licensed software – yeah we asked.

Ah, but you say you can load files, and maybe even download files. At least there would be support for shapefiles, and other apps can read those….nope:

arcgis_shapefile_support

If the data could be open by another technology obviously that is bad. In short this is what has the community quite upset about the whole affair. A non-competitive bid with zero transparency to put government data in proprietary formats that are not accessible to the community, unless you want to buy ESRI technology or look at their proprietary site.

The good news? This is all easy to fix. Make the data available in open formats. Make it available in a raw format for download. Federate the content through open standards so other projects can tap into the data feeds (not map services). Make contracts open opportunities that are competitively bid with clear requirements.

ESRI has lots of great technology but citizens should not be forced to use it in order to access public government data. This is the fundamental principle behind open data and transparent government. ESRI is an important part of the ecosystem, but it is not a monoculture. If the government wants innovation to thrive and sustained economic growth they need to truly open data and create a level playing field.

 

15 Responses to Why ESRI's GeoData.gov – Data.gov Contract Gets the Community Upset

  1. Learon Dalby says:

    This is the second blog post I have read on this issue. The first was James Fee’s http://bit.ly/ceThaT which I failed to comment on (my bad James). Sometimes if it doesn’t feel right there is reason why. Each of the blog post have clearly demonstrated why this ‘deal’ doesn’t feel right.

    No doubt ESRI is a great company. Many of us use their products, but this deal doesn’t make us feel any better about using them. Open is open and I believe that means the raw data can be accessed via any number of protocols. Yes, being open is a challenge. It might even require doing the same things a number of different ways, but it also feels right.

  2. Eskimo says:

    Maybe you will be able to download shapefiles. A Layer Package (lpk) can be just a zip of a shapefile (which is itself an awkward group of files) plus some symbology which could be ignored.

    I can see the worry that this will just be another webstacle – image maps or keyhole sets without access to the full source data – but this may be the data owner’s wish, not the software vendor’s preference.

    • undisclosed says:

      Shape files are a horrid format that should go away. Sure, it served a purpose at one time, but that time has come and gone. And the company that wrought this proprietary format upon the world, E.S.R.I. will not support it.

  3. Indeed, see my latest blogspot on same …
    http://bit.ly/ayAwo2

  4. David Sonnen says:

    It seems like we should know what “open” means by now. This scenario (“open” label, opaque process) has been repeated dozens of times with the same and, by now predictable, results.

    Need a lesson? Try searching for cloud+ultimate+lock-in and read the first 10-500 results.

    Personally, I am a huge fan of Data.gov, Geodata.gov and ESRI. I believe that each group does want to make a positive difference. But, open data means the data is open to any use, not just through mapping services or “free”.

    Making data really open is harder than providing a convenient way to access the data through a single platform. But, if the Feds want their data to be open, they need to make the effort needed.

    Otherwise, the Feds need to say “convenient” — not “open”.

    Users really do know the difference.

  5. The Lone Geographer says:

    Wow, it’s not even a very good platform. It’s not scalable, open, or extensible.

    What a shame.

  6. Sean Gorman says:

    Thanks for all the good points. I hesitated for a while about writing this post because I did not want it to come across as a case of sour grapes. I think that ESRI did a nice job with ArcGIS.com. It is a huge improvement over ArcGISonline.com and you have to give them points for seeing innovation in the market place and responding to it.

    Competition is good in my mind. It demonstrates there is real opportunity in the space. Geo is no longer niche and all these are great indicators for everyone in the space. It is exciting seeing companies like SimpleGeo and SoGeo getting significant funding from notable VC’s.

    While this is all great. If we are not vigilant about keeping data open we kill the real opportunity in the space for everyone. There may be some short term gains in pushing proprietary formats, but it keeps GIS as an IT niche, and prevents us from being a part of a much bigger market and opportunity.

    It is also not just about business. As we’ve seen in Haiti, Chile, Afghanistan and countless other disasters – open and portable data saves lives. No one has times trying to find a proprietary tool and the appropriate license when responding to a disaster. When there is no Internet – “mapping services” don’t work. This is why access to raw data is so critical. It makes data portable, remixable and reusable in any situation.

    Hopefully our government and others truly embraces the concept and these contracts and approaches to government data are speed bumps as we evolve to concept.

  7. Marten Hogeweg siad it best in his blog
    http://martenhogeweg.blogspot.com/2010/06/building-your-own-arcgiscom-client.html
    “ArcGIS.com provides a great collection of resources and, as Jack explains below, allows other people to discover the work ESRI users are doing.”
    emphasis on “ESRI users”, this is not meant to be open to start with…

  8. Sean Gorman says:

    Well at least we can all agree that it is not open. I have no beef with ArcGIS.com. It is ESRI’s tool – they spend the money and can use it how ever they like.

    My beef is when a Federal contract is created to make open government data delivered there not accessible in open formats. Especially when that same privilege is not made available to the myriad sites that do make government data available in open formats.

    ESRI has explicitly tied ArcGIS.com into their GeoData.gov and Data.gov contract as a delivery mechanism to citizens. It is not “ESRI user” data. It is government data that my and many others tax payer dollars paid for. I don’t think many tax payers would be happy about their money going to lining ArcGIS.com with content.

    You can’t have both a closed portal just for your users and use it as mechanism to deliver open government data to citizens. The two are mutually exclusive by any reasonable public policy test.

  9. Exactly – someone in the US GAO has lost the plot, and/or forgot to check all the marks in their to-do list. List which our industry took pains to draw up on behalf of the citizens. And they/we are the paymasters not the guv… Other contract extensions in US Air Force Academy and Forestry Service, I think, followed that same path, shall we call it of least resistance?

  10. charles says:

    This closed no-bid contract process is not just bad data policy, but a continuation of the same old corrupt practices. ESRI is getting payback for political favors or contributions, and the public is suffering the consequences.

  11. [...] Gorman at Off the Map hit the nail on the head when summarizing the community’s sentiment on the ESRI GeoData.gov contract. If you haven’t been following this, in a nutshell ESRI has a no-bid contract with the Feds [...]

  12. [...] slated to become.  Why?  Many claim the platform was not as open as it could be, in some cases requiring proprietary software to make use of the data.  While such a public-private partnership can sometimes be efficient, a [...]

  13. pletcherfjn says:

    [url=http://clouboutinsale.webs.com]christian louboutin on sale[/url]
    As all of us know, NIKE, lots of nike shox shoes requires a giant market, NIKE TN has swept the globe Light Up Air Force 1. Spirit of the brand is fundamentally a personalized to represent the spirit of enterprise [url=http://clouboutinsale.webs.com]christian louboutin sale[/url], which is the brand or brand decision-makers in the long-term nike shox tl1 production & management in the reason for the gradual formation of beliefs, values, or business purposes.
    “Nike won’t sell products, & only sell Nike brand, then look forward to the back of sales up with the brand.” Nike believes, “on behalf of the people the brand is very dynamic. They are not conservative, willing to innovate nike shox nz for women pink/white, to seize the chance, but also willing to name the brand, accept the people’s love, criticism or respect. “spirit of the brand promotion, Nike has made such a immense success, now in the hearts of lots of young people have become Nike a kind of character, a symbol of the persistent pursuit of self-confidence.
    Relate Links:
    [url=http://louboutindiscount.moonfruit.com]discount christian louboutin[/url]
    [url=http://cheaplouboutin.moonfruit.com]cheap christian louboutin shoes[/url]
    [url=http://christianlouboutins.moonfruit.com]christian louboutins sale[/url]
    men’s nike shox r4 shoes white/sky blueSpirit of the brand by the different business areas of content, methods, history, custom & reality of the constraints has the pursuit of diversity [url=http://clouboutinoutlet.moonfruit.com]christian louboutin outlet[/url], is the brand in the nike shox r3 shoes production & business activities have gradually formed the idea personalized color statements Light Up Air Jordan Shoes [url=http://christianlouboutins.moonfruit.com]christian louboutins[/url]. Spirit of the brand Nike TN REQUIN in various forms, but as well as a variety of communication channels, to strengthen the staff’s brand awareness & brand promotion have an important role.
    The international shoe giant Nike is promotion the brand to focus on the spirit of the master Glow In The Dark Galaxy Shoes [url=http://louboutindiscount.moonfruit.com]christian louboutin discount[/url]. “Sports, performances, free & simple, free the spirit of the athletes,” Let Nike sought after by lots of consumers.
    Spirit of the brand in the nike shox nz sl market competition has not estimated the power of the development of enterprises plays an important role Glow In The Dark Adidas Shoes [url=http://cheaplouboutin.moonfruit.com]cheap christian louboutin[/url]. View from the shoe industry, increasing competition among shoe brands focus on the influence of the spirit.
    [url=http://clouboutinoutlet.moonfruit.com]christian louboutin outlet online[/url]

  14. We recommend avoiding a silk filled duvet, which demands hand-washing. It’s well known males will likely not bathe their underclothing physically louis vuitton stephen sprouse so we sure will not be doing the work for the children.