When I first started studying geography I was geekily enthralled with all the different ways you could slice and dice data based on location. As we’ve worked on adding analytical capabilities to GeoCommons and GeoIQ it has been fun re-imagining how they would apply to the new world of real time and mobile/social data. Since it is the Christmas season we thought it would be apropos to introduce a new analytics tool for each of the 12 days of Christmas.

So, the question is what analytic is going to be the Partridge in the Pear Tree? It was a tough call but we had to go with aggregation. What is so special about aggregation? It gives you the ability to see distinct patterns in massive data sets. What happens when you put tens of thousands of points on a map – in the words of Schuyler “red dot fever”.

red_dot_fever

Above we have all the tweets from mobile devices on black friday as red dots. The fever is rampant to the point of not being able to see the map. What we can discern is quite limited from this perspective, but if we were to count the number of tweets in each state that would tell us something meaningful.

This is what the “aggregation” tool in GeoCommons does. It allows you to count and run statistics on the points that intersect a set of polygons – like USA states. In computational geometry this is often referred to as the “point in polygon” problem. When we link up the ability to create thematic maps from the tabulated statistics we can quickly create pattern maps. Which state, county, or zip code had the most tweets on Black Friday.

black friday tweets by state

Black Friday Tweets Aggregated to USA States

black friday tweets by county

Black Friday Tweets Aggregated by USA Counties

black friday tweets by zip code

Black Friday Tweets Aggregated by USA Zip Codes

So, how easy and fast is it to go from 15,000 tweets to a thematic map? Answer – a couple of clicks and about two minutes to go through the work flow:

Aggregation of Black Friday Tweets to States from FortiusOne on Vimeo.

The best part is my entire analysis work flow has been captured and metadata about it automatically generated:

data lineage black friday aggregation

No more finding files and trying to guess what the heck was done to create an analysis and what data sources they used. It is all automatically captured for you. Your entire data lineage generated with no extra work and you can control who can access it and tweak your analyses. The future of making analysis collaborative and social.

 

9 Responses to The 12 Analytics of Christmas

  1. sruizcon says:

    We are a research group on geographic education i secondary education (12-18) in Catalunya, NE Spain. We are recommendind your Geocommons to our colleagues in our web http://blocs.xtec.cat/georecursos. We are very grateful to your efforts sharing and improving cartography tools.

  2. Sean Gorman says:

    Hi -

    Thanks for the compliment and glad to hear the site looks useful. If we can be helpful with the class just let us know. Several universities have been using it and you can see an example tutorial here:

    http://multimedia.journalism.berkeley.edu/tutorials/build-interactive-census-map-geocommons/

    Appreciate you reading the blogging and using the site!

  3. [...] the recursive nature of “12 days of [Analytics]” I incorporated our new aggregation functionality to summarize the number of twitter messages and followers to US State. Doing this [...]

  4. [...] going to predict within datasets. Previously on the 12 Day of Analytics we learned about merge and aggregation. Both of these functions serve as building blocks to other analysis. Today’s functionality is [...]

  5. [...] with additional datasets if my region of responsibility spans outside of Virginia, or it could be aggregated up to Virginia boundaries if I wish to analyze at a regional rather than point [...]

  6. [...] in GeoCommons – This is one to watch. They are debuting a new function each day on their blog. FortiusOne builds their platform API-first, UI-second so everything they are showing should be [...]

  7. [...] Andrei on Twitter – @andreitr – and check out his personal blog at geocomrade.com Carrying on with The 12 Analytics of Christmas, let’s take a look at one of the simplest yet most powerful analytics feature – Buffer. In [...]

  8. [...] the power of spatial aggregation I can in fact evenly compare two datasets. We can aggregate the individual point locations to the [...]

  9. [...] data, and today we’ve added geographic analysis of data.  Over Christmas we blogged about the “12 Analytics of Christmas” we’d built for GeoIQ.  We are celebrating [...]