Planning for Big Data

The buzzword/hot concept for the year is Big Data.  Internet did it.  Business does it.  Government is getting ready to do it.  The IT world is aflutter with the idea.

What is interesting is how little of the concept trickles down to the minions in the field.  What in the world is this big data stuff?

The basic concepts circle around volume, velocity, and variety.  There is a lot of digital information.  It is moving quickly.  And it comes from multiple dissimilar sources.

So what?  This is stuff for Amazon to use in deciding what book to recommend or DARPA to use in tracking terrorists on Twitter.  As a planner, I'm stuck with writing a new staff report, and that DOES NOT move quickly.

But in this talk, physicist Kyle Cranmer breaks down how the LHC is using big data in big science.  He suggests that hypotheses are no different than business models or policies.  We can use big data to confront all sorts of ideas.

And we do have big data.  Zoning ordinances, all of them, from all across the country.  Written journal articles.  The entirety of land records.  GIS data.  All those staff reports.  Sure, it's not 140 characters long.  It will take a different kind of architecture to deal with these.

Think what would happen if you had a nationwide set of all single-family detached zones.  You could appreciate the differences in terminology from across the country.  It would show the duration and durability of the zone based on the number of amendments.  Aberrations where one city allows, who knows - tanneries - in their zone would stand out.

But part of Big Dat is not just having that set.  It is making that set ready to connect to others.  Combine that Single Family Zone analysis with earlier versions of the ordinances and look how the terminology evolved.  Combine it with the staff reports to see how these have been applied, or variations in how they are interpreted.

Combine the whole set with land records to see if amendments to the zoning were reflected in the layout of lots.  Go back to the policy decisions to know if the policy in one location was more effective than in another. Combine it with sales data to see if this is the development that people want.

Big Data is not about numbers.  It is about layers.  It is a concept that planners are intimately familiar with, drawing information from novel sources to develop new concepts.  But it is going to be at a whole new scale.