Census 2010 Take10 Map for Google Earth

March 22, 2010 in Blog Leave a reply

Census LogoAbout the Census Take10 Campaign
In March 2010, more than 130 million addresses will receive a 2010 Census form by mail or hand delivery. The 2010 Census will document the changes in our nation since the last decennial census in 2000, and tell us how we’ve evolved as a country. Because census data affect how more than $400 billion in federal funding is distributed to tribal, state and local governments, the census also will frame the future of our country and our communities for the next 10 years.

During Census 2000, the mail participation rate was 72 percent as of the April 2000 cut-off. About $85 million is saved for every one percent increase in mail participation. For these and many other reasons, we encourage everyone to participate in the 2010 Census.

Visit the official 2010 Census “Take10″ campaign website.

About the Take10 Google Earth Map
The Take 10 Layer for Google Earth displays the same daily response data as the Google Maps-based Take 10 Map that lives on the official 2010 Census website. As most major national and dozens of local news television outlets already use Google Earth for visually appealing and highly flexible geographic information display, this layer helps the media tell the ongoing story of the decennial census count.

The layer is made available as a small downloadable KML file, which opens up in Google Earth and downloads the latest data direct from the server. Response rates are indicated per geographical region according to a color scale, with a legend at top-left. The view begins at the country level showing states, and upon zooming in, the view changes to counties, places, and finally tracts. It is recommended that moves to be made live on air be “rehearsed” as it can take a few seconds for data in a given zoom level to load in the first time, but once it’s loaded it flows very smoothly. Clicking on any geographical object brings up a bubble that shows the name of the region and a meter chart comparing the 2010 response rate to that in 2000. (Note that viewing this layer requires a copy of Google Earth.)

Google and Innovation Landscape are also making this layer available to media and the public for embedding in any website as a gadget. Go to http://maplify.com/census to preview the gadget and make custom selections, such as the default view and zoom level for the gadget to load at — a newspaper, local government, or other public-facing locally-focused website will want to specify the region of interest to grab the audience’s attention. As with the downloadable layer, the map is clickable to allow deeper exploration of the data. At the bottom of the gadget is a link back to the gadget preview page so anyone can learn how to embed it themselves. (Note that viewing the embedded version requires Google Earth Plugin, which is free, cross-platform, and installs very quickly.)

To learn more about the technology behind the Census Take10 Map for Google Earth and Innovation Landscape’s role in its design, read on …

Building the Take10 Map for Google Earth
While simple in presentation, the data behind the Take10 map is exceptionally rich and required careful assembly and processing to ensure a smooth, coherent display to the end user. This project leveraged deep Innovation Landscape experience in spatial data processing and used many different tools and API’s.  The development process required obtaining the necessary source data, reducing and reorganizing the source data for processing and visualization, programmatic KML generation for visual quality and performance, and finally tying everything together into a simple map “gadget” for display and syndication.

Obtaining Raw Source Data
Spatial boundary data for the Take10 map is derived from Census TigerLine files which are publicly available at the US Census TigerLine website. Our map focuses on States, Counties, “Places” such as cities, towns, and villages, and tracts which are defined for census survey purposes. Each of these entities includes a boundary, centroid, name, and identifying number. These files are in ESRI shapefile format. Participation rate data is provided at the Take10 official website as a downloadable text file in a fixed delimited format and is updated daily during the active mail-in campaign.

Data Processing
postgis logoThe Take10 map contains over 130,000 individual geometries across states, counties, places, and tracts. Many of these geometries are quite complex in raw form. Further, tracts are only provided on a state-by-state basis in individual files. Our 1st goal then, was to simplify each geometry for suitable display in Google Earth while striking a balance between display quality and geometry size for drawing/download speed. We also needed to combine all the tract files into a single dataset. We turned to the open source postgis plugin for the postgreSQL database both tasks.

OSgeo LogoPostgis has a built-in command line ‘loader’ for shapefile import so getting the TigerLine data into the database was simple. Postgis also has a built-in ’simplify’ function for geometries with an adjustable parameter for the degree of smoothing. Different smoothing values were required at each layer level. To generate our smoothed output from postgis, we used the OGR2OGR library which is bundled in the OSGeo Open Source GIS toolset. OGR2OGR had two helpful features – the ability to source data from a postgis query and the ability to output query results directly to KML using KML’s extended data schema to preserve attribute information.

To facilitate automatic map updating on a daily basis, we generated a mySQL database on Innovation Landscape’s “Maplify.com” geo-lab website to warehouse both the static geometries for our map and the individual participation rate values for each entity. To keep things simple and allow for dynamic prototyping, we reduced all geometries to a common attribute set including Census FIPS number (essentially the unique key for each), the name, the type (state, county, place, tract), centroid in spatial point format (for ease of query), and a large text field for the KML mutlipolygon definition for each boundary (since we didn’t plan to query on boundaries – only on centroids). PHP scripts were written to initialize the database with the full set of layer geometries and initial year 2000 Census participation rates. When done, we had a database with over 130,000 rows ready for KML generation.

KML Generation
KML is generated daily by 1st downloading the Census participation rate file and processing this via PHP to update the mySQL geometry database. Once complete, our script builds out static KML files with updated values. We use many aspects of the KML specification to enhance both the visual quality of our map and the performance. First, the data is divided by layer and regions. The reduced state and county data is sufficiently small to keep in single files, but the place and tract data requires sub-division. Places are divided into 5 degree x 5 degree grids with tracts divided into 1 degree x 1 degree grids. Output KML files are ZIP compressed into KMZ files to reduce bandwidth requirements. Individual files are joined together into our seamless presentation via ‘region’ coded KML network links. With regionation, Google Earth dynamically loads and draws data as required by each user’s changing view of the map. In setting up our KML, we specify the relative ’switchover’ points between layers via region attributes in our network link files.

Google Chart APIFill colors and pop-up content is controlled via KML Styles. KML Styles apply a consistent look and feel across our data and reduce duplication in the files for faster load times. We used the Google Chart API to create our ‘meter’ icons in each balloon.

The Map “Gadget”
Finally, Google wanted to provide an easy way for the public and media to view and share our map. For this, we turned to the Google Earth API & Plugin. The Google Earth Plugin is a cross OS and browser application for running Google Earth within a web page. The Google Earth API provides a set of JavaScript functions to customize and control the map. Via the API, we added the ability to search and zoom to a given location, generate custom links for the map, and create an ‘embed code’ similar to embedding a YouTube video via simple user interfaces.

What’s Next?
Following the Take10 Campaign, we plan to provide both our raw and simplified state, county, place, and tract KML files for free download to 3rd party developers. We’re already looking at additional datasets and new features in the future.

About Innovation Landscape
Innovation Landscape is a Delray Beach, FL based innovator in geo-spatial programming services for businesses, government agencies and non-profit organizations. We are a Google Qualified Maps Developer and have an extensive portfolio of map design projects. For more information about our services, please explore our website or contact us directly by phone or e-mail.

Leave a Reply

Gravatars are enabled. Register for free!