April 2nd, 2009

Posted in Developer, Google Earth API | No Comments »

May 27th, 2009

We’re at Google I/O in San Francisco this week.  I/O is a great venue to learn about Google applications directly from the coders who right them while mixing with some of the best 3rd party developers in the world.

This marks our 3rd year at I/O and we’re looking forward to the next 2 days of sessions.

CEO Eric Schmidt opened the keynote address.  His theme was ‘It’s Time’ in terms of platforms and opportunities to solve real user problems in simple ways rather than complex ones.

Vic Gundotra followed with a talk themed ‘A more Powerful Web, Made Easier’.  Browsers have attained a 100x implovement in Javascript parsing in the last 10 years.  Nearly 500 million people are using open source browsers.  HTML 5 should offer a significant enhancement to developer options.

• <canvas> tag for in browser drawing giving pixel level control of the browser
• Matt Papakitos demo of O3D (open sourced on code.google.com ) Googles Javascript 3D inbrowser demo.  Stressed need for common 3D browser API’s and standardization for 3D
• HTML5 <video> tag to abstract playback and control of video
• Accurate geolocation moving forward via various techonologies.  Jay Sullivan (VP Mozilla) presented developer info for Firefox 3.5 (bit.ly/ff35dev) including canvas, video, geolocation, app cache + database, and web workers.  Google Maps demo of ‘My Location’ using the geolocation API to transmitt user position to Google to zoom/pan Google Maps to the user location.
• Lattitude on iPhone coming directly in the browser.  Nextgen iPhone software for OS 3.0 will include native Safari support for W3C geolocation
• Example of Gmail on Adriod in offline mode.  Google’s 1st major HTML5 app to use app cache
• Michael Abbott (Senior VP, Palm) on WebOS which primarily uses HMTL, CSS, javascript to implement applications.  Exposing Accelerometer data into JavaScript API
• Web workers defines API for background scripts.  Motion detection in video via Javascript with annotation via a background process demo.  Very impressive given typical computational complexity of image and video processing

Andrew Bowers and Kevin Gibbs talked about app engine.  Coming up- backing processing, large object store, database export, XMPP, incoming email.  App Engine is now open for Java application signups.  Demo showed using Eclipse with the App Engine plugin to quickly develop and deploy Java Apps to the web.  GWT 2.0 will provide in-browser debugging support cross-browser also a runAsync() option for functions which will allow the compliler to split optimize javascript code for faster download.

DeWitee Clinto introduced Google Web Elements for simple ways to embed Google API’s (google.com/webelements) in simple ways such as Maps, Site Search, News, Calendar, Conversation.

Roman Guy previewed ‘Donut’ the next version of Andriod.   Search UI will remember how search results where ‘used’ on each device.  Developers will be able to supply search terms in code for display in the Search UI.  A text-to-speach API will allow developers to generate voice commands in various languages and accents.  The underlying text to speech engine will be open source.  Andriod UI will support ‘finger writing’ with handwriting recognition on the screen to filter results or jump through large lists.

Andriod Devleper Challenge 2 at code.google.com/android/adc

All I/O attendees will receive an unlocked Android phone with unlimited 30 day SIM card for data and voice!

• The new Tour KML tags for GE 5.0
• Use of update tags during Tours to create animations during tours
• Using the GE API to playback tours
• Limited HTML 5 is operative within GE 5.0 including video and audio tags on the Mac (via webkit 4.5 on PC)
• Based on an audience question, JavaScript support for JS within KML description windows will be resolved soon

SketchUp API – Mano showed some neat physics simulations which use the SketchUp API.

Pamela Fox presented on Google Maps APIs.

• Travelling Salesman in Google Maps www.geweb.net/optimap/
• StaticMaps API does support directions
• Google Maps API can be used for Desktop Applications (Section 7.1c in the Terms of Service)
• Reverse Geocoding, you can get full jSON on reserve lookups for large areas and the bounding box will be returned as the recommended viewport
• ClientLocation API to use IP to rough guess user location
• Monetization: GoogleBar and Maps Ad Unit
• Recoloring tiles in Flash (mush be approved by Google)
• 3D Flash Map with GE API like rotation control in Flash 9 and 10 (coming soon)
• Google Maps “Qualified Developer” programhttp://code.google.com/qualify

• Gmaps API v3 using an MVC programming model and optimized for fast loading on mobile devices
• Open registration for Java Apps in App Engine
• Lots of HMTL5 evangelism
• A certified developer program for Gmaps

We were left with an expectation of more announcements in today’s keynote which is about to start.  We’ll blog them here as they come out.

1st up is Steven Canvin from LEGO Group (each chair this morning had 6 standard 8 ‘peg’ LEGO blocks).  He asks, “How many combinations can you create from 6 blocks?  (Answer: 951,103,765) He gave a short history of LEGO Mindstorm and LEGO’s use of adult enthusiasts to help feed the R&D effort.  Use of open-source SDK’s has helped drive Mindstorm product adoption.

Now Vic Gundotra (Google VP of Engineering) is back up.  New personal communication/collaboration tool in early form.  Will be open source.

Named Google Wave

Engineered by Lars Rasmussen and others.  Stephanie Hannon is the PM.  Developed in Sydney Australia.  I/O attendees will be given developer beta accounts.  The product will launch later this year.

More info at:

wave.google.com

code.google.com/apis/wave

www.wareprotocol.org

• Wave assumes a ‘convsersation’ which is a shared,hosted object which represents a discussion thread
• Blends e-mail and chat paradigms into a common interface where ‘threads’ can be updated in non-real time ala e-mail or realtime ala chat depending on the connectivity of the recipients
• Because the ‘thread’ is a hosted conversation recipients can make inline comments to previous replies without copying whole messages
• Adding new participants to a thread allows then to ‘playback’ the sequential messaging up to the existing conversation ’state’
• In real-time mode, users have the option to view others’ typing live
• Private replies are supported
• Drag and Drop attachments (photos demonstrated).  Has inline processing to thumbnail images and transmit to the server prior to uploading the full photos.  Requires Gears for this feature.
• Can copy attachments (such as photos) from one conversation (Wave) into another or new message
• Can link automated ‘agents’ to a wave to allow direct publishing via Wave API’s.  Links go both ways so responses can be sent via the blog but replied to via other interfaces
• Allows users to partipate in threads across multiple sites through a single interface by bringing active threads in the Wave Client – think Google Reader, but for discussion threads – so you can write, not just read
• Connections created via an Embed API
• Edits are carried through all instances of a Wave thread
• Can use the editing feature to provide a collaborative environment for discussions.  Change markup is provided to other users.  Author receives a change notification.
• Playback will provide powerful tools for filtered, reversion to prior versions, version labels, etc.
• SVN like ability to merge changes from multiple Waves into a single ‘production’ Wave
• Concurrent edits can be made with live preview by any number of users at the same time
• Multi language support, right to left languages, … supported
• Wave is built using Google Web Toolkit (GWT)
• Shared tagging, folders, subjects for Wave organization, can link to external Waves – interlinking similar to wikis

Wave Extensions:

• Context-based spell checker
• Automatic link detector from plain text
• Robots act on the server to modify Waves via the collaborative editing engine
• Add links into Wave directly from inline Google Search
• Spell check, linking engine, etc. to be available as external API’s for developers
• Various Gadgets were shown using client-side Wave extensions – a Soduku game, Chess game, Collaborative Map
• Robot App for making a Poll demonstrated
• Extension for interacting with Twitter via Waves
• Robot to do live translation of Wave messages – ‘Rosie’ with support for over 40 languages

Google is planning to open source most Wave code so that others could create ‘competing’ systems.   Multiple instances could cross-communicate via invitation.  Private replies within Wave server instances are not communicated outside of the originating server even in the case of ’shared’ Waves.

Care must be taken in mobile designs though due to:

• Screen size
• User interaction
• Speed

For example, iPhone Safari is 10x slower to parse JS than desktop with limited caching.  Networks also induce significant latencies which often dominate load times relative to total file sizes.  Improve performance by compiling/obfuscating code.

Current Gmaps API (v2) is poorly designed for mobile apps.  v3 means to address these concerns and enhance mobile support.

• Only 35K in size (vs 190Kb in current v2)
• Static map is built in
• Optimized for mobile devices
• ~9s to load js and tiles on v3 vs over 18s on v2
• Chrome and iPhone supported
• no domain keys required!
• Default UI construct so app can by automatically keep up with control updates
• New geocoding API

v3 is based on a Model-View-Controller architecture to allow fast loading of initial models and wait for on-demand loading of required views.  New features can be added as additional views/controllers with less impact to the overall code base.

This is the final geo session for Google I/O 2009.  Links for Roman’s presentation can be found HERE.

As Roman pointed out, this marks the 1 year anniversary of the Google Earth API.  New features over the last year include:

• Mac Support
• KML Tour Playback
• Underwater Terrain
• Mars mode
• Enterprise server connectivity
• Over 80 samples in the demo gallery

Roman introduced ‘My Earth’ a mashup of the new Google MyMaps Data API and the Google Earth API.

To support the app, he’s released new Earth API open source libraries:

1. geojs which provides common geo math and geometries
2. earth-api-utility-library (GEarthExtensions)

• creates KML objects via ‘JSONic’ methods
• new methods for feature by id, dom walk, and ‘clear’ features
• Animations/effects framework
• 1-line draggable placemarks and line strings
• Screen overlay ‘Buttons’ including a control set like MyMaps geometry editing
• Apache 2.0 License

Other Recent Developments G_SATELLITE_3D_MAP has been upgraded significantly:

• all KML features supported in ggeoXML will now transer to the Earth mode
• it appears that javascript based updates to ggeoXML now propagate to the Earth mode
• 3D objects within KML called by ggeoXML will now show in Earth mode even though don’t in the 2D modes
• switches to and from the Earth mode now include nicer animations in tilt and rotation

All in all a great session with some fun new tools!

This entry was posted on Thursday, May 28th, 2009 at 11:04 pm and is filed underDeveloper, Google Earth API. You can follow any responses to this entry through theRSS 2.0 feed. You can leave a response, or trackback from your own site.

During Census 2000, the mail participation rate was 72 percent as of the April 2000 cut-off. About $85 million is saved for every one percent increase in mail participation. For these and many other reasons, we encourage everyone to participate in the 2010 Census.

Visit the official 2010 Census “Take10″ campaign website.

About the Take10 Google Earth Map
The Take 10 Layer for Google Earth displays the same daily response data as the Google Maps-based Take 10 Map that lives on the official 2010 Census website. As most major national and dozens of local news television outlets already use Google Earth for visually appealing and highly flexible geographic information display, this layer helps the media tell the ongoing story of the decennial census count.

The layer is made available as a small downloadable KML file, which opens up in Google Earth and downloads the latest data direct from the server. Response rates are indicated per geographical region according to a color scale, with a legend at top-left. The view begins at the country level showing states, and upon zooming in, the view changes to counties, places, and finally tracts. It is recommended that moves to be made live on air be “rehearsed” as it can take a few seconds for data in a given zoom level to load in the first time, but once it’s loaded it flows very smoothly. Clicking on any geographical object brings up a bubble that shows the name of the region and a meter chart comparing the 2010 response rate to that in 2000. (Note that viewing this layer requires a copy of Google Earth.)

Google and Innovation Landscape are also making this layer available to media and the public for embedding in any website as a gadget. Go to http://maplify.com/census to preview the gadget and make custom selections, such as the default view and zoom level for the gadget to load at — a newspaper, local government, or other public-facing locally-focused website will want to specify the region of interest to grab the audience’s attention. As with the downloadable layer, the map is clickable to allow deeper exploration of the data. At the bottom of the gadget is a link back to the gadget preview page so anyone can learn how to embed it themselves. (Note that viewing the embedded version requires Google Earth Plugin, which is free, cross-platform, and installs very quickly.)

To learn more about the technology behind the Census Take10 Map for Google Earth and Innovation Landscape’s role in its design, read on …

Building the Take10 Map for Google Earth
While simple in presentation, the data behind the Take10 map is exceptionally rich and required careful assembly and processing to ensure a smooth, coherent display to the end user. This project leveraged deep Innovation Landscape experience in spatial data processing and used many different tools and API’s.  The development process required obtaining the necessary source data, reducing and reorganizing the source data for processing and visualization, programmatic KML generation for visual quality and performance, and finally tying everything together into a simple map “gadget” for display and syndication.

Obtaining Raw Source Data
Spatial boundary data for the Take10 map is derived from Census TigerLine files which are publicly available at the US Census TigerLine website. Our map focuses on States, Counties, “Places” such as cities, towns, and villages, and tracts which are defined for census survey purposes. Each of these entities includes a boundary, centroid, name, and identifying number. These files are in ESRI shapefile format. Participation rate data is provided at the Take10 official website as a downloadable text file in a fixed delimited format and is updated daily during the active mail-in campaign.

Data Processing
The Take10 map contains over 130,000 individual geometries across states, counties, places, and tracts. Many of these geometries are quite complex in raw form. Further, tracts are only provided on a state-by-state basis in individual files. Our 1st goal then, was to simplify each geometry for suitable display in Google Earth while striking a balance between display quality and geometry size for drawing/download speed. We also needed to combine all the tract files into a single dataset. We turned to the open source postgis plugin for the postgreSQL database both tasks.

Postgis has a built-in command line ‘loader’ for shapefile import so getting the TigerLine data into the database was simple. Postgis also has a built-in ’simplify’ function for geometries with an adjustable parameter for the degree of smoothing. Different smoothing values were required at each layer level. To generate our smoothed output from postgis, we used the OGR2OGR library which is bundled in the OSGeo Open Source GIS toolset. OGR2OGR had two helpful features – the ability to source data from a postgis query and the ability to output query results directly to KML using KML’s extended data schema to preserve attribute information.

To facilitate automatic map updating on a daily basis, we generated a mySQL database on Innovation Landscape’s “Maplify.com” geo-lab website to warehouse both the static geometries for our map and the individual participation rate values for each entity. To keep things simple and allow for dynamic prototyping, we reduced all geometries to a common attribute set including Census FIPS number (essentially the unique key for each), the name, the type (state, county, place, tract), centroid in spatial point format (for ease of query), and a large text field for the KML mutlipolygon definition for each boundary (since we didn’t plan to query on boundaries – only on centroids). PHP scripts were written to initialize the database with the full set of layer geometries and initial year 2000 Census participation rates. When done, we had a database with over 130,000 rows ready for KML generation.

KML Generation
KML is generated daily by 1st downloading the Census participation rate file and processing this via PHP to update the mySQL geometry database. Once complete, our script builds out static KML files with updated values. We use many aspects of the KML specification to enhance both the visual quality of our map and the performance. First, the data is divided by layer and regions. The reduced state and county data is sufficiently small to keep in single files, but the place and tract data requires sub-division. Places are divided into 5 degree x 5 degree grids with tracts divided into 1 degree x 1 degree grids. Output KML files are ZIP compressed into KMZ files to reduce bandwidth requirements. Individual files are joined together into our seamless presentation via ‘region’ coded KML network links. With regionation, Google Earth dynamically loads and draws data as required by each user’s changing view of the map. In setting up our KML, we specify the relative ’switchover’ points between layers via region attributes in our network link files.

Fill colors and pop-up content is controlled via KML Styles. KML Styles apply a consistent look and feel across our data and reduce duplication in the files for faster load times. We used the Google Chart API to create our ‘meter’ icons in each balloon.

The Map “Gadget”
Finally, Google wanted to provide an easy way for the public and media to view and share our map. For this, we turned to the Google Earth API & Plugin. The Google Earth Plugin is a cross OS and browser application for running Google Earth within a web page. The Google Earth API provides a set of JavaScript functions to customize and control the map. Via the API, we added the ability to search and zoom to a given location, generate custom links for the map, and create an ‘embed code’ similar to embedding a YouTube video via simple user interfaces.

What’s Next?
Following the Take10 Campaign, we plan to provide both our raw and simplified state, county, place, and tract KML files for free download to 3rd party developers. We’re already looking at additional datasets and new features in the future.

About Innovation Landscape
Innovation Landscape is a Delray Beach, FL based innovator in geo-spatial programming services for businesses, government agencies and non-profit organizations. We are a Google Qualified Maps Developer and have an extensive portfolio of map design projects. For more information about our services, please explore our website or contact us directly by phone or e-mail.