Last fall we participated in the exploration and standardization of KML in the OGC as part of the OWS-5 testbed. The culmination of this effort was demonstrating some of the key features of KML 2.2, integration with other OGC Web Services (hence the OWS), and the broader GeoWeb.

Mapufacture volunteered to put together a demonstration use and screencast for illustrating this that we presented at the summary session of the three day TC (Technical Committee) meeting. The following video is a 3-minute demonstration using KML and the other tools in Mapufacture to create a “Personal Decision Support System” to help find an apartment and office space in San Francisco. It brings together both complex geospatial data such as Census Demographics of Median Age (as a WMS from GeoServer), as well as user-generated content from Craiglist (via a Yahoo Pipe to generate a GeoRSS feed), news from Topix, and events from Upcoming.org.

The resulting map is then viewable on a mobile phone via uLocate’s WHERE platform as you actually travel around to view the suggested listings. We even took it a step further showing an example of using Socialight to leave geolocated notes and attribute information such as number of rooms that is then dynamically brought back into the map and can be used for decision making and visualization.

Mapufacture Demo - Personal Decision Support from mapufacture on Vimeo.

The video is slightly technical, but gives a good, quick overview of the power of linking together various data sets form both qualified sources as well as user-generated content to aid in decision making and collaboration.

Last week I attended the OGC Technical Committee meeting in St. Louis, Missouri closing up the OWS-5 testbed. More on that soon.

The OGC has different types of groups: Domain Working Groups (DWG) and Standards Working Group (SWG). Where SWGs are very formal, and working on defining a specific standard, such as WFS, a DWG is a more broadly scoped discussion about an area or application space.

One of the DWGs we participated in was the Mass Market (MM) DWG. Ed Parsons shares his thoughts on the benefits of the MM-DWG. Namely, to track standards and trends that are occurring outside the OGC, especially outside GIS-specific domains.

So for the session, I put together a presentation briefly outlining some of the very recent happenings in standards: REST, AtomPub-geo, OpenSearch-Geo, GeoJSON, and GeoRSS-Multi.

These formats have been very successful in their development and adoption. GeoRSS was one of the first and took awhile, but is now supported by most of the major mapping libraries and many GIS tools. GeoJSON and other formats have seen a much quicker adoption (GeoJSON is still ‘going 1.0′ but already used by Yahoo Pipes and FireEagle).

One of the big reasons these formats are so popular is that geo-developers worked to add geographic capabilities to already ubiquitous standards (e.g. RSS, Atom, JSON, OpenSearch) instead of trying to create a geographic specific format and redesigning the system and bringing people to them (e.g. SLD, WFS, GeoRM).

We’ve been tracking and help to put together a number of these agile geography formats with the rest of geo-community and also keeping a mind to what’s already been developed outside of the Geo world.

The OGC is starting to engage the non-geo community by championing how GML can be the geographic markup within other standards such as GeoPriv, GeoRSS (non-simple) and so on. This is the right approach, though GML is still too heavy for your average non-geo developer to just pick up and easily add into their toolset.

Mapufacture utilizes a number of these formats. We’re huge proponents of supporting and sharing data via standards - especially ones that encourage broad adoption. It allows customers and developers to more easily integrate with your software and data sets. This is especially true since we’re working with non-GIS experts who want to utilize their common tools for building maps; whether they are spreadsheets, wikis, or RSS readers.

The slides are very XML oriented - keeping a mind to my audience.

There was a very good discussion today during the OWS-5 Agile Geography telecon - with almost all attending interested parties participating and a lot of good points brought up.

The foremost is a clarification of the goals and purpose of the OGC Web Services Testbed. The testbed is important to remember that it’s partially a venue to experiment and brain-storm on future standards. It is not an actual formal development of the standard.

In the OGC, as I understand it, a testbed is used to formulate, and implement, potential standards. Then a summary report and presentation is made to the OGC Technical Committee (TC). The TC then actually decides and votes on the standard.

So the current work that has referred to “KML3″ should be construed as hypothetical development. But, it also seems why it’s all the more important to have many users, developers, thinkers, hackers share their thoughts and example implementations now - while the iron is hot and malleable (to make a bad metaphor).

The Goals

One original goal, that wasn’t clear to me (and therefore reflected in my posts) was the primary desire to just determine and demonstrate the ability of existing OGC services to generate KML 2.1/2. This means showing how to take a WFS service that outputs GML, apply SLD, and get KML with styling. Or do the same with WCS or a WMS (just as KML vectorized data). Then to point out any small problems with KML 2.2 that make this conversion difficult.

This is definitely a useful goal - but I don’t feel that was made clear from the beginning. I also don’t think it will be that difficult - but since Mapufacture already outputs KML natively, I’m not sure - which then I imagine does make the case for having a testbed to show that.

Other goals that have been discussed, but just haven’t been blogged (or documented) include: KML as Context document, KML and GeoRSS interoperability - though I blogged about it previously and have heard general acceptance of the ideas, Sensor streaming, and styling imported geometry.

I’ll cover some of these in follow up blog posts.

Hopefully that clears up a little the OWS Testbed goals. Mapufacture is new to the OGC process, though I don’t think it’s very well understood in general amongst the geo-community from outside the OGC. So as we learn and figure things out, hopefully we can help share this information with others.

As part of the OGC OWS-5 work, the committee has a telecon every Tuesday afternoon. This is to ensure that progress continues throughout the 6-month project and any issues are raised early.

At this week’s telecon there was general agreement that there haven’t been too many concerns publicly raised about the current proposed KML modifications - and that in the next month the various participants would each prototype parts of KML 3 that are pertinent to their products.

For Mapufacture, we’ll be prototyping the updated KML Core, Metadata module, Service Links, and a Mobile version of KML3.

We agreed on a couple of other things as well regarding the scope of the OWS-5 effort. We won’t be addressing the possibilities of external schemas. Also, we won’t, as OWS-5, work on KML AtomPub (APP).

This doesn’t mean no one should - in fact Sean Gillies and Charlie Savage are getting together a demo interop that Mapufacture will participate in to work out APP with geo-apps, including possibly using KML as the content encoding.

No issues?

I did say there were no publicly raised issues. However, there are rumors of privately raised concerns about some of the KML concepts, especially as related to styling, by developers that have a lot of experience developing KML visualization clients.

However, despite multiple requests by individuals, and the committee, these developers, or company, have not yet shared their experience, thoughts, or concerns with the committee. This is surprising considering this company is a core part of the Agile Geography thread and has a vested interest in the future of KML.

Why they would choose to not share their insight either means they’re too busy, open to new suggestions and really want the insight of developers and users not intimately tied to a particular implementation - or that they don’t really care about the public standardization of KML and will do what they want, regardless of what OGC decides for KML.

I hope they are just waiting to see what emerges, and in the end they’ll completely support the standard. However, based on their experience and supposedly having gone down some of these roads, it would be very appreciated if they would offer their advice.

Other standards

While not within the scope of KML3 and OWS-5, the committee is somewhat participating in other working groups within the OGC - which is usually referred to as “Mass Market”. The OGC will be proposing standardizing on the proposed Geolocation in HTTP headers draft spec.

Need feedback

No one currently on the committee currently is heavily involved with 3D visualization. We would definitely welcome feedback on features, standards, supports, and use-cases you would like to see in KML for supporting models, visualization, and immersive reality. Should KML support X3D? Only Collada? Do you want to specify linking of bodies in KML?

FOSS4G

As the last item of business, we agreed that we’re going to try and get a demo together for FOSS4G of the current spec. This will mean various implementations of Metadata, Styling, Service Links to play with in your own clients and server systems.

As I mentioned on HighEarthOrbit, I’ll start blogging about the OGC OWS-5 KML 3 effort here on the Mapufacture blog. We’ll be implementing the proposed standards on Mapufacture and pushing the edge of the GeoWeb. These posts will be fairly technical, which is why I’ve made a new category “Technical” - so if you want just these posts, subscribe to that category - or the opposite if you don’t want the nitty-gritty details of KML. If you need to catch up on what this is all about, check out my first post on KML 3.

So far I’ve covered the modules: Core, Styling, and Metadata. Another important one is the Services module. This defines how KML can link to external resources like OpenSearch, WFS, WMS, or others.

What are the use cases for the Service link?

• Load KML geometry based on location and a search term from a search service like Mapufacture
• Ground imagery from a WMS such as GeoServer
• Streaming sensor data like from GeoBliki
• Loading GML and a styling SLD from a WFS such as Galdos suggests

You may have noticed that the examples I gave in the use-cases above include the participants of the OGC OWS-5 Agile Geography thread. Just a small insight into the priority of use-cases when developing a format. However, if you have your own use cases, speak up now in the comments or your blog (and tag your posts/links with ogckml so we can easily pull them up).

Current NetworkLinks

Anyways, a KML link currently looks like:

<NetworkLink>
  <name>NE US Radar</name>
  <flyToView>1</flyToView>
  <Link>
    <href>
        http://www.example.com/geotiff/NE/MergedReflectivityQComposite.kml
    </href>
    <refreshMode>onInterval</refreshMode>
    <refreshInterval>30</refreshInterval>
    <viewRefreshMode>onStop</viewRefreshMode>
    <viewRefreshTime>7</viewRefreshTime>
    <viewFormat>
        BBOX=[bboxWest],[bboxSouth],[bboxEast],[bboxNorth];
        CAMERA=[lookatLon],[lookatLat],[lookatRange],[lookatTilt],[lookatHeading];
        VIEW=[horizFov],[vertFov],[horizPixels],[vertPixels],[terrainEnabled]
    </viewFormat>
  </Link>
</NetworkLink>

KML 2.2 already lets you specify a base URL, and by default appends a BBOX parameter based on your current viewing window in a client like GoogleEarth. You can also optionally append on other view parameters that allow you to optimize your server results.

There are a few problems with the current service link definition. You are limited in what optional parameters you can include in your URL. You can’t template time parameters, geographic bounds other than bounding box (such as point/distance or polygon), or the service type.

There is a rather hidden way to add WMS layers currently to GoogleEarth. Go to “Add” -> “Image Overlay…”. click the “Refresh” tab, then click “WMS Parameters”. You can then choose a WMS server and add some layers. However, this method just builds up the WMS query for you and isn’t really part of the KML spec, but just a feature of GoogleeEarth.

Full Featured Network Links

To look at options, we investigated the current Web Context Service (WCS) definition, which includes methods for linking to WFS and WMS services. We then pared this down to be simpler, but kept he functionality for specifying the type of service, version, and other parameters:

<Server service="OGC:WMS" version="1.1.1" title=""
href="http://demo.cubewerx.com/demo/cubeserv/cubeserv.cgi"/>

By specifying the service type, OGC:WMS, clients can then determine if they support the service (they know how to build urls and parse the response), and also what a fully formed request looks like. For example, WMS requires appending various parameters for coordinate reference system, request type, format, and so on. By saying OGC:WMS, and being in a KML document, a lot of this information can be inferred.

In addition, we investigated supporting OpenSearch-Geo and -Time based templating for more generic searches.

Here is a possible example OpenSearch (or really any templatable URL):

<Server service="opensearch" version="1.1"
  title="Mapufacture keyword search" href="http://mapufacture.com/search.kml"/>
<viewFormat>dtstart=[timeStart]&dtend=[timeEnd]&keyword=[keyword]
  &bounds=[bboxWest],[bboxSouth],[bboxEast],[bboxNorth]</viewFormat>
<param name="keyword" editable="true">pizza</param>

What this shows is extending the viewFormat to include template parameters for time bounding (which GoogleEarth already supports in the GUI), and also using the existing BBOX template definition. By default, all params would be appended in as keyword=value pairs to the URL.

However, we’ve also considered adding the ability to extend the template with other parameters, keyword in this case, and also specified that it was editable, so the service would let the user change it via a UI and reinsert it into the URL to retrieve an updated KML document.

Imagine a KML client that parses these service definitions and sees that there are optional parameters such as “keyword”, “tags”, layer-type, or username, and then displays an input box for users to easily alter this value and request new data, without having to modify the service URL in the KML itself.

Here is a possible full KML 3 document with a WMS and OpenSearch service definition:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.x">
<NetworkLink>
  <name>WMS Link</name>
  <description>generates
   http://demo.cubewerx.com/demo/cubeserv/cubeserv.cgi?...
    VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG:4326&WIDTH=512
    &HEIGHT=512&LAYERS=BNDTXT_1M:Foundation,ROADL_1M:Foundation
    &STYLES=black,0xaa1818&TRANSPARENT=TRUE&FORMAT=image/gif
    &BBOX=WSEN
  </description>
  <Region>
    <gml:envelope>
      <gml:lowerCorner>42.943 -71.032 500</gml:lowerCorner>
      <gml:upperCorner>43.039 -69.856 5000</gml:upperCorner>
    </gml:envelope>
  </Region>
  <Link>
  <Server service="OGC:WMS" version="1.1.1" title=""
    href="http://demo.cubewerx.com/demo/cubeserv/cubeserv.cgi"/>
    <viewRefreshMode>onStop</viewRefreshMode>
    <viewBoundScale>0.75</viewBoundScale>
    <param name="LAYERS">BNDTXT_1M:Foundation,ROADL_1M:Foundation</param>
    <param name="STYLES">black,0xaa1818</param>
    <param name="FORMAT">image/gif</param>
    <param name="KEYWORD" editable="true">Interstate</param>
  </Link>
</NetworkLink>
<NetworkLink>
  <name>OpenSearch service</name>
  <description>
    generates http://mapufacture.com/search.kml?keyword=pizza&bbox=wsen
  </description>
  <Link>
    <Server service="opensearch" version="1.1" ...
      title="" href="http://mapufacture.com/search.kml"/>
    <viewRefreshMode>onStop</viewRefreshMode>
    <viewBoundScale>0.75</viewBoundScale>
    <viewFormat>
      dtstart=[timeStart]&dtend=[timeEnd]&keyword=[keyword]
        &bounds=[bboxWest],[bboxSouth],[bboxEast],[bboxNorth]
    </viewFormat>
    <param name="keyword" editable="true">pizza</param>
  </Link>
</NetworkLink>
</kml>

This demonstrated a couple of additional concepts that already exist in KML (but now using the GML spec) such as <Region/> that specifies the available region that this network link applies to. Don’t bother (in fact, please don’t) query outside of this region because the service doesn’t have meaningful data.

Does this cover your use cases? Tim Schaub has brought up some good points on how to handle encoding of the parameter values, especially when trying to parse comma separated values in non-well-formed strings (that may include commas as part of a name, but shouldn’t be parsed on).

More soon on the possibilities of using KML 2.x/3 as a ‘context document’ and also KML 3D.