Finder Maker

GeoCommons API

Finder API

Finder is a data repository where users can upload, find, and download data. The API makes it easy to programmatically tie in existing data stores, or updating feeds, into a Finder repository. In addition, the API provides for easy ways to search and download data.

Overall Capabilities

  1. Upload data
  2. Update metadata
  3. Search for data
  4. Download data

Resources

Finder has a number of RESTful resource endpoints you can use for searching, listing, editing, or creating new overlays.

All endpoints support Atom publishing protocol (APP), as well as output via JSON.

Overlay

Here are some example representations of Overlays, or datasets, in Finder:

Atom

 

<entry>
    <title>World Bank, Trends in Average Applied Tariff Rates in Developing and Industrial Countries, Global, 1981-2005</title>
    <link type="text/html" href="http://finder.geocommons.com/overlays/167" rel="alternate"/>
    <link type="application/atom+xml" href="http://finder.geocommons.com/overlays/167.atom" rel="alternate"/>
    <link type="application/vnd.google-earth.kml+xml" href="hhttp://finder.geocommons.com/overlays/167.kml" rel="alternate"/>
    <link type="application/atom+xml;type=entry" href="http://finder.geocommons.com/overlays/167.atom" rel="edit"/>
    <id>http://finder.geocommons.com/overlays/167</id>
    <updated>2008-04-29T03:20:52Z</updated>
    <contributor>
      <name>data</name>
      <uri>http://finder.geocommons.com/users/data</uri>
    </contributor>
    <category term="business" scheme="http://localhost:4001/tags"/>
    <category term="economics" scheme="http://localhost:4001/tags"/>
    <category term="finance" scheme="http://localhost:4001/tags"/>
    <category term="money" scheme="http://localhost:4001/tags"/>
    <category term="tariff" scheme="http://localhost:4001/tags"/>
    <category term="tax" scheme="http://localhost:4001/tags"/>
    <category term="trade" scheme="http://localhost:4001/tags"/>
    <content type="html">This dataset tracks the average applied tariff rates in both industrial and developing countries. Data is averaged for the years 1981-2005. Figures for 2005 have been estimated. Notes: All tariff rates are based on unweighted averages for all goods in ad valorem rates, or applied rates, or MFN rates whichever data is available in a longer period. Tariff data is primarily based on UNCTAD TRAINS database and then used WTO IDB data for gap filling if possible. </content>
</entry>

Upload data

You can currently upload data using two methods: via files from your hard drive, or via a URL from the Internet.

Upload files

GeoCommons Finder accepts files via multipart form data uploads by posting to the /overlays endpoint. You can explicitly specify the file format you are uploading using any of: kml; shp,shx,dbf,(prj optional); csv; and rss/atom/xml. Just specify the individual files in an overlay[FORMAT]= parameter in the multipart upload.

An example of uploading a CSV file:


curl -i -X POST -u "yourusername:yourpassword" -F "overlay[csv]=@spreadsheet.csv;name=spreadsheet.csv" http://finder.geocommons.com/overlays.xml

An example of uploading a Shapefiles parts:


curl -i -X POST -u "username:password"  -F "overlay[shp]=@elect_precincts.shp;name=elect_precincts.shp" -F "overlay[shx]=@elect_precincts.shx;name=elect_precincts.shx" -F "overlay[dbf]=@elect_precincts.dbf;name=elect_precints.dbf" http://finder.geocommons.com/overlays.xml

Link to URL datasource

Finder can register data source URL's from the internet or a network. This allows users to define data sources that will be continually updated, aggregated and archived in GeoCommons.

To register a datasource, provide the URL to a file type that is supported in GeoCommons and POST using the overlay[wild_url] parameter to /overlays.xml.


curl -i -X POST -u "yourusername:yourpassword"  -d "overlay[wild_url]=http://earthquake.usgs.gov/eqcenter/catalogs/1day-M2.5.xml" http://finder.geocommons.com/overlays.xml

URL's can only be registered once in GeoCommons. So if the URL has already been registered by a user, you will receive a redirect to this existing Overlay. You can make a copy of that overlay if you want to modify the metadata.

Data sources are automatically updated periodically depending on the frequency of updates in the data and availability of the service and aggregation server.

Ping updates

It is possible to ping the GeoCommons server explicitly when new updates are available. Ping updates are only possible by the owner or authorized users of the registered data source, or URL.

Send an authenticated GET request to the overlay that you want updated. When a request is made, the aggregation is queued and will be updated as soon as possible:

curl -i -X GET -u "yourusername:yourpassword" http://finder.geocommons.com/overlays/#{id}/fetch

Update metadata

Finder uses the AtomPub protocol for accepting modifications to Overlays. To modify the metadata, you must do an HTTP PUT to the overlay resource you are modifying.


cat updates.atom | curl -i -u "yourusername:yourpassword"  -d @- -X PUT http://finder.geocommons.com/overlays/9313.atom

updates.atom


<entry>
    <title>Proposed Stimulus Spending, US, 2009</title>
    <source>http://www.stimuluswatch.org/project/by_state</source>
    <id>http://www.stimuluswatch.org/project/by_state</id>
    <updated>2009-02-03T12:00:00</updated>
    <author>
        <name>SwimulusWatch.org</name>
        <uri>http://stimuluswatch.org</uri>
    </author>
    <content type="html">
        These are "shovel-ready" projects for which the mayors of this state have requested federal stimulus funding. Data was aggregated to each city, so if there were multiple projects for one city the number of jobs and cost would be for all projects in the city.
    </content>
</entry>

The response will be an "OK" from the server:

 HTTP/1.1 200 OK
Connection: close
Date: Wed, 21 Jan 2009 22:19:59 GMT

Geocoding

For datasets without geographic coordinate information or if your latitude and longitude columns have non-standard names you must define the column types in order to be able to map. Currently GeoCommons supports U.S. geocoding only, but will be adding additional countries in the future.

Acceptable data types:

Standard:

string: Plain Text
integer: Whole Numbers
decimal: Decimal Number
datetime: Time or Date

Geographic:

address: Full or Partial Address
street: Street Address:
region: State of Province
postal_code: ZIP or Postal Code
city: City
country: Country
latitude: Latitude
longitude: Longitude

This is an example file with the various data-types defined.


 <?xml version="1.0"?>
  <entry xmlns="http://www.w3.org/2005/Atom" xmlns:gm="http://www.google.com/schemas/gm/1.1">
      <title>Overlay Title</title>
      <source>http://www.source.com</source>
      <id>http://www.source.com</id>
      <updated>2009-02-03T12:00:00</updated>
      <contributor>
          <name>Authorname</name>
          <uri>http://sample.com</uri>
     </contributor>
     <content type="html">
         Description of the dataset would go here.
     </content>
     <gm:attributes>
       <gm:attribute name='id' type='integer' />
       <gm:attribute name='name' type='string' />
       <gm:attribute name='percentage' type='decimal' />
       <gm:attribute name='Date' type='datetime' />
       <gm:attribute name='address' type='address' />
       <gm:attribute name='street' type='street' />
       <gm:attribute name='state' type='region' />
       <gm:attribute name='ZIP' type='postal_code' />
       <gm:attribute name='city' type='city' />
       <gm:attribute name='country' type='country' />
       <gm:attribute name='address' type='address' />
       <gm:attribute name='lat' type='latitude' />
       <gm:attribute name='lon' type='longitude' />
     </gm:attributes>
 </entry>

A minimum amount of data is required in order to successfully geocode, below are column combinations that are acceptable:

Full/Partial Address
Zipcode/Postal Code
Street, City, State
Street, Zipcode/Postal Code
City, State
City, State, Zipcode/Postal Code
Latitude, Longitude

Download data

Finder datasets are available in the following formats:

Format Content-Type Example URL
KML vnd.google-earth.kml+xml http://finder.geocommons.com/overlays/167.kml
CSV text/csv http://finder.geocommons.com/overlays/167.csv
Shapefile (zipped) application/zip http://finder.geocommons.com/overlays/167.zip
Atom* application/atom+xml http://finder.geocommons.com/overlays/167.atom
JSON* application/json http://finder.geocommons.com/overlays/167.json
Spatialite* application/x-sqlite3 http://finder.geocommons.com/overlays/167.sqlite

Sort and Filter data

GeoCommons currently supports accessing the entire datasets as a single download as well as requesting a subset of data by filtering. When filtering data, developers can set conditions for limiting features using any of the attributes in the dataset. Additionally it is possible to sort the returned data features by an attribute in ascending or descending order and limit the number of returned features. This may be useful get obtaining just the top 10-most populated regions, for example.

http://finder.geocommons.com/overlays/{overlay_id}/features.json?sort={sort_by}&limit={limit}&order={order}

Sorting currently only supports JSON response.

*BETA

Atom Output

The Atom output includes attribute data using the Google Data elements, &lt;g:data/&gt;

 

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:georss="http://www.georss.org/georss" 
    xmlns:g='http://base.google.com/ns/1.0'>
  <title>Election Precincts</title>
  <id>gc:overlay:1</id>
  <rights>Creative Commons Attribution-Share Alike 2.0</rights>
  <link rel="self" href="http://finder.geocommons.com/overlays/1.atom" type="application/atom+xml"/>
  <link rel="edit" href="http://finder.geocommons.com/overlays/1.atom" type="application/atom+xml;type=entry"/>
  <link rel="alternate" href="http://finder.geocommons.com/overlays/1.html" type="text/html"/>
  <link rel="alternate" href="http://finder.geocommons.com/overlays/1.kml" type="application/vnd.google-earth.kml+xml"/>
  <updated>2009-05-18T11:49:28-04:00</updated>
    <entry>
      <id>gc:overlay:1:feature:75dfecc9</id>
      <g:precname name="Precname">2415</g:precname>
      <g:objectid name="Objectid">1.0</g:objectid>
      <georss:polygon>...</georss:polygon>
    </entry>
</feed>

Maker API

Maker provides cartographic visualization and analysis. Using the Maker API you can create new maps, layer in data overlays and assign styling, search for existing maps, and download maps in number of formats.

Overview

  1. Create a map
  2. Add a layer to a map
  3. Search for maps
  4. Download a map

Resources

Maker has a number of RESTful resource endpoints you can use for searching, listing, editing, or creating new maps and layers.

All resources are available as JSON and KML output, and most endpoints currently support JSON for creation and modification.

Download a Map

Maker maps are available as: JSON or KML via the format parameter: http://maker.geocommons.com/maps/{id}.{format}
So for example, http://maker.geocommons.com/maps/5104.json or http://maker.geocommons.com/maps/5104.kml.

To download the actual features and styling information for a map's layers, as linked to in the map description, get the overlays resource:
http://maker.geocommons.com/maps/{id}/overlays/{index}.{format}

where {index} is the layer id in the map, starting at 0. So for example:
http://maker.geocommons.com/maps/5104/overlays/0.kml

Create a Map

To create a new Maker map, POST a new resource to the /maps endpoint:

cat new_map.json | curl -i -u "username:password" -d @- -X POST http://maker.geocommons.com/maps.json

Where new_map.json is a JSON encoding with the parameters for: title, description, tags, and bounding box. The bounding box, bbox, is West, South, East, North order for the longitude and latitude.


{ 
  "title": "Stimulus package: estimated highway funding for large urban areas",
  "description": "The map is based on estimates of highway funding for large urban areas under the stimulus package bill HR1...",
  "tags": "Highway funding,Stimulus package",
  "bbox": [-124.817, 26.4684, -69.27, 47.307],
}

The HTTP response is a Created:

Response
HTTP/1.1 201 Created
Location: http://maker.geocommons.com/maps/45
Content-Type: application/json; charset=utf-8
... extra headers snipped ...

Update a Map

To update an existing map, just PUT an new JSON document to the map resource:

cat update_map.json | curl -i -u "username:password" -d @- -X PUT http://maker.geocommons.com/maps/45.json

The HTTP response should be a normal 200 OK:

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
... extra headers snipped ...

Add Layers

The GeoCommons API does not currently support adding layers or styling. This will be included in future releases of the platform

Search API

GeoCommons provides a common search interface across the multiple applications using the OpenSearch protocol.

Responses are available as:

The OpenSearch description document can be found at http://core.geocommons.com/opensearch.xml.

Atom Example

The Atom template is:


http://maker.geocommons.com/searches.atom?query={searchTerms?}&limit={count?}&page={startPage?}

Query

An example query is http://maker.geocommons.com/searches.atom?query=Stimulus

Response


<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:georss="http://www.georss.org/georss" xmlns="http://www.w3.org/2005/Atom">
  <title>GeoCommons Search: Stimulus</title>
  <id>http://maker.geocommons.com/searches?format=atom&amp;amp;limit=10&amp;amp;page=1&amp;amp;query=Stimulus</id>
  <link type="application/atom+xml" rel="self" href="http://maker.geocommons.com/searches.atom?query=Stimulus"/>
  <link type="application/vnd.google-earth.kml+xml" rel="alternate" href="http://maker.geocommons.com/searches.kml?query=Stimulus"/>
  <link type="text/html" rel="alternate" href="http://maker.geocommons.com/searches?query=Stimulus"/>
  <updated>2009-06-06T22:16:10Z</updated>
  <link type="application/opensearchdescription+xml" rel="search" href="http://core.geocommons.com/search.xml"/>
  <entry>
    <title>Shovel Ready - Stimulus Funding</title>
    <link type="text/html" rel="alternate" href="http://maker.geocommons.com/maps/5104?format=html"/>
    <id>http://maker.geocommons.com/maps/5104</id>
    <updated>2009-05-05T15:14:23Z</updated>
    <author>
      <name>emmills</name>
    </author>
    <content type="html">
    </content>
  </entry>
...
</feed>