Finder Maker

GeoCommons Finder User Manual

What is GeoCommons Finder?

Finder! is a browser-based application for finding, organizing, and sharing GeoData in common formats. Designed to both host and store GeoData, Finder! enables you to:

Store and organize disparate data from all ends of your organization in one central location.

Increase data efficiency with easy-to-use open standards through our browser-based application that allows both technical and non-technical users to find, create and share GeoData rapidly in a variety of formats that work in both traditional geographic information systems (GIS) and newer GeoWeb workflows.

Types of Data Supported

GeoCommons is designed to provide easy sharing and conversion of your data with others. The platform supports many popular formats from geospatial as well as enterprises and the web. Using GeoCommons Finder, you can upload data and Finder will automatically convert and store in our highly scalable data store and provide access to download in numerous formats, or access via our programming interface.

Description of Data Types

Shapefile: A complete shapefile is a set of files that contains geospatial vector data, created using ESRI software applications. This file type is commonly used for GIS software. Shapefiles are used to define point, polyline, and polygon files that are spatially referenced.

A shapefile is comprised of multiple .files/extensions. The required files for use with Finder! are:

*.shp - the file that stores the feature geometry

*.shx - the file that stores the index of the feature geometry

*.dbf - the dBASE, or database, file that stores the attribute information of features

*.prj – the file that stores the projection used (only required if using projection other than WGS 1984)

CSV: CSV is a type of data format in which each piece of data is separated by a comma. This is a popular format for transferring data from one application to another because most database systems are able to import and export comma-delimited data.

Data pulled from a database and represented in comma-delimited format looks something like the following:
Lastname, Firstname, Age, Sex, Location, Latitude, Longitude
Adams, Jane, 46, female, New York, 40.46, 73.54
Doe, John, 32, male, California, 38.16, 121.56

Each column’s value is separated by a comma from the next column’s value and each row starts a new line. Each feature must have a latitude and longitude in order for GeoCommons to ingest and map the data.

KML: KML is an XML-based language for managing the display of geographic data in an Earth browser such as Google Earth, Google Maps, Google Maps for mobile, and NASA WorldWind.

GeoRSS: GeoRSS is an emerging standard for encoding location as part of an RSS feed. Finder! currently supports GeoRSS-W3C, GeoRSS-Simple, and limited GeoRSS-GML.

Importing

The Finder! platform supports import of shapefiles (SHP), comma separated values (CSV), Keyhole Markup Language (KML), and GeoRSS.

Exporting

The Finder! platform supports export of many popular data formats: Shapefile, CSV, KML, GeoRSS Atom, Spatialite, and JSON.

Creating an Account

Click on the ‘Sign Up’ button located in the upper right corner of the screen.

Finding Data

Finder! enables you to search and browse within your own appliance or across multiple GeoCommons appliances through federated search.

Type a keyword or phrase into the search bar on the Finder! homepage. You can improve your search experience by searching for data by source, location, time period, or title keywords. The results page will display any datasets that match your keywords in either their title and/or tags.

Sort By

This feature allows you to arrange your search results by relevance, age and name. Your initial search results will automatically be sorted based upon relevance to the search term or terms. Finder! will display the results that it believes best fit your search criteria. Your results can also be sorted by date of upload. Simply, click the Newest First button to sort your search results to display the most recently uploaded layers. Select the Oldest First button to show the opposite. Your results can also be placed in alphabetical order by selecting the Name – A to Z or the Name – Z to A buttons. This will place your results in alphabetical order based upon the title of the data layer.

Browse

The browser function makes use of our topical and locational vocabulary and enables the user to browse the repository by category. These broad categories break up the thousands of datasets available through Finder! so you can quickly find the data you want. In the Browser, click the category link to see the category menu and then select either the main category or any of the subcategories.

Next Steps

Once you’ve found data of interest, you can select from the following options, all of which can be accessed from either the search results page or the details page:

Visualize data in Maker! by clicking the ‘Make a Map’ button. You will be linked to the Maker! application, and the data is automatically loaded into the Map Brewer.

Download the data as a spreadsheet (CSV), shapefile, or a KML file.

Save a copy to My Layers by clicking the ‘Edit a Copy’ button.

Downloading Data

As stated above, you can download from the search results page or the details page. Simply select which of the three supported file types you prefer; CSV, SHP or KML.

You will be prompted to select a location to which the data will download. If downloading a SHP, Finder! automatically compresses the file into a zipped format in order to make it as fast as possible. All you must do from here is unzip the file, and it is ready for your personal use.

Uploading Files From Your Computer

Finder! has been updated including some great new features like batch upload, international geocoder, and a pending layers page. All of these new features work to streamline the management of your data layers.

When uploading files from your computer select the Upload Files button found on the Finder! homepage. Use this function to upload:

After clicking on the Upload Files button, the batch upload screen will appear. This is a new function to Finder! that allows users to upload multiple files at the same time. These files will be saved to a page titled Your Pending Layers.

When uploading a shapefile you must include the .shp , .shx , .dbf file extensions. The batch upload process will merge all files into one data layer.

To get started, click the Add File button, and select which files are to be uploaded. The upload process will begin automatically . As each selected file is being loaded a status bar will show the progress of each file. When the file is finished uploading, the status bar will display a complete message.

After all files have been uploaded click the next button to continue with the upload process. The next page is the Your Pending Layers page; this contains all data layers that have not completed the upload process.

Please note that when the entire upload process is completed for a layer that file will be moved from Your Pending Layers to the Your Layers page.

Clicking the Next Step button for the desired dataset will begin the Locate process. There are three options to choose from when locating the data layer:

Depending on the content provided with the data layer, select the most appropriate option.

Use the existing Geographic data in the file you uploaded: Select this option when the data layer already has geographic data associated with it. For example, when uploading KML and shapefiles, geographic location data is already present, and there is no need to geocode or join with a boundary dataset.

Please note that if the data layer does not have any geographic data associated with it, this option will not be available.

Geocode based on an address or place name: The Geocommons geocoder is a powerful tool that allows you to obtain Latitude and Longitude coordinates from an address or place name. By integrating it into the upload process, Finder! streamlines the geocoding/upload process, and has eliminated the need for secondary geocoders. Select this option when your data layer contains address or place name information.

Join with a boundary dataset: This option allows users to merge a personal data layer with an existing boundary dataset in Finder!, for example: joining a CSV file with boundary (polygon) file. Use this option when the data layer contains locational data such as a state/country name etc. that matches the same data in the existing boundary file. For example, if the data layer contains state level data in the United States, the layer can be joined to a boundary file by joining the state names together.

Using Existing Geographic Data in the File You Uploaded

Select this option if your data layer contains geographic data. Finder! will automatically analyze the geographic data and create an official dataset. Upon the successful creation of the dataset, you will be taken to the details page. From here select the Edit button to begin the description process.

During the editing process users contribute important background information associated with the dataset. This is an essential part of the uploading process.

Please see the description portion of this manual for more information regarding this process.

Geocode Based on an Address or Place Name

Choose this option when your data layer contains address information such as a street addresses or place names. This option is predominately used when uploading a CSV file. After selecting this option users will be taken through the geocoding process. It is here that users define the locational information, and obtain the latitude and longitude coordinates for each point. The resulting layer and map will be a point based layer created from the lat/long coordinates.

The next step in the geocoding process is to geolocate your data. After selecting the Geocode based on an address or place name option, you will be directed to the locate page as seen below. The geocoder will automatically detect column titles that it recognizes as address information or place names. Titles such as city, state, address, etc. are examples of column titles that the geocoder will recognize. If you believe that Finder! has detected the appropriate columns, click the Continue button to start the geocoder. Finder! will obtain the latitude and longitude coordinates and bring you to the review screen.

You also have the option to define the columns that you wish to use in the geocoding process. Click the Select different columns link to choose exactly which columns are used to locate your data. The next page will list each column title as an attribute. You can then select the format of each attribute by selecting the edit button in the Data Format column. After clicking the edit button, three tabs will appear to the right; those tabs are titled Address, Geographic, and Standard. Select the proper format from these tabs for each of your attributes.

Note the sample data section of this page. a portion of the attribute’s data is shown here in order to help with the format selection.

The Default Country feature helps in accurately locating your points. If all of your data is contained within a single country, please make note of that using the drop box. If the dataset spans multiple countries or you are uncertain, select the Entire World option, and Finder! will do its best to locate each point given the information provided.

Once you have selected all of the appropriate options, click the Continue button to retrieve the coordinates from your specifications. Upon successful geocoding, you will be brought to the Review screen.

Finder! will offer a summary of the geocoding results, noting the exact number of geocode matches made, and will also add two attributes to the data set including Precision and Score columns titled geo_precision and geo_score, respectively.

Precision: shows the geographic level at which the data was geocoded to. Range has the finest level of granularity (the house number was interpolated). Street is the second finest level, the street name was matched and the start of the segment number closest to the house number is where the point was placed. If the Precision is Zip code, City, or State the row was geocoded to the middle of the geography.

Score: (Geo-confidence score) An estimate of how likely the location assigned by the geocoder sorresponds to the place you intended. 0.0 – 1.0 scale. 0 zero being the least confident, 1 being very confident.

This page displays a sample of the data results. If you wish to see the results in their entirety, click the Expand View button. This will direct you to the full list of results including the added columns: Precision and Score.

If the results are not as you expected you can return to the locate page by clicking on the Back button. On the other hand, if everything looks to be geocoded properly, click the Continue button.

On the next page you will be brought to the description page; this is where you enter important metadata regarding your dataset, including title, tags, description, source, etc.

Please make note that the FortiusOne Geocoder was used to process the data in the How was this data processed section. Please see the description portion of this manual for more information regarding this.

Join With a Boundary Dataset

Choose this option when merging a personal data layer with an existing boundary file. This allows users to join a CSV file with boundary (polygon) files through common names or identifiers found within both datasets.

To get started select the Join with a boundary dataset option; this will take you to the geolocate page where you will search for an existing boundary file within Finder! to Join your layer with.

To find the appropriate data set, search or browse Finder!. On the geolocate page there is a search bar where you can enter a key word or term. You can also browse the Finder! library by selecting from the categories listed on the left side of the screen.

When you have found a data layer that you wish to join with, click the Select button to continue the joining process.

On the next page you will select which attributes you wish to join; there are two boxes which contain all attribute titles from your data layer and the selected layer. Your Data’s attributes will appear in the box on the left, while the selected layer’s will appear in the box on the right. Scroll through each box and find the layers that contain matching information.

To aid in selecting the proper attributes, the Attribute Preview function shows a brief sample of the attribute data.

As you can see in the example above, the two layers were joined by state name. A brief summary of the join results can be found under the Continue button. In this case 51 of 54 features matched exactly and were joined.

Please note that the attribute data needs to match exactly for the join to be made; the join process is case sensitive.

Users can view the complete description of the data set by clicking on the link: Open the complete description and stats in another window.

When you are satisfied by the results of your join, click the Continue button to move on to the review screen. Here you can view the results of the Geo Join. As always, click the Expand View button to view all of the results.

After reviewing the data, click on the Continue button to Describe your data layer. During this process users contribute important background information associated with the dataset. This is an essential part of the uploading process.

Please see the description portion of this manual for more information regarding this process.

Use External Data From the Network

Finder! currently supports RSS and ATOM formats for linking to a dataset that is available through a web URL.

KML/KMZ: KML is an XML-based language for managing the display of geographic data in an Earth browser such as Google Earth, Google Maps, Google Maps for mobile, and NASA WorldWind. KML files are very often distributed in KMZ files, which are zipped files with a .kmz extension.

GeoRSS: GeoRSS is an emerging standard for encoding location as part of an RSS feed. Finder! currently supports GeoRSS-W3C, GeoRSS-Simple, and limited GeoRSS- GML.

Text Documents for GeoParsing: Using MetaCarta’s technology, geographic references contained in a document are identified and made useable as a data layer. Text documents can be uploaded as .txt files or as an URL link.

Creating a link to your desired webpage is simple. Starting in the Store, Organize and Share your GeoData section of the Finder! homepage, select the Link to Data option. On the following page:

  1. Paste your desired URL in the first box.
  2. Select the file format from the second drop box.
  3. Select an external database source to load from drop box (if appropriate).
  4. Click the Upload button.

After completing these steps, click the continue button. Finder! will automatically poll your URL for geographic data. On successful detection of features, you will be taken to the Locate screen where you will continue with the uploading process, ending with describing your data.

Describing Your Data

Titling Data

We have established a general guideline for naming your datasets. We suggest including the following information in the title and keeping them separated by commas: Description, Location, Time Period Covered.

Here is an example of a bad title and a good title for the same dataset:
BAD: Last year’s income data
GOOD: Average Household Income, Maryland – Block Level, 2000

It is important to create accurate titles so that both you and other users can enjoy easy searching and browsing of the data repository.

Tagging

Tagging allows you to link basic keywords and terms to a dataset. These tags will be picked up by the search and the browsing capabilities within Finder!. This is another step that will help users quickly and easily find the data that they need.

We advise tagging your dataset with one of the categories and/or subcategories listed in the ‘Browse’ section on the Finder! homepage.

Note that the area under the tagging bar is populated with possibly relevant tags that you can click to add.

Sharing Your Data

In the Share section of the description page, you can elect to share your data with the entire network by choosing “Anyone,” or you can keep your data private by highlighting “Only you.”

Be sure to select the Done button after making any changes in order to save them.

How Was This Data Processed

If your data layer went through a geocoding process to obtain geographic coordinates, please make note of what geocoder was used in this section. You can also add a link to the geocoder as well as a brief description on the geocoding process.

Entering Metadata

Metadata is essentially “data about data.” Metadata (sometimes written 'meta data') is used to facilitate the understanding, characteristics, and management usage of data. Within Finder!, this includes a description of the dataset, when it was originally published, the source, citation URL, metadata URL, contact information of user who uploaded the data, and descriptions of the attributes.

Once you have completed the description process, select the Save button. Your new dataset will be moved from Your Pending Layers to Your Layers with all of the saved information. From here you can visualize all of your data layers in Maker!.

The importance of Metadata:

Protects investment in data:
Sets the stage for data re-use and update
Mitigates the effect of staff turnover and individual memory loss
Provides documentation of data sources and quality

Helps the user to understand the data:
Provides consistency in terminology
Focuses on key elements of data
Helps the user to determine the data’s fitness for use
Facilitates data transfer and interpretation by new users

Enables discovery:
Provides flexibility in searching to support interdisciplinary usage

Limits liability:
It can prevent data from being used inappropriately or provides protection if the data is inappropriately used.

Evidence of prudent data stewardship:
An organization that takes the time to create and maintain quality metadata will also most likely develop quality, clean data.

Reduces workload associated with questions about data:
Many of users’ basic questions can be answered through the metadata.

Editing Data

You can edit the title, description, metadata, and attributes of any dataset by clicking the ‘Edit’ button under the title of your own datasets or the ‘Edit a Copy’ button under the title of others’ datasets.

If you need to edit the actual data, you will need to download the data, make your changes, and then reload it to Finder!.****

Supported Flash Players and Internet Browsers

The GeoCommons suite has been built and tested to support multiple versions of Flash web players, as well as internet browsing applications. In order for Finder! and Maker! to work at full capacity, please be sure that your system is up to date and running a supported Flash player and browser. The following is a list of supported Flash and Browsers:
Flash:

Here is a link to the download page for Adobe Flash: http://www.adobe.com/shockwave/download/alternates/

Internet Browsers: