... geocoding continued

As I had mentioned in previous email Esri has retired it's public-facing geocoding and routing services. These services are now only available behind their paywall (through an ArcGIS Online subscription). Esri gives UD a fixed number of credits on our subscription that we can use towards their various services.  The cost for use of their Geocoder service is relatively expensive, as far as their services go.  For example 300,000 would take nearly our whole allocation of service credits.  However, for lesser numbers of geocode (for one-off needs) we can allow you to join our subscirption through an ArcGIS Online account.  Contact us for more information.

For all other geocoding needs, I recommend the following desktop solution:

Download Disc 2: StreetMap North America (Detailed Road Network, Geocoding, Routing) from the UD GIS Data page.

You can then choose one of the .loc files in streetmap_na\data when it asks for an address locator in these instructions: http://resources.arcgis.com/en/help/main/10.2/index.html#/in_ArcMap/002500000026000000/

Plotting and Projecting Coordinates

So you have some data that includes latitude and longitude, you add it to the map ... and it doesn't line up.  You've got some spatial reference/projection problem.  Figuring out what coordinate/projection system your data came from is not always straightforward, and often involves looking more deeply into the source of your data (for metadata).  One rule of thumb: if your data shows up around the horn of Africa in the Atlantic Ocean, you're probably trying to display unprojected coordinates in a view (or with other data) that's in a projected system.  Most of the time that you have unprojected coordinates the data will be in WGS 1984 or NAD 1983 (HARN or not).  You can often figure out which one by trial and error.

In any case here are all steps for going between unprojected coordinates and a projected system.  These are instructions for ArcMap, but similar would apply for QGIS.

We'll plot the coordinates, export the data, and project it.

1.  Add the tabular data (xls or csv usually) to ArcMap.  Right click on the layer and choose "Display X Y data", choosing latitude, longitude and WGS

2.  From the created plot layer, export to feature class (shapefile), and add this to the map

3.  Project this shapefile into the projection of the GIS data

ArcGIS Online for Geocoding

As I noted in my last post, UD has an ArcGIS Online subscription. As part of that service we have credits to be used (frugally) towards geocoding. Just let me know you are interested, and I'll sign you up. You'll receive an email with details about logging in and completing the sign up. Instructions for pointing to the service from ArcGIS, with a link to instructions on geocoding a table of addresses, is here: http://resources.arcgis.com/en/help/main/10.1/index.html#//00250000004v000000

Geocodes cost UD 1 service credit per 25 addresses. If you would like to geocode more than 100 addresses, I may refer you to other options for geocoding.

UD's ArcGIS Online Subscription

You'll be glad to hear that the ArcGIS Online Subscription is included as part of our campus site license.  We've been granting access to the subscription for no extra charge.  If you are paying your annual ArcGIS charge, you're already paying for a portion of our campus site license.

However, we have a limited number of service credits (12,500 remaining at the moment).  The administration tools for the subscription are pretty limited, so we ask that you use credits carefully.  Here's a link to see how credits are used: http://www.esri.com/software/arcgis/arcgisonline/credits.  In many cases there are desktop-based alternatives for the most "expensive" operations (e.g., tile caching, geocoding, analysis).  I would be happy to suggest alternatives if you think something is looking expensive.  We may also end up charging for the subscription at some point, but I don't anticipate that we'll do that in the coming year.

Once I've invited you to the subscription, you'll see an email notification letting you know how to connect.  If you already have an Esri account, you can sign in with that.  Otherwise you'll be prompted to create an account.  If you already have a "public" ArcGIS Online account, you may want to keep it separate from the subscription.

Doing "Bulk Imports" to OpenStreetMap

Uploading existing datasets of any significant size ("bulk imports") to OpenStreetMap involves serious consideration, above that of ordinary updates and edits.

I have done bulk imports to OSM before for UD/Newark Data.  I used shp2osm (http://wiki.openstreetmap.org/wiki/Shp2osm) to convert from shapefile to osm format and then used JOSM (http://wiki.openstreetmap.org/wiki/JOSM) to compare to osm data, edit where needed, and upload the data/changes.

Here is a comprehensive page on doing bulk imports: http://wiki.openstreetmap.org/wiki/Imports

OSM gives priority to smaller edits, as part of its grassroots philosophy.  Partially because of this, it's important to tread lightly when doing bulk updates.  That's why I like JOSM. I can see the existing data and how my updates will change that (if so) or potentially create duplicates.  The imports guidelines link (http://wiki.openstreetmap.org/wiki/Import/Guidelines) is especially helpful in understanding norms and rules for updates.

Excellent public remote imagery webinar

best webinar I've ever seen on public satellite imagery, though it's by Exelis so expect lots of ENVI examples https://www3.gotomeeting.com/register/697671558

csv2gft

I've posted a public branch of code from the Lima Bean project to github as csv2gft.  This might be very useful for someone who is doing daily pushes of new records to a Google Fusion Table from a csv dump.