Monday, August 12, 2013

Doing "Bulk Imports" to OpenStreetMap

Uploading existing datasets of any significant size ("bulk imports") to OpenStreetMap involves serious consideration, above that of ordinary updates and edits.

I have done bulk imports to OSM before for UD/Newark Data.  I used shp2osm ( to convert from shapefile to osm format and then used JOSM ( to compare to osm data, edit where needed, and upload the data/changes.

Here is a comprehensive page on doing bulk imports:

OSM gives priority to smaller edits, as part of its grassroots philosophy.  Partially because of this, it's important to tread lightly when doing bulk updates.  That's why I like JOSM. I can see the existing data and how my updates will change that (if so) or potentially create duplicates.  The imports guidelines link ( is especially helpful in understanding norms and rules for updates.

Friday, August 9, 2013

Excellent public remote imagery webinar

best webinar I've ever seen on public satellite imagery, though it's by Exelis so expect lots of ENVI examples

Tuesday, July 30, 2013


I've posted a public branch of code from the Lima Bean project to github as csv2gft.  This might be very useful for someone who is doing daily pushes of new records to a Google Fusion Table from a csv dump.

Wednesday, May 8, 2013

Render Priority and KML from PHP/Postgres in one example

Working from a Google example I wrote some PHP that outputs KML from Postgres data, using region and lod to provide 'render priority' feel, based on a DB field.  Here you go:

Wednesday, April 24, 2013

Sublayers with ArcGIS Online webmap and Javascript API

ArcGIS Online ( does a nice job of providing front ends for your web mapping services (particularly of the ArcGIS Server variety).  In fact, ArcGIS Server 10.1 no longer comes with out of the box web mapping applications, relying instead on deployment through provides a couple templates, for different use cases, that you can download and modify (though the performance hit of having the middleman is a concern).

One of the stumbling blocks I've identified in this new model is that -- and by extension ArcGIS Server -- does not provide any templates to allow the end user to control sublayer visibility.  Thus, in migrating from ArcGIS Server pre-10.1 you may find that the sublayer control that was previously available by default  through the out-of-the-box .NET (Java?) web applications is missing ... halting your entire migration.

After contacting Esri, I found it was necessary to code a custom solution.  

Here is a modified Basic Viewer web template, which supports sublayers, through the layers checkbox widget (dojo dijit):

You'll need to add your webmap ID to that code, for it to function.

Or if you'd like to patch it yourself, apply this patch to /javascript/layout.js:

Note that this code will probably break if your map contains multiple map services with sublayers.

Thursday, April 11, 2013

Running GDAL and arcpy together (ArcGIS 10.1, GDAL 1.9.2, python 2.7.4)

There are many use cases for running GDAL and arcpy in the same python instance.  GDAL is the most popular open source geopspatial geoprocessing toolset and arcpy is Esri's python wrapper for the entire ArcGIS geoprocessing toolset.

However, doing so can get quite tricky.  GDAL's python bindings are not easily configured for Windows (and ArcGIS/ArcObjects is Windows-only).  The OSGeo4W project can be used to do so, but the bundled Python does not play well with arcpy.  arcpy itself depends on numpy and requires multiple path inclusions.

Here are steps I took to import gdal and arcpy.  
I tested by successfully running arcpy.RasterToNetCDF_md()  (though I haven't tested gdal yet).

Step 1: Install standalone python to C:\Python274 (download from
Step 2: Add numpy (version bundled with ArcGIS) and Desktop10.1.pth to C:\Python274\Lib\site-packages (download from
Step 3: Install GDAL binaries and python bindings (download installer from
Step 4: Run import commands "import arcpy" and "from osgeo import gdal"

Some considerations:
arcpy is 32-bit (since it wraps arcobjects), so 32-bit python (etc.) are necessary
all versions of wrappers must be compatible with the same Python interpreter