Importing Data Dump into Postgres

I needed to take four steps to import an Oracle binary dump into PostgreSQL

1. Open Oracle dump with something like notepad++ or grap and do a find/replace for all the binary characters (these can be highlighted in notepad++) and replace them with something like a pipe character ("|")
2. If you know the special characters that are not compatible with UTF-8, then do find/replace on these too now ... otherwise you can do this after running the copy command (step 5), since that will give an error for each non UTF-8-able line/character
3. Open the piped file in Excel and replace with tabs.  Tabs are the standard delimiter in Postgres and by saving out of Excel, you get a uniform number of columns for each record, thereby avoiding "missing data for column" when executing the "copy" command
4. Create a table in Postgres with the number of columns seen in Excel, using generic data type such as text
5. execute the copy command ... no delimiters param need be given, since it's tabs

   COPY data FROM 'C:/temp/data.txt'

ArcGIS Weighted Overlay Error #999999

Client was getting error 999999 (generic) error when running weighted overlay analysis.  Fixed this by changing Windows environment variable for temporary folders to c:\Windows\Temp from the users temporary directory, which had a complicated file name

Caching (GWC) Imagery

Issues I've run into while caching imagery on Geoserver/GWC

• Out of Memory Errors (java.lang.OutOfMemoryError: PermGen space): Solved this (as suggested here) by adding -Xms48m -Xmx256M -XX:MaxPermSize=128m to Geoserver's startup.bat/startup.sh on the line that starts the JVM (for me it was line 120)
• also found a post suggesting adding UseCompressedOops fixed some issues with caching imagery ... from

"%RUN_JAVA%" -DGEOSERVER_DATA_DIR="%GEOSERVER_DATA_DIR%" -Djava.awt.headless=true -DSTOP.PORT=8079 -DSTOP.KEY=geoserver -jar start.jar

to

"%RUN_JAVA%" -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSClassUnloadingEnabled -Xms48m -Xmx512M -XX:MaxPermSize=256m -XX:+UseCompressedOops -DGEOSERVER_DATA_DIR="%GEOSERVER_DATA_DIR%" -Djava.awt.headless=true -DSTOP.PORT=8079 -DSTOP.KEY=geoserver -jar start.jar

• ERROR [seed.MTSeeder] - Problem communicating with GeoServer : was getting this error because a DB parameter had recently changed (DB name)
• Error ... Could not list layers for this store, an error occurred retrieving them ... then something about null argument ... this was also fixed with these memory options
• I also found it necessary to scale down my imagery using the outsize parameter  with gdal translate

gdal_translate -of GTiff -co "TILED=YES" -outsize 50% 50% input-file.tif output-file.tif

• I think I may also have been having trouble not adding a world file (possibly prj) which I then created with ArcGIS (not sure how to do that with GDAL, since that saves the geographic info directly to the TIFF)

JQuery Each Iterator

Is your each iterator not working on an Array? Try constructing the array as an object instead. Jquery v1.3.2

Setting LM_LICENSE_FILE Environment Variable for ENVI/IDL 4.8 on Pre-Lion MacOS

To set an environment variable for ENVI/IDL on a pre-Lion MacOS System, I would recommend using the environment.plist file. This persists through whatever shells you have installed as well as in X Windows (and therefore when you double-click the ENVI executable from the MacOS GUI). I did not find any correct documentation for doing this from ITT or NASA.

1. Install the software
3. Define the environment variable in the file ~/.MacOSX/environment.plist
----- Open the Terminal
----- You will probably have to create the directory ... mkdir ~/.MacOSX)
----- ... and the file vi ~/.MacOSX/environment.plist

Paste the following into vi in Insert mode (press i) and save (:x) ... if you have more than one environment variable, make sure to add another key/string pair ... make sure you subsistute PORT and HOST below

<!--?xml version="1.0" encoding="UTF-8"?--><!--?xml version="1.0" encoding="UTF-8"?-->
<plist version="1.0">
<dict>
   <key>LM_LICENSE_FILE</key>
   <string>PORT@HOST</string>
</dict>
</plist><plist version="1.0"><dict></dict>
</plist>

4. Close terminal

You should now be able to use ENVI/IDL.

Openlayers Geoserver WMS Caching with GWC

Now that Geowebcache is integrated with Geoserver and WMS (among other OGC service types), map tile-caching is supposed to be really simple. I found this not to be the case. Among other challenges, document is lacking or blatantly wrong. Here are a couple key concepts that I learned the hard way:

Use the demo from the Geowebcache section (http://localhost:8080/geoserver/gwc/demo) NOT the regular OL "Layer Preview" which does not use the correct gridset parameters (nice demo, eh?)


Issues I was having with labeling and tiles are mostly solved by creating a database view from the centroids of polygons to be labeled and label these instead, and include in a layer group.

Don't use the "Recycle Tiles" option ... especially if you don't have native JAI. This causes a memory leak and subsequently causes the server to crawl/crash after X requests. Also you probably shouldn't tweak max render time/memory unless you have to (despite documentation saying that this helps performance ... would this be helpful at another time? I don't know).

Despite documentation saying to use the wms endpoint and that no special endpoint is needed ... a special endpoint is needed ... it's of this format http://host:8080/geoserver/gwc/service/wms

The gridset scheme is static and opaque, unless a custom config file is created. The documentation gives the impression that the integration of WMS/GWC creates arbitrary gridsets given wms grid request characteristics. One could could create their own geowebcache.xml (saved to GEOSERVER/webapps/WEB-INF/geowebcache.xml ... though there is some confusion about that). See more about geowebcache, the gridset schema, and geowebcache.xml here

Don't pass anything on the wms request that you don't have to! Forums and documentation say that you need to pass things like tilesOrigin, which will cause the request to fail most of the time.

GWC saves tiles to your data directory under the gwc folder. You should check here to make sure tiling is working.

Lookbehind/Lookahead Assertion with Npp Regex

Today I was looking for a way to get the last character of a pattern so that I could replace it. In this example, I wanted to find the last pipe in a record like 8|34346|. Normally with regex, one could do this using a lookbehind assertion. To do this with notepad++ you must utilize the implicit search buffer, by using a \1 in the replace field (for example, to access the first set in the buffer).

So by searching on: 1|10007|someemail@someplace.com|lastname,first name|X

"Find What:" ([0-9]*[|][0-9]*)[|]
"Replace With:" \1%

I could get: 1|10007%someemail@someplace.com|lastname,first name|X

BIG THANKS TO DEAN NAIRN!