Monday, December 21, 2009

Table to table bug (ArcGIS)?

Is there a bug with the "table to table" tool in ArcGIS when using it to export a DBF from an Excel file (XLS, XLSX)? I don't know. BUT, I would avoid using any Excel files in ArcGIS. This would be one of many poorly documented bugs ... support for Excel in ArcGIS is a weak link, and you really don't want to be basing multistep analysis off some poorly imported data (trust me!). So you have an XLSX file that is more than the limit of rows for XLS ... how do you get it to DBF (since Excel 2007 doesn't export DBF). If you have MS Access, I would recommend going that route. I tried this out with a file that I was getting super incorrect output on, and it worked. Seems like a nice, clean, consistent solution.

Cloud basemaps, so elusive

I wanted to use openstreetmaps as the base data layer for the new UD Campus Maps, but am running into the same labeling issues as when I use a google maps base layer ... labels are prerendered to appear on streets, cities, etc. of course not taking into account placement of "UD" layers (e.g. buildings). When the two are displayed together there is a very unsightly overlapping effect.

So the way around this is to use a custom renderer, which is actually quite a bit of work ... the custom renderer requires a large amount of space, not to mention an initial span of CPU time. It also requires that I create custom style specs, which address all sorts of issues, including a plethora of OSM tags, as well as all the details about base layer appearence ... all to be done manually of course.

... not to mention, the existing baselayers seem to be disagree with cloud services ever so slightly

CloudMade offers a very cool service for rendering custom basemaps from OSM data, all through an efficient GUI, and they even host it for you. Trouble is their terms do not seem to permit "non-personal" use, :.(

Sooo, I can:

  1. set up a custom render, which *might* require a lot of space/time, and *might* require a lot of work in symbolization, and will consume local resources instead of leveraging the cloud
  2. *or* use ArcGIS server to tilecache a national basemap ... having much the same disadvantages as above, except that I have a better idea of how much time/resources it will take
  3. *or* render on top of satellite data (which doesn't really look that good, and not very informative for directions), plus requiring me to create custom label and symbol scaling and symbology for Newark basemap details (tending to be less attractive than those already available in the cloud, and causing disuniformity with surrounding areas)
  4. *or* I can mask the extent of all "UD layers" and use this as the default extent for the map, putting in some scale sensitivity which causes UD detailed layers to disappear when viewed at a larger "regional" scale and instead showing some extent rectangle ... then showing these layers again when at a closer zoom (having some of the same drawbacks of disuniformity/attractiveness)

... as of today the last option seems the best ... I sure hope it bares fruit, we need to get this project rolling!

Wednesday, December 16, 2009

unexpected rows in tables

Occasionally, the number of rows in a table* is not really the number of rows in the table. Sometimes this peculiarity only makes itself known when you export a table and only see a subset of the total rows in the original attribute table ... where all the other rows go to?

One reason that this happens (perhaps the most common), is that when you use the "keep only matches" option when doing a join, the non-matched rows are kinda there, and kinda not. To fix this you would need to go back and use the "keep non-matched" option.

*as counted by the "Records (1 out of x)" at the bottom of an attribute table window

Thursday, October 1, 2009

license server down or not responding -96,7

was getting an error like "license server down or not responding -96,7" when trying to get ArcGIS to run on a Vista machine. Seems like the problem was that since the host name could not be set (I could not browse to the license server and therefore it could not be set through the ArcGIS Administrator dialog) the host's license file wasn't validating (although the machine I was setting up could ping the host). I got around this by setting all registry entries = "Not_Set" (actually were "@Not_Set") to the name/port of my license server

Thursday, August 27, 2009

area cropped when skinning flv to swf

try using the noscale option and enlarging the dimensions exported ... you can use negative margins on the html output to make it look decent

Monday, August 17, 2009

notepad++ insert carriage return

In notepad++ you can easily find a carriage return by copying a section of texxt with a carriage return in it and doing ctrl+f or ctrl+h to replace, but you cannot copy that block and paste into the replace box. I found the answer for this here:

In the replace area, type

\1\r\n\2 -- \1 is the first group, \r\n is a carriage return and a new
line, \2 is the second group.

Friday, July 17, 2009

connecting to sde

Connecting to my new SDE was exceedingly difficult. I'm not sure if it's because it was built on postgres, but ... geez!

Here are a few items you should be aware of
  • contrary to ESRI's documentation, you need to do a very important piece of configuration on the client, namely add a reference to the service in the services file which is found (for most Windows users) at C:\WINDOWS\system32\drivers\etc\services. At the bottom add the following line (replacing servicename with the name of your sde service): service name serviceportnumber/tcp #ArcSDE for RDBMS
  • make sure your server is listed in the pg_hba.conf file (in your postgres data folder)
  • this entry must also be in the same file on the server, and also in C:\Program Files\ArcGIS\ArcSDE\pgexe\etc\services.sde, but that's usually less of a problem since the post-install takes care of it
  • if you add your port number in the server field in the "connection properties" from your client (in my case arccatalog), it will return an error
  • make sure your desktop software is the same major version as the sde (example: 9.3 to 9.3)
hint: was getting the following error: "Failed to connect to the
specified server. Entry for SDE instance not found in service file" ... note 1 above, particularly on the client side, fixed that

Wednesday, July 15, 2009

setting percent property as3

when setting percent height property in AS3 was getting the following erro"1084: Syntax error: expecting identifier before semicolon" ... was because I need to use percentHeight, percentWidth, etc. for percentage properties ... I believe this is similar to HTML DOM

Tuesday, June 23, 2009

Importing ESRI GRID to ENVI

I'm not sure if ENVI documentation is lousy or just not searchable, but I could not find any good information on how to import ESRI GRID to ENVI ... and the ENVI UI is so unintuitive (hint to ITTVIS, why not just streamline the whole "external file" thing into the same "open image" command used to import geotiffs and the like?) as to make a simple task need explaining!


1. File > Open External File > IP Formats > ESRI GRID
2. Navigate to the hdr.adf within the folder with the same name as the vector you are trying to open and open this

Monday, June 22, 2009

"select * into" in mysql

"select * into" is not allowed in mysql. Here is an example to get around this limitation:

CREATE TABLE newtable SELECT * FROM oldtable

originally found here

Monday, June 8, 2009

find/replace text mysql

update TABLE_NAME set FIELD_NAME = replace(FIELD_NAME, ‘find this string’, ‘replace found string with this string’);


Monday, April 20, 2009

Classify Dialog does not open

Problem: Classify dialog does not open when hitting button "Classify ..." under Spatial Analyst "Reclassify" tool

Solution: Was trying to do this on an image (JPG). Must convert to a native ArcGIS grid to get the Classify dialog with histogram.

Tuesday, March 17, 2009

EveryDWG CAD Command Line

EveryDWG is a utility provided by the Open Design Alliance, a consortium of firms mostly organized around DWG and other CAD standards, for doing conversion from all versions of DWG to DXF (DXF is the more interchangeable format). I was having a hard time getting this utility to work from the command line -- whenever I'd run the command I'd getting a window giving me information about proper syntax. The only trouble is THE SYNTAX USED IN THE WINDOW IS NOT CORRECT! The window says "Command line format is: Quoted input folder Quoted Output folder", but if you actually put quotes on these the command doesn't run and you're back in the same place! Here is an example of the proper syntax:

everydwg C:\TEMP C:\TEMP\OUTPUT ACAD2000 DXF 0 1

Friday, March 13, 2009

make on windows (GNU), dmake

finally found a suitable (and open source!) make executable for windows 64x, dmake. I posted it on a serve I control, so now it'll always be available (it's GNU)

Thursday, March 12, 2009

Blocks not displaying in their regions?

Blocks not displaying in their regions? Delete the block and then create it again and add again to the appropriate region. This worked for me.

Wednesday, March 11, 2009

copy mysql tables

mysql> create database db2; \! /usr/local/mysql/bin/mysqldump -u root –password=pass db1 | /usr/local/mysql/bin/mysql -u root –password=pass db2

originally seen on:

Wednesday, February 11, 2009

wildcard regex in notepad

The general wildcard regex for notepad (simliar to % in sql) is (.*).

So to match both of these lines:

</option><option value="COM">COMPAQ ALPHA UNIX
</option><option value="DGV">DATA GENERAL AOS/VS

The regex/search value looks like:

</option><option value="(.*)">

Just don't forget to toggle search mode to "regular expression"

Monday, February 2, 2009

all changes cause syntax error, python

Problem: getting a syntax error whenever making any change to a py (python) file

Solution: stop editing py file in regular text editor, needed to edit within development environment, such as pywin

load csv to mysql from local file

LOAD DATA LOCAL INFILE '[path to your csv]'
INTO TABLE [table to import into, must be created before running with proper columns]
([field 1], [field 2], [field 3], [...]);

link to original article

removing carriage return/new line character in mysql

on importing a csv to mysql I saw these little black square characters, which were carriage return symbols of course. To remove these little artifacts run the following query:

update [table_name] set [field_name] = replace([field_name],'\r','');

terminal server has exceeded max number of allowed connections

if you get the error just run the following command to override the error:

mstsc -v: /f -console

(substitute ip for

link to original article