Geographic Data in Open Format

This was one of my earlier attempts to provide geographic data in an open format. Ruby on Rails makes this one a snap.

Just for an example, let’s say you want to search for all the surveys with the word “hill” or “dale” in them. Like most websites, you can go to the home page and navigate to some specific search. Here’s the link: http://uber.summitengineer.net/survey_surveys/

Just type “hill dale” in the search box as usual, but watch down below the box, where it starts flashing blue. Notice that it’s composing a URL for that search? So, http://uber.summitengineer.net/survey_surveys/surveys/hill_dale/_/_/_/_/ is a URL for a search for “%hill%” or “%dale%” Whether you click the “Search” button or the link, the result is the same: your browser displays the results of that URL.

Notice that the results are grouped down the left side by community. There are some other conveniences – clicking a balloon on the map highlights the corresponding row in the table, and vice versa. Without that, it’s really a pain to link up the two in your mind.

So, it’s pretty easy to:

• Bookmark a search so you can rerun it whenever you like. 
• Compose a new search, even without going to the “start page”
• Read the URL and know what you searched for

Searching in a map area is almost as simple. Go back to http://uber.summitengineer.net/survey_surveys/ and click on the map: “Limits On/Off”. A search box appears, but did you also notice the URL gets composed just like for a text search? If you play with the box by dragging the red balloons, you can adjust the geographic search area. For instance, just draw the box around the word “Hudson” on the map: you’ll get a URL like http://uber.summitengineer.net/survey_surveys/surveys/_/41_221/41_243/-81_468/-81_409/ . You can see that the latitude and longitude of the corners are in the URL. If you click that link, you’ll get a map of the surveys around Hudson. (Click “Zoom Limits” to zoom in on your search box.)

Now, to this point it is just a convenient way compose, store, and communicate live searches. However, since the URL is “guessable,” you don’t have to go to the landing page at all. In fact, you don’t even need to see the HTML page we render for humans. For that matter, you really don’t need a human involved at all.

Your system could just as easily compose a search for a term, or for an area. If your system adds “.xml” to the URL like http://uber.summitengineer.net/survey_surveys/surveys/_/41_221/41_243/-81_468/-81_409.xml/ , the results come back in a machine readable format.

In the future, we’ll be adding “.shp” for a shapefile download, and “.kml” for KML. Some of our applications support “.xls,” but Internet Explorer caused an instability and we had to drop that option.

Am I buying a data silo? Ten questions you need to ask.

Let's say you're interviewing a vendor that offers Project Tracking software. Is it going to communicate with the software you already use? Here’s a list of questions that help determine whether software will “play well with others.”

1. If our Project Manager changes her last name, will your software use our Human Resources database, or will we have to enter the change every place her name occurs?
   In other words, do you even INTEND for your software to interact with other software?
2. What is the name of the table with our basic project data? Hint: if the answer is anything but "projects," watch out.
3. What is the name of the column that holds our latitude data? Again, any answer but "latitude" should make the hairs on the back of your neck stand up. Keep checking.
4. How do your web pages perform in http://validator.w3.org/ or some other html/css validator? "What web pages?" -- Wrong answer. People expect to “mash up” our data as needed. In fact, we do, too. 
5. Would you please show me how easy it is to make a Google map of the projects? Any data that has a location on the earth needs to be mappable in the open-API mapping services.
6.  What would be the URL of a typical project's HTML representation? Good answers look something like, 'projects/1234' or 'projects/show/1234' or 'projects/59th-street-bridge' -- something readable, anyway. Bad answers like "script.nnn?object=project&id=234x_guid_lsdlkjfll0980089" aren't going to make you happy for very long.
7. How do we add a field to our 'projects' data table? Obviously, “we don’t recommend that” is the wrong answer. If the answer doesn't mention a common database manager like pgAdmin or MySQL GUI tools, then you'd better thoroughly review what's possible inside the program. 
8. Would you show me how you structure your HTML? Your CSS? Your JS? Even if the language of the program is hidden (Closed Source), the outputs should be in well-structured open formats, so that you can use them elsewhere.
9. What language and what other software (typically a database) does your software use? If it uses off-the-shelf, but obsolete technologies, even the vendor can’t hire programmers to make changes when needed.
10. How do I make an ad-hoc query against my data? Again, if you can't use common database management tools, you need to be very specific in your questions. "Canned" queries never cover all the questions people dream up.
11. (Bonus) Show me your source code (if it’s not proprietary.) Some people write good maintainable code, others obscure their code intentionally with weird names and spaghetti-logic. It runs, but no one else can read it. If you need a change, or a fix, you can’t write it yourself. You’ll have to pay for the next upgrade, and hope it's included.