It’s All About Data

Earlier, I discussed how coordinates can be used to refer to locations on Earth. While a surprisingly hard problem, ultimately you end up with a named coordinate system (e.g., UTM10N/WGS84 or Irish Transverse Mercator) and move on to working with your data. In many cases, this means storing, maintaining, and analyzing spatial data within a database.

This leads to the question, “How are coordinate systems modeled in spatial databases?” and perhaps “Why do you care?”

(If you exclusively access your spatial data via GIS software, you might answer the second question, “I don’t!” If so, feel free to stop reading here.)

One common approach is to use the SPATIAL_REF_SYS metadata table defined by the OGC. In this scheme, each coordinate system in use is given a unique local ID (called a SRID), which in turn is mapped to a textual description (called “Well Known Text”; see here) and/or a second, centrally registered ID (see EPSG below). Then, each geometry in the database is associated with a SRID: directly, via a second metadata table, or both. (Well Known Text shows up in a number of non-database contexts as well, such as the “.prj” companion to ESRI Shapefiles.)

One of the strengths and weaknesses of Well Known Text is that it’s underspecified. On the plus side, it’s easy to extend – nothing prevents you from adding new projection algorithms, datums, or more. On the down side, not all applications understand all the same terms or refer to the same coordinate systems in the same way. Registries, such as the EPSG, are a big help as they authoritatively assign IDs to commonly used coordinate systems, and these can be referenced in the metadata tables, Well Known Text, and many other contexts. If you’re curious about the many flavors of Well Known Text or the plethora of other ways to describe coordinate systems, it’s worth a visit to spatialreference.org.

Coordinate system metadata is most commonly used by GIS software to present data in context (e.g., with units or layered over a base map), or to correctly overlay data referenced to different coordinate systems (as discussed earlier). However, some databases provide additional capabilities. For example, they might be able to reproject data natively, allowing for spatial queries that compare tables using different coordinate systems, or that transform results to a desired coordinate system for a lightweight client. Also, some databases endow a rich set of spatial operators with knowledge of our round world, answering queries like “How far is New York from Paris?” or “What’s the area of an airspace described by a latitude, longitude, and a radius?” while taking into account a coordinate system’s model of the Earth’s shape.

Learning how your database stores and manipulates coordinate systems will help you ensure that you’re storing this metadata correctly. Further, you may discover powerful new ways to transform and ask questions of your data.

From time to time, folks have commented about Safe in the blogosphere and made the comment that Safe is something special in the industry. Wow, thanks for the kind words!

Stu’s post in particular has got me thinking that if I had to pick one thing – and only one thing – that has been responsible for our success it would have to be that we have Fun with a capital “F”.

When Dale and I started Safe in 1993 we didn’t start with a big business plan. We simply found a customer with a need and we had fun building the solution. I can still remember those early days (and nights) chatting on the phone about interfaces while we worked. I would write one part and Dale would write the other and then we would bolt it together and hey it worked. We would deliver software to our customers, and boy were we ever having fun!

Time went on and we continued to have fun, being joined by other smart people who also like to have fun. Before long we had a group of fun-loving, hard working, passionate people. The thing about fun is that the more fun people who join, the more we fun have! This fun-loving is not restricted to Safe. Look at folks like James Fee and Jason Birch. Are they having fun? One click to their blogs and the answer is obvious.

There are, however, people who do not like to have fun and hate it when other people are having “fun”. They believe that business must ONLY be serious and professional and may even try to make you feel guilty about having fun, saying that “work is not a place for fun, it is a place for work!” To them I say: What is wrong with having fun building high quality software? What is wrong with having fun making each release of FME faster? What is wrong with having fun making the next release of FME so good that the previous version seems unusable? What is wrong with giving fun, entertaining, and informative presentations that keep the audience from falling asleep?

I can assure everyone that at Safe we have “serious” and “professional” fun, all while working to make FME even more fun to use! I hope that I never work for an organization which does not enjoy having fun as once the fun leaves so does the passion, and without passion an organization is dead. Fun is the fuel for many things, like brainstorming which by definition needs to be a fun-filled crazy idea-throwing experience. It is also the fuel for excitement and passion, and can even make working hard a fun experience in and of itself. Fun and life go hand-in-hand.

What can you do to be more productive and more passionate about your job? It is as simple as having fun. At the end of the day I hope that when people think about Safe, FME, our employees and our customers that they think and feel - FUN!

————————————————————————————————————————————————————–
A Fun Example:
Here is just one fun example of how we have fun! I have put together a collection of some of the splash screens and splash screen concepts we have had over the years. How can you have fun with splash screens? Check them out and you decide.

(1) (2)

(3) (4)

(5) (6)

(Check out this page for other examples)

One final way to have fun - free stuff!
Some of these splash screens we shipped and some we did not, but they were all fun! Can you tell the difference between those we shipped and didn’t ship? Make your guess and I’ll send an FME 2010 t-shirt to one (random) person who gives the correct answers by the end of the week.

About a week ago, I came across a great article from Tim Bray, Sun’s Director of Web Technologies, on the state of corporate IT, “Doing it Wrong”. Essentially Bray describes his opinion that enterprise IT could benefit from borrowing ideas off of tech startups in terms of fast, iterative development, rather than long development times for a single release. Bray’s post, along with the dozens of excellent comments both for and against his ideas, got me thinking about where Safe Software fits into this.

On the FME Server support team, we are finding that it is becoming fairly common to use FME Server as a platform to write applications on top of, rather than only using it out of the box. One of the new capabilities in FME Server 2010 that may not have received quite as much attention as others is an enhanced FME Server Web Services API.

Hidden away in the FME Server installation directory (under Server/sdk/services/apidoc for those following along at home) there is some new documentation covering how to control many aspects of FME Server entirely through HTTP requests and get responses back in HTML, XML, or JSON.

This new documentation, along with the new XML and JSON response options should make these features much easier for customers use. This opens FME Server up to being used as a back-end in pretty much any environment. We still have the .Net, Java, C++, and SOAP APIs, but now you can easily pull FME Server’s functionality into applications written for any platform that has an HTTP library (which is pretty much everything).

For example, a common use case for writing a custom application with FME Server is a self service data download portal. Our Data Download service will run a workspace, zip up the results, and serve back the zip file. Previously writing a custom application that piggy-backed on this service was quite difficult, requiring a developer to navigate fairly complicated C++, .Net, Java, or SOAP APIs that were really designed for far more complex things that writing a front-end to our data download service just to get the URL of that zip file and give it to the user. It was technically possible to get the URL without using an official API, but it would have required manually parsing an HTML response, which is generally not a good thing. Now you can simply append “opt_responseformat=json” to an HTTP request and get a simple, easy to work with JSON response with all of the information your custom application would need.

We’re really looking forward to seeing the applications people build on top of FME Server with this new easier to use API. We’re hoping we have given our users the tools to “Do it Right” as Tim Bray would put it, and I think this will enable some amazing new applications.