FME 2012 Use Case: DMTI and the FME Store
As you may very well be aware, FME2012 offers a new facility to our users and partners: the FME Store.
The FME Store is not some new technology for caching data, but is simply an online shop for buying FME add-ons*
So this post is partly to let you know about the store – which officially launches today, April 12th – but also to highlight an example of tools produced by one of our partners, DMTI Spatial.
The Store – What is it For?
The FME Store is a way to provide add-ons that extend the functionality of FME. Such a tool might work fully within FME, or interface with a web service or external program to transform data. I can even see how data providers could make datasets available for purchase in FME.
In the example below, DMTI’s transformers integrate with their online geocoding service, to allow Canadian FME users to geocode and QA their address data in a variety of ways.
The Store – What FME Tools?
The FME Store currently supports a number of FME tools:
- Workspace and Workspace Templates
- Custom Transformers
- Custom Formats
You can access store workspaces and templates from the Create Workspace dialog in Workbench. Here I’m highlighting a partner’s store (PenBay Solutions), but also note things like workspaces for training and tutorials:
Custom Transformers can be accessed from the transformers gallery (under a special section marked “FME Store Transformers” or also from any Quick Add transformer with a “download” symbol next to it. Here are transformers called MeasureInterpolator, and MeasureGenerationFromZ:
Custom Formats will be supported in FME2012-SP2 onwards. At the time of writing it’s not released, so I don’t have any screenshots, but I imagine custom formats will appear in the usual Format panel in the ‘Add Reader’ and ‘Generate Workspace’ dialogs.
The Store – Costs?
At the same time as creating the FME Store, we’ve also added a new transformer called LicenseChecker. This transformer is part of the setup that allows store items to check for a related license.
In order to break up this paragraph with a picture, I’ll remark that if the LicenseChecker were a person it might look like this:
But don’t worry! Like a German autobahn, FME has no speed limit. And in the FME Store you can download any item for free, and without registering first.
However, some items will be password-protected and require a special license to unlock their functionality. Supplying the license file is how partners will be able to charge for their products (if they wish to do so).
At the moment, all Safe Software items in the store are completely free and open and don’t need a license to run. Other items should mention in their description that a charge applies, and even then may be designed in such a way as to allow a free evaluation.
So, there are six DMTI transformers available in the FME Store:
These transformers allow FME users to leverage DMTI’s Location Hub® address recognition and geocoding web service. Running your address data through these transformers will not only geocode them, it will also validate them and standardize them to a common schema. This would certainly make the data more reliable and easier to carry out matching and change detection on.
Let’s look at three of these transformers:
This transformer lets you geocode the contents of a single attribute; i.e. where the contents of the attribute are a whole address such as “7445, 132nd Street, Surrey, BC, V3W 1J8”.
The result is a geocoded feature – with or without geometry – that includes a fully parsed set of attributes for the returned address:
For example, you get separate attributes like _municipality and _postalcode and _uaid, even though they never originally existed. So the data is cleaned and standardized, in addition to geocoded.
The “Unparsed Address” geocoder works in exactly the same way, but assumes the input is stored as a series of attributes, instead of a single one. Therefore, rather than a single parameter for Address, this transformer has separate parameters for each part of the address (“First Address Line”, “Second Address Line”, “Municipality”, etc).
The idea is that the data is fairly basic in its composition; i.e. it is much the same format that you would prompt users to enter in an online form, and has not been processed further.
The “Parsed Address” geocoder is again similar, but this time works on a more complex address definition. It has fields for Street Number, Number Suffix, Street Name, Street Type, etc.
With individual parameters for unusual address components like suffix and unit number, it is for data that has been parsed a little more than the typical online form.
These different transformers exist to minimise the pre-processing you need to do on your source data; for example there is less need to split up or concatenate attributes prior to geocoding.
The other three transformers from DMTI carry out the same functionality as these three, but are designed for working on batches of data. They have a separate parameter called “Number of Addresses Per Request”:
As you would guess, this parameter lets you cache a set of features and send them off to be processed in a single request. For a bulk process it would make more sense to process in batches, rather than fire off requests for each individual address.
To finish, here is some other basic info about these DMTI transformers:
- The area of coverage is Canada only.
- They need FME2012-SP1 or newer to work.
- The transformers can be downloaded without cost but – when run – require an account with DMTI to connect to their servers.
For more info on DMTI visit dmtispatial.com and look up their Location Hub® products.
Well, as always, I hope this post is of use. I certainly learned a lot about the DMTI capabilities from writing it. I’m most impressed by the idea of using the service to clean address data, as well as geocode it.
But the DMTI transformers are just one example of what can be achieved. If you are either a partner, reseller, consultant, or plain-old FME aficionado, and are now burning with the desire to get your work added to the FME store then simply email us at email@example.com. There’s a form to fill in and terms of service to agree to, but that’s about the only hoop you have to jump through.
In return you’ll get your own repository on the server (yes, it’s powered by FME Server of course) and an application with which to generate licenses for your products.
And if you’re a user looking to extend your FME capabilities, then just fire up FME 2012 and go for it. At the time of writing there are at least 40 Safe-provided custom transformers available to download; including ones that I authored myself such as the ParityFilter, MeasureGenerationFromZ, and the updated KMLDiagrammer.
And keep an eye out for a web page listing items in the store with – and I hope we can do this sometime in the future – the ability to provide feedback and ratings.
* I originally wrote this amazing description, which I will keep here to remind me to use shorter words in future: “it is literally an online retail facility for obtaining additional FME functionality.” The Plain English Campaign would probably give me an award for something that bad!