It might not make a thrilling episode of the X-Files, but at first glance, HTML must be the strangest format that FME supports.
It isn’t a spatial format, but it doesn’t have any of the usual features of a tabular (non-spatial) dataset.
It can’t store data like a database, it isn’t a spreadsheet like Excel or Google Sheets, and it’s designed to present content, not store it like XML.
It’s not even structured like CSV or CAT, in a way that you can read the contents back that easily.
So let’s see why this format is so important and – better still – how you can use it to spectacular effect!
Why HTML?
To me, HTML is a hugely important new feature for FME. It’s one of the final pieces in the jigsaw of a complete, integrated corporate information system.
For instance, there’s a reason a new transformer for creating HTML is called the HTMLReportGenerator: it’s because we’re dealing with information, not data.
With these tools we go beyond simple data translation and transformation, higher than Server-based processes, and above the notion of notifications and real-time. HTML allows us to carry out all of these tasks as before, but communicate the results in a way that is dynamic, interactive and convenient.
Imagine – at the click of a button – being able to run a workspace that reads data; integrates it; merges, splits, and concatenates it; generates statistics; and – finally – presents a series of charts, maps, and tables that report on the results.
Maybe the report includes links to download the raw source data? With this new capability, almost anything is possible!
What HTML?
Technically you could have written HTML with FME for a long while now. We even had a training exercise that demonstrated the technique. But it required manual creation of the HTML – the FME equivalent of writing your web site in a vi editor – or making use of the intimidating XMLTemplater transformer. Plus there wasn’t a dedicated HTML format writer.
With FME2016.1 we have new tools specific to HTML.
The HTMLReportGenerator transformer is a tool for constructing HTML pages from incoming features. With this it’s possible to generate basic HTML (headers, text, images) but also all sort of reporting diagrams such as tables, pie charts, and bar graphs; plus it can create an interactive map to display the spatial features.
Each content type has its own set of parameters, like so:
…and the transformer as a whole has a group-by option so that you can create a report per group of features.
The HTMLLayouter transformer enhances the HTMLReportGenerator by combining all of the incoming reports into a single HTML feature, with each report laid out in either a vertical or grid arrangement:
Once that is done, the feature is routed into the new HTML format writer.
But! This new functionality – besides being great by itself – integrates very nicely with a lot of other FME tools.
For example, I can use a StatisticsCalculator to calculate figures to add to a HTML pie chart. I could use a PointOnAreaOverlayer to find all addresses that fall inside a certain zone, and write those addresses out as a table. I might even use the GeometryValidator to test my spatial features and write a report on which features failed, and why.
And it’s not just long-existing functionality; there are numerous new tools in 2016/2016.1 that integrate very nicely with HTML reporting.
2016.1 introduced full support for web service connections and those can be used to retrieve data from a web service for integration into a HTML 
report. There’s also a new RCaller transformer, enabling you to compute more powerful and complex statistics.
The FeatureWriter transformer writing HTML is a great combination too, because you can write HTML reports and then do a follow-up action such as sending an email (with the new Emailer transformer) or moving them to a new location (maybe with the DropboxCaller). And the FeatureWriter also let’s you integrate with many other third-party tools and extract data from them.
And finally, you might be thinking that HTML’s usefulness is strictly limited to FME Server, but that’s not true. Yes, this is a great tool for FME Server because it integrates so well with the Data Streaming service to deliver reports on demand. But, even when reports are generated in FME Desktop, there are still ways to deploy them online, as the following example shows…
How HTML?
To dive a little deeper into how HTML functionality works, let’s look at an example demonstrated during the last FME World Tour.
This workspace (click to enlarge) has a few more than 10 transformers (sorry) but it does include some truly remarkable functionality:
The first section reads Air Quality information from cities around the world and selects the closest one to a chosen World Tour event. The second section generates statistics about the air quality.
The next two sections are the important ones for us. The third section generates a series of HTML reports. Rather than one HTMLReportGenerator, there are three. That’s because each report has a different group-by option set. If that wasn’t necessary then all the information could be created using a single transformer. The HTMLLayouter is used to merge those reports, using a display order set by the Sorter transformer.
The importance of the Sorter transformer is worth highlighting. When you want to create a multi-report document, where the order of the reports is important, then the Sorter is the tool to use to ensure that order.
Finally, the FeatureWriter transformer is used to write the data and the S3Uploader transformer is used to publish it to an Amazon S3 store. This is how FME Desktop can deploy reports online. S3 storage can be accessed as if it were a static web site, so by pushing the HTML reports into a properly configured S3 bucket, we’re creating a reporting web site that anyone can access.
The final report? Well click here to see what it looks like. I replaced the S3Uploader with a DropboxCaller, so that’s actually a link on Dropbox, rather than S3 – but the effect is (hopefully) the same: a report containing a bar graph, summary statistics, a list of sample sites, and a map of the area (I replaced the original OSM map with Google).
If you want the original workspace, please do download this copy of it, being aware that the keys for S3 have been removed, so it will work only up until that part of the process.
Who HTML?
So who might use this HTML functionality? Well, us of course. We love to “eat our own dogfood”.
If you use FME Server you may well have noticed the new dashboards in 2016, that provide reports on FME performance and usage. For example, here (click to enlarge) is a graph of jobs per day:
How do you think that graph was created? With the new HTMLReportGenerator transformer and HTML writer. You can download the workspaces involved from the Dashboards repository of FME Server and see for yourself:
The transformers following the HTMLReportGenerator are a good example of how the HTML can be adjusted and altered after it is created. For example, the AttributeCreator is used to create a style in CSS and the following StringReplacer merges it into the HTML header. The HTMLLayouter isn’t necessary because only a single report is being created.
Apart from us, this is a great set of tools to use for anyone who wants to dynamically create web-based reports, especially those centred on spatial data. Hopefully you’ll have seen the potential from the above examples.
Summary
As with many new FME features, I suspect this will be functionality that is built upon and improved over time. I can already see a lot of fixes and enhancements targeted for 2017, including a HTML table reader that is already part of the 2017 beta.
I’m so convinced this is going to be big, that I will try and post a few smaller articles (maybe here on the blog, maybe on the knowledge centre) showcasing some other uses for the HTMLReporter. So keep an eye out for those.
Regards
Mark Ireland
Mark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)
