It’s been over a year and a bit since I last talked about Fusion Tables, and Google has continued to emphasize it as their solution for hosting user maps and tabular data. So, I’m pretty excited that our recently released FME 2012 can get data into and out of Fusion tables in a ton of interesting ways.
What’s Fusion Tables?
It’s a way to share tabular data, visualize it, and then build on it collaboratively. For example, if you have a spreadsheet with locations inside (addresses, place names, etc.), you can turn it into a map and then share it with a link or embed it in a web page, all in a few minutes. Updates are reflected as they’re made, so the map is never stale.
Similarly, there are simple tools to extend or combine other people’s data (which is tagged with refreshingly minimal metadata – a name, description, attribution, and link); the combined results stay up-to-date too.
What’s changed with Fusion Tables since my last post?
Some things have stayed the same. You can still add and share your own places and shapes in Google Maps, but this feature is now called “My Places” instead of “My Maps”. As mentioned above, Fusion Tables is evolving and getting more traction. For example, tables are now included in Google Docs, there are new styling options, and the everything-can-be-commented-on part of the collaboration model is being rethought.
One interesting development is the introduction of Google Earth Builder, which tackles some of the same hosting-user-data challenges but with an enterprise focus and a much bigger scale. The fact that Earth Builder leverages Fusion Tables internally as a key part of its architecture (discussed earlier) is a good sign that Fusion Tables will be here for some time.
What does FME bring to the table?
FME complements the strengths of Fusion Tables by making it easy to load data from any format, data model, or coordinate system. In my earlier post, I used FME to convert some sample data to KML, which I then imported into Fusion Tables; with FME 2012, this can be done in a single step. Similarly, you can export a table to any of FME’s supported formats, transforming the data as needed along the way.
Updates and deletes are supported too, but these are only suitable for small numbers of rows at a time. This is because Fusion Tables can insert several hundred rows per request, but only update or delete a single row (or delete all rows) in one request.
In addition to making it easy to get your data into and out of Fusion Tables, FME also enables new workflows. For example, Don has an example (15 minute video) showing how to configure FME Server to dynamically update a Fusion Table based on sensor data pushed to it in real time. The possibilities are endless!
Here’s a quick video that I put together which gives a taste of what’s possible.
Want to learn more?
If you’re interested in making maps with Fusion Tables, the Wisconsin State Cartographer’s Office has published a fantastic guide. Interestingly, they also note the desire to import other kinds of data like Shapefiles, and discuss workarounds similar to those in my earlier post (convert to KML or use a web service).
If you’re interested in using FME with Fusion Tables, you’ll find some preliminary information on our FMEpedia page, and I encourage you to sign up for our webinar on the topic.
If you’ve used or thought about using Google Fusion Tables, what has your experience been?
One year ago on this blog I wrote about putting LiDAR in its place with FME 2011. One year (and a whole bunch of hard work) later we’re back with FME 2012, and we’ve got LiDAR files everywhere scurrying for cover. They aren’t going to know what hit them.
In the past year we’ve been repeatedly impressed with what we’ve seen our users do with the point cloud and LiDAR support that was part of FME 2011. But as much as we’ve been impressed, we have also been challenged with the increasing data volumes and scenarios our users put before us. FME 2012 is our answer.
A Billion Point LAS File? No problem.
The first thing we did was address performance in the case of astronomically large input. So if you were using FME 2011 to clip or tile your billion point LAS files, you are going to be very pleasantly surprised when you try FME 2012. To put it in perspective, clipping or tiling moderately sized LAS files on a standard computer is about 4-5x faster in FME 2012 as compared to FME 2011 (which was already no slouch at this).
Support for LAZ – Compressed LAS Files
For users looking to archive or exchange LiDAR data, the incorporation of Martin Isenburg’s most excellent LASzip compression technology allows FME 2012 to really cut these files down to size. Thanks Martin for making this available for us all to use! We typically see LAZsip’ed LiDAR files being 30% the size the original LAS, and others have reported reductions down to 10% of original. If you have the need to archive or share any amount of LiDAR data, these space reductions can really begin to add up and save significant storage space. All you need to do to use this is flip on the “compression” setting of the LAS writer in FME 2012, and you’re golden. The standard LAS reader in FME 2012 can just read these as if they were LAS files to begin with.
Geocentric Coordinate Systems
Paul wrote about the importance of Geocentric Coordinate Systems a while back, and FME 2012 is our first release with this support. So bring on those files that store their position as a distance from the center of the earth, and we can turn them into something a bit easier to deal with on old terra firma. We can also go back the other way, taking projected or geodetic coordinates and making geocentric, which in conjunction with 3D output formats like PDF can cast your data in a whole new light.
Oracle Spatial Point Clouds
While we’ve partially supported using FME to load Oracle Spatial Point Clouds for some time now, in FME 2012 we’ve taken it to a whole new level. Got a billion points you need loaded? We’ll load it and block it for you in under 18 hours (on a 3 year old desktop computer, on less). So if you are doing a lot of Oracle Spatial work with point clouds, you really need to check that out.
Support for .lasd (LAS Dataset) in ArcGIS 10.01
Looking forward to ArcGIS 10.1’s support of LAS files? But not so much the need to have a .lasd (LAS Dataset) file first? Don’t worry, FME 2012 is ready when ArcGIS 10.1 is and will create .lasd files at the same time as it writes out LAS files from whatever input source. Just make sure you have ArcGIS 10.1 installed, flip on the setting, and go straight into ArcCatalog to start using the results.
LiDAR Ninja Tricks – Colorizing Point Clouds
My personal favorite LiDAR Ninja trick is the ability to take a point cloud that has no color information on it, and overlay it into an orthophoto to produce a colorized point cloud. As more software becomes available for visualizing massive point clouds, the volumes of data collected without color can be made that much more interesting and useful by colorizing them.
We’re really looking forward to meeting you and showing off the point cloud prowess of FME 2012 on this year’s FME World Tour, so check out the dates and sign up if we’re coming to your neighbourhood. We’ve also got an FME & LiDAR focused webinar coming up in March (actually, two in one day – if you like the first showing you can tune in again for the second!) where we’ll show you how to do all this and more. So check one or more of these opportunites out – and I’m sure you’ll agree that using FME to tame the LiDAR beast is fast, productive, and, dare I say, fun.
We wouldn’t endorse using FME 2012 to colorize the Three Stooges, but if you had a point cloud and an ortho, we wouldn’t stop you either.
It is with great excitement that we release FME 2012 today! With this release we have taken our spatial data transformation technology to the next level. If you have data to move, more than ever FME 2012 is the tool you need to power the flow of spatial data.
While our website has full details and we’ll be sharing more on the blog in the next 2 weeks, here are a few of my own personal highlights from FME 2012.
- Ease of Use
At Safe there is nothing more important than user experience! With each release we strive to make all FME interfaces easier to use. While a technical tool we understand that most of our users are not programmers and we don’t fantasize about turning our users into programmers. Rather we work hard to remove the need for a user to write code in any language. This drive continues to make the FME environment more productive with each release.For our users who are programmers we also work hard to make them more productive. For example, with FME 2012 our Python technology has taken a huge step forward.
- Performance
As you know if a new release is not faster than a previous release then we won’t ship it. It goes without saying that FME 2012 is again the fastest FME ever. Taking this to a new extreme we have added some “multi-processing” capabilities in FME 2012 so that users are able to take advantage of multi-core processing. While we don’t count this when comparing performance with FME 2011, you will find that turning this on can make a huge additional performance boost in certain situations. - Real-Time Data Moving
With FME 2012 we have built a new “real-time” architecture for FME Server making it the perfect tool for moving real-time data wherever it comes from. So whether you have real-time data being generated by a web application or a sensor network, FME in 2012 is ready to move that data to wherever you need it. Stewart and I will be writing more about this in upcoming blog posts, but in the meantime check out this video, or sign up our upcoming webinar on FME Server 2012. - Better Integration, Data Push and Notification Services
Integrating FME Server with other Web services is now easier than ever. Now with FME 2012 you can invoke FME Server data transformation capabilities through a simple “http post” mechanism.Want FME Server to consume data on the web? You can now do this by simply sending it the URL of the data. No more need to first download the data to your desktop and then upload the data to the server for transformation.
With FME 2012 we’ve added a whole new way of getting you the data you need, with our new “data push” capability. Now you can let FME Server do the work and send you the data that is of interest to you when the data is available.
We’ve also add an amazingly powerful Event Processing service which provides new ways for FME Server to power the flow of spatial data. You will be hearing more about this in the weeks ahead too.
I am very excited about all of these additions and look forward to seeing how they’ll be used.
- User Community
Recognizing that there is more to a release than product upgrades we are also working to extend our user community experience with FME 2012. In the year ahead look forward to more informative videos, webinars, online training, and a whole new user group experience that puts our users in closer contact with us and with each other. - FME Store
The FME Store is one part of this user community improvement enabling partners to publish and sell (or share) custom transformers and workspace solutions that they have developed. For FME 2012 this is just one step in helping our users share with other users. - XML (and JSON)
No release would be complete without us reducing the effort required when reading or writing XML (and JSON). With this release the XML challenge no longer has anything to do with the XML encoding and is now strictly a data modeling exercise. You will also notice that every dialog has been completely redone making them simpler, and even easier to understand.
Your Feedback on FME 2012?
I look forward to getting your feedback on FME 2012. Let me know what you like, don’t like and what you would like to see moving forward. For a full description of everything FME 2012 see the official announcement and details on what’s new and great for both FME Desktop and FME Server.
Whether we meet online or in person, I look forward to engaging with all of you.
FME 2012 World Tour and FME 2012 Contest
Later this year we will be traveling across the globe as part of the FME 2012 World Tour. Check out the 30+ dates and locations here – there’s also going to be a live stream. Also, until January 27th you can enter to win an adventure trip or other prizes like an iPad2 or PS3 by testing your knowledge of FME 2012 – you can enter here.
