In 1988, my 3-year-old cousin went missing from his backyard. On a property in rural Vancouver, the home backed onto a complex landscape of hills, holes, dense Pacific Northwest rainforest, and a raging river. The police came with sniffer dogs and the entire community banded together to help search. They couldn’t find him. Had he fallen into the river? Did someone take him?
Several hours later, he turned up on his own, the neighbour’s dog shepherding him. Both were soaking wet. We suspect he fell in the river and the dog pulled him out. Three cheers for dogs, but that’s beside the point. The point is that nobody was able to find him. Police, sniffer dogs, and an entire community were unable to locate this missing child. The forest was too dense, the landscape too extreme for a traditional search and rescue to succeed.
How might that search have changed if they’d had a more effective means of combing the landscape? What if they had an airborne device — something smaller and more maneuverable than a helicopter — that could scan every hole, tree, and chasm?
Let’s talk about drones.
Today, search and rescue efforts are just one area where drones are proving to be a revolutionary tool. Many markets and industries are also using them to monitor areas, gain awareness of landscapes and assets, and ultimately make better data-based decisions (e.g. pipeline monitoring, assessing storm damage, agriculture).
Drones are becoming a safer, more efficient option for search and rescue efforts. [image]
- Drones are effective new platforms for collecting great quality data in a safe, affordable way.
Okay. Let’s agree that drones are really cool. Ready to get started? Maybe you’re part of a business using them commercially, or maybe you’re considering getting one for recreational use (oh hey, you qualify for a free FME home use license).
Challenges with drone data
Your first step is to get a drone (aka UAV, UAS). You can buy them all over the place, great ones priced around $1000 and up.
I’ll wait.
Got one? All right, now let’s talk about data. Drone data includes imagery, video, laser scans, telemetry files, and more, often in very high volumes. This means challenges around data organization, processing, and distribution.
Further, a new area of technology like this means lots of new data, and probably entirely new types and formats to work with. This can pose serious interoperability problems, especially when you go to map all the location data the drone has collected.
Drones are a new frontier for FME, and one we’re very excited about. FME was built for interoperability challenges like this. It’s meant for moving and shaping data, can easily connect to APIs, and works well with the cloud — which means huge potential for every stage of the drone mission cycle.
Let’s look at what goes into a drone mission and our recommended steps for each phase.
1. Pre-flight planning
Start by planning the drone mission.
Look at regulations on airspaces and airports to ensure you comply. Airport/airspace data is likely in AIXM format, and it’s helpful to convert this to KML to visualize it in Google Earth. Before flying, you’ll need to consider DROTAMs, weather and wind information, proximity to restricted airspace, forest fires, and general aviation activity like paragliding/parachuting.
Visualizing airports in Google Earth – AIXM to KML translation via FME.
Generate the route/waypoints automatically. With an app/website like Litchi, you can plot your points on a map and export the mission as a CSV file. Here, you might consider creating a workflow that generates the points automatically (e.g. calculates pipeline centrelines) or converts a path from Google Earth (kml) to Litchi (csv).
Integrate various formats to get the most accurate world representation possible. Consider real-time maps, CAD building structures, GIS maps of neighbourhoods, and databases for infrastructure.
Preview the flight. Getting a realistic flight preview is key. Blend your data sources and convert to KML to see what the flight will look like in Google Earth. For commercial applications, you could build a web service that shows the flight preview in a web browser. Users could upload their flight path to the web service and get it automatically validated before seeing it on a map.
Check the flight path both visually and automatically to identify risks and errors. The automated check should perform various tests on the flight plan, e.g. what airspaces do you cross into and do you need permits.
Generate a report from the validation results. For commercial applications, you could again build a web service that allows users to upload their flight plan and automatically generate a report.
Automated flight planning viewed in the Litchi web interface, generated via FME
2. Flight
Launch the drone and wait for it to go collect data.
Make the flight mission automated, because automation means less human error. This means more efficient use of limited flight time and safer operations, since automated systems can respond to changes in environmental conditions faster than a human can. Automation also means multiple missions can be quickly generated and flown back-to-back rather than having to plan and fly each waypoint manually.
Drones store a lot of information: flight logs, rasters, point clouds, video, video sidecar files (SRT format), all with embedded lat-long values. Every drone collects data a little differently, but you’ll most commonly see image metadata stored as JPEG EXIF or NFO.
High data volumes can be a challenge at various stages of the processing workflow, so you’ll want to put some thought into how best to collect the “MYO” (minimum yet optimum) number of pictures during the flight.
3. Post-flight review
When the drone comes back, you’ll then take out the memory card and copy all the data to your computer. Depending on the complexity of your mission, you might have thousands of pictures, videos, and laser scans to deal with.
Post-processing might include detecting changes, classifying, rendering point clouds, adding z values, or any number of other tasks. It really depends on what you plan to do with the data.
Create a catalog to index all of this data. An Excel spreadsheet is one way to do this, but you’ll probably find a friendlier interface for the end user if you use HTML.
Convert the photos to a useful output, like Google Earth. For this, your workflow would include extracting lat/long from the JPEGs, georeferencing the image, and loading it to KML. Other format conversions and georeferencing will probably be necessary for whatever data comes back, like the flight path info stored as SRT.
Leverage automation. This is key. Again, the huge volume of data is potentially the bottleneck, so make sure you’ve optimized your drone-to-disk workflow for repeatability.
Automated photo georeferencing via FME, viewed in Google Earth
4. Data distribution
Delivering the data involves converting it to formats usable by your target audience.
Put the data in the cloud so others can download the mission results.
Again, commercial applications might benefit from a web service that lets the user download any part of the results, a report/summary, or the flight path for use with their own drone. You might also consider using an app like Esri Survey123 to let users update the master database with new data from their mission.
Create a report. Excel spreadsheets, HTML, PDF, business intelligence software, whatever you prefer for the format (see also: data reporting webinar). Automating the summary and distribution process is important for getting the results to decision-makers quickly.
Consider how you can integrate with tools and drone APIs, here or at any stage of the drone mission cycle. Check out DJI, OpenDroneMap, DroneDeploy, Drone2Map for ArcGIS. Apps commonly write data as XML, CSV, and JSON. You could also leverage cognitive services like Microsoft, Google, or Watson — for example, to automatically guess what the photo contains.
UAVs in the real world
For a more technical look at this lifecycle and a detailed real-world scenario, watch the FME UC 2017 presentation by our drone experts, Dean Hintz and Steve MacCabe, and Renato Salvaleon of Alabama Power Company:
- Watch recording: UAVs and FME: Powering Your Drone and Its Data with FME
Hans van der Maarel’s FME UC presentation also looks at drones in the real world, with a focus on post-processing steps. His FME workflow generates PDF reports from point clouds.
*
Data preparation, integration, visualization, automation, and distribution are all part of the drone mission cycle, and it so happens that FME was designed for all of this. We see huge potential and can’t wait to hear what FME users accomplish.
If you’re a drone enthusiast, whether commercially or recreationally, let us know what you need! New format support? Specific functions or transformations? A web service? Automated flight path generation? We want to know about your drone challenges and how you’re integrating with drone web services. Leave a comment or send me a note.
Episode 5: Coders on Couches Drinking Coffee
I chatted with our Safe Software drone experts Dean and Steve for an episode of Coders on Couches Drinking Coffee. Watch it here.
Tiana Warner
Tiana is a Senior Marketing Specialist at Safe Software. Her background in computer programming and creative hobbies led her to be one of the main producers of creative content for Safe Software. Tiana spends her free time writing fantasy novels, riding her horse, and exploring nature with her rescue pup, Joey.
