The Safe Software Blog
Author:
Erin Lemky

Google
Get the Blog Newsletter

Delivered by FeedBurner

About Our Customers    |   November 24, 2016   |   By Erin Lemky

3 Inspiring Earth Observation Apps Built Using FME Cloud

EO AppsSupported by the UK Space Agency, Sterling Geo have been embarking on an innovative project to demonstrate the practical value of earth observation data. Sterling Geo used FME Cloud to quickly create apps that turn raw data into useful intelligence. Their project has uncovered a way to rapidly deploy purpose-built applications without requiring capital investment or major IT infrastructure changes.

Awarded a grant by the UK Space Agency under its Space for Smarter Government Programme, Sterling Geo needed to deliver between 1 and 100 applications in one year, to 1-100 public sector and government organizations, demonstrating the value of earth observation data derived from satellites such as Landsat and the Sentinel series.

“There’s an enormous opportunity to evolve governments, educational institutes, and charities by translating that data into something they may need,” Sterling Geo’s Andy Wells explained in an interview on the Amazon blog this summer.

The aim of the project, named 100:1, was to create a pragmatic, sustainable and affordable approach to the long-term use of earth observation data. Their objectives were to deliver services, not technology, around the needs of the clients, without capital investment, training, or major IT infrastructure changes.

“The challenge that Sterling Geo faced was standing up as many applications as possible in a very short period of time, without a pool of developers to create functionality,” explains Gavin Park of Sterling Geo.

“We chose FME to create APIs and rapidly deliver web services to take this multi-temporal data and summarize it in a web client, without writing code.”

They deployed FME in the cloud (FME Cloud), meeting the satellite data where it’s housed, on AWS under Amazon’s Open Data Program. This allowed the powerful server environment to be stood up and used for the duration of prototyping, and only paid for during demonstrations. It also enabled no infrastructure changes upon deployment into production.

Equipped with the technology, Sterling Geo set out to identify scenarios where earth observation data could solve the problems that government agencies were facing. Here are three inspiring examples.

Use Case: Automated Vegetation Health Monitoring

The most popular use case by far, with nearly 80 local authorities and government departments expressing interest, was related to satellites’ ability to pick up the near-infrared light that plants give off when they photosynthesize. Ongoing analysis of this data could provide monitoring of plant health and quantity of greenery in cities.

Automated Vegetation Health Monitoring Screenshot

The Automated Vegetation Health Monitoring app provides a web client showing statistics summarized in graphs and on a web map. Users can choose to be notified via email as new information becomes available.

Triaging Relevant Imagery Updates

Gavin’s team set out to provide this information in a web service. Because satellite data is continuously being gathered, they didn’t want users to be inundated with data. Instead, they designed the service to only notify users when new information, not data, became available.

They started by creating an FME workspace that uses the FME Server Publication Service to listen to the SNS/SQS message queue for earth observation data to become available on Amazon S3 buckets.

For each scene that is added, FME checks the new data against spatial and non spatial business rules, such as target area and cloud coverage, to filter out only the data that is of interest.

Converting Imagery Into Information

FME then provides this data to Hexagon Geospatial Spatial Modeller, which performs that NDVI calculation on the satellite imagery and uploads the resulting image to S3. There, FME picks the data up again, clips the NDVI image maps to the relevant administrative boundary, classifies the data, turns it into vectors, and calculates relevant statistics which are then output to PostGIS on RDS as summaries and linked back to the core geography.

Automated Vegetation Monitoring Diagram

Delivering Data to the Web Map

They then used FME to create web services that feed this information from the PostgreSQL database into the web client. An FME service provides the content via GeoJSON, and the graphs are powered by FME workspaces that write out JSON content. Security for the source dataset is provided by Lambda. The user is logged in via AWS authentication, and associated with a geography which pulls the appropriate URL for their region via GeoJSON. The HTML code is stored on an Amazon S3 bucket.

Automated Vegetation Monitoring Architecture

Now that the data has been converted into relevant information, users are notified via an email notification.

Use Case: Slack & Email Data Delivery

Several government agencies were interested in being notified when new satellite imagery was available for their region of interest, if it met their specific criteria.

This functionality is especially useful when users want to see the raw data imagery, but don’t understand how satellite images are collected, nor how to do a catalogue search; they just want to download the image scene. In the UK especially, where it is often cloudy, this kind of email notification is helpful, as users are only notified when relevant data is available with limited to no cloud cover.

“FME’s ability to send notifications via email and Slack was invaluable, as we could disseminate the information to teams as soon as it was available,” Gavin describes.

“We used FME to dynamically generate HTML content and include this in an email to be sent out. Operating in the cloud, it was possible to embed the download links directly from S3 into this email as well, eliminating any additional data storage costs.”

“FME was so fast at processing this data, that the users were receiving the email from FME before the data was available in the quick looks being generated on Amazon for satellite imagery!” says Gavin. “We had to slow it down with a Decelerator transformer.”

Sentinel Workspace Screenshot

FME was used to monitor newly available Sentinel data and test is against business rules, notifying users when the information they need is updated.

“Not only do users get timely notifications of data, but the ability to integrate with Slack means that these updates even work with wearable technology, so you can get notifications of new scenes pushed to your iWatch when you’re at the bar or up a mountain!”

Use Case: Fire-Related Building Development Status

The Fire and Rescue Service wanted to verify the data they have regarding new builds’ development status by comparing it to satellite imagery. An app was built to check the available data for addresses, status such as in development, complete, and planning permission granted, and allow users to cross-reference this with aerial imagery or high resolution satellite imagery to see what’s actually happening on the ground.

Specifically, FME is used to extract this data from the Ordnance Survey AddressBase product, and then to scrape a housing development website for all locations of new builds. The FME workflow then calculates the statistics for the properties of each ward, and assigns hex colour codes to each property based on its status (% of new or planned builds).

Fire and Rescue Service Building Development Status

Housing developments in Kent county are flagged for exploration by fire services.

The FME workflow outputs the resulting data as JSON to be easily be consumed by the web page and the chart API. This allows a spatial dashboard view to be created, displaying areas of new development which can be easily explored.

The dashboard is displayed against the imagery layer, providing context and reference. It also allows the fire and rescue service to visually identify prospective nearby water sources in case of emergency.

As an initial proof of concept, the app was designed to allow for expansion, so the concept could be transferable to other regions and data requirements. Therefore the JSON was written to S3 and stored next to the HTML code, and various layers and satellite imagery can be swapped out or added.

A Successful Prototype

The 100:1 project has certainly achieved what it was commissioned to do.

“The apps have enabled users first hand to understand how long term earth observation monitoring could feed into strategic planning cycles,” says Gavin.

Gavin and his team have worked with over 150 government agencies to identify and build approximately 20 apps to leverage this data so far. Many of these apps are now being used in production. The principles and technologies leveraged in the above use cases have been applied to many apps, and have provided a basis for app templates both inside and outside the use of earth observation data.

The 100:1 project has also been successful in identifying use cases that would be better suited to other data sources. For example, Sterling Geo assisted UK National Parks with applying satellite data to verify tourist caravan site usage against the licenses that had been purchased, by identifying the number of pitches per site. After investigation, photography proved to be a better data source for this use case than satellite imagery.

Sterling Geo and UK National Parks did, however, identify seven ways the government organization could successfully use Sentinel data. For example, they created a service that compares the planned schedule of heather burns to the real world using satellite imagery after the burn service. This has generated significant interest due to the differences being clearly visible, enabling park rangers to prioritize ground investigation to areas where burning appears to have occurred well outside the permitted areas.

“Whilst you’re monitoring your region for loss of greenspace, you’re also monitoring the protected areas, the watershed, legal, and billings. It’s the same data, different processes, and different outcomes. What a fantastic way to save money.”

The Sterling Geo team also uncovered another savings, made available by designing apps that monitor the data for you.

“Everyone has to monitor the world for change. But 9/10ths of the time nothing has changed, and the cost was the same for finding nothing,” explains Gavin. “These apps can now monitor this — it’s a platform looking for xyz with little or no capital investment, on a revenue shared platform.”

“Once the app is online, it is listening for new data, which when available is checked for relevant information, and only if it is found are users notified. They are provided information, not just data, that may affect their policies moving forward.”

Gavin’s colleague, Phil Cooper, adds, “Sterling Geo’s ability to drive value from earth observation data using the power of integrated operators is an illustration of how far Sterling Geo have been able to move in such a short time, and where we can see the power of GeoEngines performing in the future.”