The summer of 2017 has been an interesting time. 
For one thing, I got into reading old Erle Stanley Gardner novels, particularly the Perry Mason detective stories. They’re pretty good books – although he’s no Raymond Chandler (who is?) – and it gave me a chance to come up with fun titles and headings for this post.
Another summer event concerned the town of Killarney, Manitoba, where I live and work. The town has a very loud siren, located downtown, that sounds out every day at 12 noon to alert everyone to the time. It’s our equivalent of the Edinburgh or Vancouver cannons.
But earlier this year I noticed it was not sirening (it that a word?) as usual. It appears that it’s been turned off except for emergency alerts! Well, I for one missed the roaring sound every midday, so I decided to make my own version using FME…
The Case of the Scheduled Twittering
I didn’t want to actually be a noise polluter myself, so the obvious thing was to create a Twitter account to announce the midday hour. Lots of inanimate objects have a Twitter account, and at least this one would have a purpose. So I hopped over to Twitter and created an account called KillarneySiren.
In FME, I then created a workspace. It looks like this:
OK, that’s certainly not many transformers. In fact it’s boringly simple. A feature is created and it triggers a tweet. The feature does have coordinates set, so the siren’s tweets are precisely geo-located (in case anyone cares). The Twitter account is – thanks to the web service capabilities in FME – authenticated and stored locally for reuse in any workspace.
All I need to do now is have it run at noon every day. That would be simple enough with an FME Server/Cloud setup, but it’s just as easy to do using a scheduled task in Windows:
With this result in Twitter:
And there we are, as simple as that. All done.
Oh! You wanted more? That’s OK, so did I…
The Case of the Weather Forecast and the HTTPCaller
As I mentioned above, besides signalling midday the Killarney siren also alerts townsfolk to emergency scenarios such as bad weather. So it made sense to expand my workspace to do something similar, like fetch a weather forecast.
I looked up weather services with an online API and among the many I eventually fixed on OpenWeatherMap as being easy and convenient. My simple URL is this:
http://api.openweathermap.org/data/2.5/forecast?q=killarney,mb&units=metric&appid=xyz
As you can see, the call to their API includes my location (killarney,mb) and the units I want the data returned in (metric). I also append my personal api key. All that is very straightforward to apply in an FME HTTPCaller transformer, and I can even use a published parameter for the id:
The response is a georeferenced point feature with some JSON containing the weather forecast, a small snippet of which looks like this:
{"weather":[{"id":800,"main":"Clear","description":"clear sky","icon":"01d"}],
"clouds":{"all":0},
"wind":{"speed":12.11,"deg":280.505},
"sys":{"pod":"d"},
"dt_txt":"2017-10-18 21:00:00"}
Here’s the workspace (click to enlarge):
A Creator transformer kicks off the process and a HTTPCaller sends my query to the API. Notice there is a bit of duplication there because I’d noticed the service sometimes fails with a 500 code. A Tester tests the server’s response (stored in the attribute _http_status_code) and if it’s not a success waits five minutes (the Decelerator) and tries again. If it fails again then the Emailer contacts me. Yes, I could have created a looping custom transformer to try every five minutes until successful, but this is fine for now. If, in the future, both attempts fail then I’ll implement the loop.
The Case of the Nested JSON
The JSONFlattener simply flattens the JSON content out into a list attribute. It’s a list attribute because the service is not just sending me a forecast for a particular time, it’s sending me forty forecasts; one forecast for every three hours for the next five days!
Here’s that section of workspace:
Notice I’ve actually got two ListExploders. The first splits the data off into a separate feature per forecast. The second splits each of those forecasts up into pieces. That’s because there seems to be the ability to have multiple forecasts for any particular datetime. I don’t think there has to be, but because FME returns a list I have to assume there could be.
I think the JSON is nested (maybe I should’ve used XML, easier to read!) into multiple layers, something like this:
{
"forecasts": {
"datetime": {
"forecasts": [
{
"weather": "rainy"
},
{
"weather": "cloudy"
}
]
}
}
}
Anyway, now I’ve split the JSON into individual forecasts and just have to pick which one I want to keep. I do that by formatting the datetime string and comparing it to today’s date:
That I should really update. In 2017.1 I should be using the DateTimeConverter, DateTimeStamper, and calculating a difference with the DateTimeCalculator. Then test for where the difference is zero. I promise I’ll update the workspace soon.
But we’re on the homeward stretch now…
The service provides an icon that matches the weather, so I fetch that with a HTTPCaller, and create/send my tweet. The Tweeter specifically allows me to upload an image file so I save the icon locally and point the Tweeter to that file.
The result looks like this:
Hurrah! We’re done. I just set up another scheduled task so at 8am each morning I tweet out what the day’s weather will be.
The Case of the Reinvented Wheel
Although the workspace has more than ten transformers, the nice thing is that none of it is complex. Some of the interesting parts are:
- The HTTP response code actually means something! Don’t ignore it.
- The Decelerator transformer is a very simple way to pause a workspace before retrying later.
- FME web services makes it easy to share the workspace but keep my Twitter and Gmail authentication details private.
- But because I don’t have OpenWeatherMap added as a web service, my api key is not secure.
- FME moves pretty fast. If you don’t stop and look at the What’s New once in a while, you might miss new datetime transformers!
- You don’t need FME Cloud/Server to schedule tasks
- But I do have to leave my computer on 24/7 to ensure the scheduled tasks work
- And if the power goes out the scheduled tasks stop until I turn my PC on again
Plus there’s always something new I learn. I thought the JSONFlattener and ListExploder were a great way to handle nested JSON, but it occurs to me now that the JSONFragmenter might do the job by itself! I shall try that.
For my final lesson… always check the FME Hub. Had I done that I would’ve found Dmitri’s OpenWeatherMap transformer! Although I didn’t spend much time on this workspace, about half of the time I did spend was “reinventing the wheel”.
The Case of the User Who Tried It For Themselves
If you want to check out my workspace, and use it as a template for your own neighborhood siren, you can find the basic Tweeter here and the Weather Tweeter here. I’d love to see other FME users around the world set up some automated tweeting. I already know of @DailySatPic and @FMEBetaBuilder (although we’re not very good at keeping that one up to date). Or maybe we can create a worldwide network of FME-powered sirens!
I’m hoping to take the KillarneySiren a step further by incorporating real-time weather alerts into the picture. That’s quite hard if I want to stick to Desktop only, but really easy if I want to incorporate FME Server. I was hoping to keep this project at a level where you could use an FME Home Use License, so I’ll have to think more about real-time.
I’m also thinking I should post to Facebook too, since I only have 4 siren followers, and one of those is me! People where I live are more into Facebook than Twitter.
Oh, I forgot to mention that you need to have at least FME2017.1.1 to run Twitter transformers. The authentication methods for Twitter changed and FME had to be updated to match. If you haven’t upgraded yet, you can download it from our web site.
Mark Ireland
Mark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)
