Website Automation with Salesforce, Amazon, MySQL and FME

Dogfooding is an actual term.

“Eating your own dog food” may be a more familiar phrase, but this business-speak shorthand version strikes my funny bone. One rumor has it that it originated with the president of Kal Kan Pet Food, eating a can of dog food at shareholders’ meetings –but in any case, it’s come to mean using your own products in your company’s operations.

Our very own Michael Ha, Web Developer/Designer here at Safe (and the invaluable technical go-to guy behind this blog and Safe’s website) eats our own dog food every day.

And he likes it – a lot.

Salesforce to Amazon/MySQL

Our website lives on Amazon. And like many companies these days, we’ve chosen the cloud platform of Salesforce.com for most of our business data. That’s where we store not only our customer information, but our FME training calendar details, partner information, and FME Certified Professional and Trainer credentials too – and all of those are important resources on our website.

Now, we could link directly to that data in Salesforce, but adding a jump from cloud to cloud plus constant query refreshes would likely add more of a delay to viewing those pages than we – and you – would like.

Michael’s solution is a simple one – create a workspace (that only has one reader, one writer, and zero transformers) that FME Server executes at a fixed interval using the Scheduler.

The reader queries Salesforce for records matching “Training”, and loads the results directly into a MySQL table on Amazon. Login credentials are stored securely, and our Training Calendar gets automatically updated on the hour. If the information was more time sensitive, we could update it as often as we like by merely tweaking the Scheduler.

The end result is a dynamic web page that refreshes itself with every change entered by our Training team, including courses hosted by our partners. And should there be an issue with Salesforce availability (though to their credit, it’s a rare occurrence) our website isn’t affected.

Statistics-Driven Page Content

Many of our users opt in to share some usage statistics with us (it’s an option when you are entering your FME license). That information remains strictly confidential, but it tells us what formats and transformers are most widely used. Apart from helping to guide our development efforts, this information can also be analyzed to see what the most common tasks are.

One page on our site that is driven by that data is this list of common format conversion workflows. An FME workspace (again, scheduled in FME Server), automatically runs a statistical analysis, loads the results to Amazon, and this list is refreshed based on the results.

And so on, and so on…

All sorts of simple little workspaces are busily doing housekeeping all over the website. Safers – like our webinar team and our bloggers – use simple and well-known interfaces like Google Docs and WordPress to get their jobs done, and FME Server simply takes care of getting things where they need to be. Some load things into databases; some write XML or JSON files to the Amazon platform; others sync up production and staging environments.

And almost none of it is spatial.

“If I didn’t have FME, I’d be writing scripts and cron jobs to make this happen,” says Michael. “I could do it that way, but this is just easier, faster, and clearer – not just for the initial setup, but for maintenance too. Whether it’s me or someone else wanting to adjust or add something, it’s simply a better way of working for us.”

And so, while Michael and the Safe web team have insider access to FME technology, that doesn’t mean these same techniques can’t apply at your shop. Maybe you already have FME Server, and a bit of website automation won’t add any appreciable load.

Or maybe you want to try out FME Cloud – where you can use FME Server on a low-cost pay-as-you-go basis. Simply signing up gets you a free $250 credit – perhaps wisely spent on seeing how FME could automate things you hadn’t even thought of!

– @KrisAtSafe