What's Up Wednesday (16th October, 2024): Exploding ISO Time Intervals, Thanksgiving Heroes, Get Ahead with FME, Community Updates, and More!

Hello FME friends,

How is your month going? I'm still busy working on the 12 Days of FME Contest, which will launch in December. It's a contest powered by FME, and developing those workspaces has been the usual fun and interesting process.

One of my tasks was to create a workspace to process date intervals. My selected method used an unusual mathematical function in the arithmetic editor, so I thought I would start today by sharing that with you.

Exploding ISO Time Intervals

FME has several great datetime transformers, particularly the DateTimeStamper and DateTimeCalculator.

The DateTimeStamper creates a datetime value in an FME format that looks like this:

20241016173245.1431739        

That is, YYYYMMDDHHMMSS.S (year, month, day, hours, minutes, seconds).

The DateTimeCalculator can calculate the period between two datetimes and return it as an ISO time period, such as:

P21Y11M2DT21H27M45.3S        

That is, 21 years, 11 months, 2 days, 21 hours, 27 minutes, and 45.3 seconds.

Both of these strings are similar because they are made up of multiple individual values, and are also similar in that sometimes I want to extract those individual values.

For a standard datetime, I came up with the DateTimeExploder transformer, which has been on the FME Hub for a few years now (and is probably due for an update).

However, this week I wanted to split up an ISO time period. That's tricky because you can't process it in a DateTimeConverter, it isn't supported by the Python isoformat() method, and it can't easily be split using a single regular expression.

Even if I could split it up easily, what added to the difficulty was that I wanted only days, hours, minutes, and seconds; I needed years and months to be converted to days, and then hours, minutes, and seconds kept separately.

So, here's what I did:

• Changed the original output from ISO interval to days, to give a decimal value such as 24.343654

• Extracted the number of hours from the days decimal part using this formula:

@fmod(@abs(@Value(ElapsedTimeDays)),1)*24        

The fmod() function takes the decimal part of a number (i.e. the .343654). The abs() function deals with negative numbers. I multiply the result by 24 to give me a decimal hours value.

• Extracted the number of minutes from the hours decimal part using this formula:

@fmod(@abs(@Value(ElapsedTimeHours)),1)*60        

• Extracted the number of seconds from the minutes decimal part using this formula:

@fmod(@abs(@Value(ElapsedTimeMinutes)),1)*60        

• Finally, I round down all of the values using an AttributeRounder, in order to have whole numbers only. That gives me the result of 24 days, 8 hours, 14 minutes, and 51 seconds.

Notice the repetition in how I drill down, taking the decimal part of the previous result as the starting point of the next result.

I haven't made this into a custom transformer yet. I'm still trying to determine if this is the best solution. But I wanted to share the technique and highlight the fmod() function, which is very useful at times and yet not one that is very well known.

In fact, maybe we should have a related fmod() transformer, just like the int() and round() functions are exposed in the AttributeRounder. What do you think?

Custom Transformers on FME Flow

When an FME Flow workspace depends on a linked custom transformer, there are various ways to handle this. You can embed the transformer, you can upload it to the same repository as the workspace, or you can upload it to the Transformers folder in Resources:

I've been doing the latter method recently. Because multiple workspaces share the same transformers, I don't need to upload the custom transformer every time.

You will keep getting a warning when you publish a workspace:

But if you are confident that all transformers have been uploaded, you can check the box to skip that warning in the future.

FME For Good: Thanksgiving Heroes

It was Canadian Thanksgiving last week, and American Thanksgiving is coming up soon. Plus, I know there are other rituals around the world to celebrate and give thanks for harvests. So, I think this user story is appropriate to all of us.

FME user Adam Radel has been using his FME skills for the past 4 years to optimize delivery routes for Thanksgiving Heroes, a non-profit organization that delivers meals to families in need.

Adam uses one of my favourite transformers (the HTMLReportGenerator) and one of my favourite technologies (QR Codes) to help volunteers navigate to different addresses; plus uses FME to validate the data and look for duplicate addresses.

I think this is a great example of using FME for good.

Personally, I get upset when I see small charitable organizations using paper maps and forms for route finding, campaign mapping, and service planning. I know that sometimes it's due to a lack of technical skill, and sometimes it's a lack of resources.

Fortunately, we users have those skills, and we have the ability to apply for FME charitable licenses.

If you want to help a local organization in this way, you know what to do. I'll be very supportive of your efforts and give whatever help I can to get you started, as will most of the FME community.

The Peak of Data Integration

New on the blog today is "5 Reasons to Present at the Peak of Data Integration, 2025."

Let's be a little selfish and say that a big reason to present is to gain recognition and expand your FME network! I think it's safe to say that the most respected and familiar names in the community have all presented at the FME user conference at some point.

I also believe that taking an active role like this enhances your conference experience. I like to feel involved and important, and being able to present to others really helps.

So, I just wanted to say that it's not a crime if these are your main goals!

Of course, you might also want to speak because of the price discount, exclusive swag, the chance to improve your speaking skills, or even because you want to share your knowledge to help your fellow FME users!

If either the blog or I have inspired you, then click here for more details on how to submit a presentation idea.

Community Updates

Question of the Week

Last week, the question coincided with Canadian Thanksgiving, and so it was all about giving a shout-out to folks who had helped you in the FME Community.

I don't think I have space to mention everyone whom I respect and admire for their contributions (you'll know who you are) so let's go onto this week's question:

"What are you bringing to a desert island?"

You actually get to bring 3 things, not just one, so you can be wide-ranging and maybe pick a luxury item as well as a survival item. So I think my three would be:

• A knife. A big one. A Crocodile Dundee "now this is a knife" type of knife.
• My fishing rod. Oh, and a spare reel of line, because I do tend to snag on rocks a lot.
• Either my music collection or book collection. Obviously on a solar-powered device.

Books or music? That's a tough choice. Or I could be like phillipeB and pick FME beer, an FME umbrella, a comfy chair, and the FME Lizard to share them with me:

So? What are your choices?

User Spotlight

Speaking of users who have contributed to the community, this month's user spotlight is shining bright for Joep Kuijvenhoven, otherwise known as joepk.

Joep is a GIS Consultant who works for Avineon-Tensing, a GIS consultancy (and Safe partner) based in Utrecht, the Netherlands.

Click the link above to learn more about him, his tips for FME, and why he wants to work from a small cabin with a herd of cows!

Weekly Quiz

This week's quiz on the FME Community is a numbers quiz. You simply have to guess the correct answer to an FME-related question, where the answer is a number from 0-100.

For example, today, I want to know how many different units of measurement FME coordinate systems use.

For sure, we all know about metres, feet, and degrees. But what about the Indian Yard? The Gold Coast Foot? The Benoit Chain?

Click the link above to make your guess and to answer the other 4 FME-related questions.

Idea of the Week

Checking out this week's new ideas on the community, I was most struck by this idea to improve searching and filtering in FME Flow.

"We would find it very helpful to have more search criteria for the forms in FME Flow. A general rule of thumb would be to be able to search/filter rows based on any column displayed on the form."

I chose this idea because I, too, feel that it could be improved. In particular, I feel that the Jobs page needs a search field because it's too awkward to have to use the Filter dialog to drill down to a particular workspace (admittedly, I'm still using 2023.2 for various reasons).

If you have any thoughts, then check out the idea, leave a comment, and upvote it.

Breakfast of Champions

Remember, the first "Breakfast of Champions" - a community-oriented event to bring users together in person - is taking place in Dallas on... this Friday! Be sure to sign up if you would like to attend this event.

A second event is taking place in San Francisco on 18th November.

Miscellaneous News

• On the FME Hub, I noticed that Martin Ekstrand has created a series of transformers called the ArcGISAutomagicDocumenter series. These transformers can be used to list all ArcGIS services data sources and to create reports from them. If I understand correctly, they are tools for automatically documenting the state of your ArcGIS server. Check them out, starting with the ArcGISAutomagicDocumenterServer.

• Also on the FME Hub is an OvertureMapsDataDownloader transformer from Arda Durmus, which allows FME users to directly access and download geospatial data from the OvertureMaps Foundation using DuckDB.

• The Data and Analytics for the Public Sector Summit is now on. We're there. Are you?

• Sweco presents: FME on tour 2024 - Stockholm. 6-7 November at Nalen

• It's rare that I see the AttributeSplitter, ListDuplicateRemover, and ListConcatenator described as the power trio of FME transformers, but today is that day! There's no doubt that lists can be very powerful and a great tool for tasks like extracting the unique values in a set of data.

• Here's my contribution to the List conversation. My power trio is the Aggregator (to create the list), ListHistogrammer (to count the different values), and ListSearcher (to extract those counts).

• October 24th is the date for the next con terra Espa?a FME Accelerator course.

• Meanwhile, con terra Germany presents "FME in a Nutshell" (by the time you read this, it might be later today!)

• Struggling with Regular Expressions? For me, the only way to not struggle, is to not use them! So I should definitely check out this video demonstration of FME's new AI Assist tool.

• Lo?c finds that FME swag is the literal truth!