FME has now colour vision

Recently, I published RasterColorAnalyzer, a custom transformer, which maps the colours of an image to the named colours. Here, I'd like to share some details how it works "under the hood" because the idea behind the transformer can be useful for similar datasets and transformations.

FME Workbench can transform pixels of a raster into vector polygons with an RGB triad as an attribute, and with some practice, you probably can say that, for example, "227, 38, 54" (or #e32636 in a hexadecimal form) is a reddish colour with maybe a hint of pink, but wouldn't it be nice if FME could tell you quickly and accurately what colours a picture, a photo, or a map has? Not just RGB values, but the names of the colours. That is, our string "227, 38, 54" would become "Alizarin crimson".

Back in 2015 when I spent a year playing #Minecraft for living, I had to solve the following problem - if we take a SketchUp model, how could we map its colours to a limited set of 16 hues available for some Minecraft blocktypes? I created a point cloud that combined all possible RGB24 combinations (24 bit per colour). The point cloud was a cube with 256 units along each side. Then, I calculated 3D distances to the 16 Minecraft RGB triads (where RGB=XYZ) for each point, and chose the closest standard Minecraft colour to each of the 16,777,216 RGB points. After that, I replaced points with voxels and merged them by Minecraft palette colours. At that moment, I realized, I generated a 3D Voronoi diagram:

So, the process of converting a SketchUp model to Minecraft looked as follows:

• Turning the model to a point cloud (colours, even texture colours, are preserved);
• Switching temporarily the XYZ coordinates to RGB coordinates;
• Passing RGB points through the filter depicted above;
• Assigning colour (Minecraft blockData) depending on the sector each point falls into;
• Restoring the original XYZ coordinates and saving the Minecraft world.

Here is an example of such a transformation:

This was a nice solution for the problem, but I never turned it into a more universal process. Until now - when our FME Evangelist Mark Ireland asked me whether it would be possible to tell what colours make a flag of a country.

I thought I could quickly implement my Minecraft workflow to work with images, but quickly realized, it wouldn't work very nicely for pale, muted, pastel tones - they may have a colour, which would be distinctively pink or green for a human eye, but fall into the gray part of my colour filter cube - just because the RGB values are close to each other. Instead, I decided to use a slightly different approach.

I found a few sources of named colours - W3C, Wikipedia list of named colours, Pantone colours (taken from hexcolor.co), and built a transformer that extracts RGB triads from images with RasterToPolygonCoercer and maps them to the closest named colour.

We can think of this method as a 3D implementation of NeighborFinder. Under the hood, it works as follows:

• We extract polygons with RGB values from the input image. For images with a lot of colours, we can reduce their number by generating and then resolving a palette (we need resolving to keep RGB values instead of colour indices)
• We build 3D points using RGB values of the image as XYZ;
• Inside the transformer, there are several JSON datasets with named colours, which also have RGB triads, which we also use for creating 3D points.
• With the next step, we connect image point to all named color points, measure the lengths of the connecting lines, sort the image colours by the distances to named colours and keep only those named colours, to which the distances are the shortest.

This is it, now we know, what is the nearest named colour to each colour of the image. I want to note here - we find the closest color, not the exact match, although sometimes, we can get such matches.

Have a look at this #ThreeJS visualization of the mapping. I took the photo of two macaws my family fostered for year, and applied the process described above to it. The cubes represent the colours of the images (the reduced number), the spheres - the named Wikipedia colours. The pipe connectors show the mapping. You can click on cubes to see their RGB values, and spheres to get the RGB values, names, and shades.

You may notice that some colours have pretty poetic, imaginative, but not very telling names. Do you know what do Byzantium, Flirt or Paua look like? I probably could guess the first one, but I'd be completely lost with the other two.

This is why I added a shade to each colour. I did it by scraping the Encycolorpedia.com website - I was sending requests with a hexadecimal value of the colour as a parameter, extracting the shade name from the response, and adding it to the colour name JSON inside the transformer:

If you are a fan of regular expressions, here is the regex I used to find the shade string in the HTML body. The regex find not only single words like magenta, but also can find hyphenated words like pink-red:

shade of <strong>([\w\s-]+)</strong>

With colours, we use 3 dimensions - one for each element of the color triad. However, this method is not limited to 3D - we can easily adopt it to 4 or more dimensions - for situations where we need to map values by more than 3 sets of values. We can't use the normal VertexCreator and LengthCalculator transformers - they are limited to 2D/3D, however, we can calculate distances between multidimensional points with AttributeCreator - all we need is to apply Pythagorean theorem:

The rest is the same. I think this is really cool.

So, this is how RasterColorAnalyzer came to life. You can try it yourself with FME or check the report I generated on the world flag colours. Maybe just one note about the report - I cannot guarantee the flags in the dataset contain accurate representations of the colours established by their respective countries.