Free Form Programmable Image Processing with 3D Applications (Part 1)

Image Processing or any of its sibling imaging technologies are not for the faint at heart! Image Processing is a fantastically large field of algorithms, software and digital hardware overlapping with numerous scientific fields and countless applications.?

Free Form Programming Language originally was conceived to make the software programming of the Image Processing a much easier and more intuitive endeavour and also make the field attractive to students by overlapping with 3D applications such as gaming and animation.

While the goal was a noble one, alas, the journey to this very moment showcasing Free Form Imaging was daunting and riddled with never ending challenges in software and language design.

Did I just say ‘never ending challenges’ ?

Finally we stumbled upon a better intuitive language for Image Processing, ironically found in the most abstract and unintuitive field of mathematics called Category Theory !?

We borrowed the Category's Arrow concept which renders a function/map between two objects. In the case of Free Form programming, the objects are variables or APIs, and Arrows rendering a pipeline/sequence of these APIs evaluating one after the other in order, back to back, composing more and more complex computing objects to transform images.

This compositional design technique is that of applying back to back processes/functions until the desired effects (e.g. image transformations) appear.

compose: in mathematics, function composition is an operation that takes two functions f and g, and produces a function h such that h(x) = g(f(x)).

This epiphany transformed the standard procedural programming gibberish below:

image2=Imagerecolor[image, {colornegate[binarize[colordistance[image, orange]]]], blue}];        

to a compositional sequence:

image ->  color distance [orange] -> binarize [ ] -> 
negate color [ ] -> recolor image[blue] -> image2;        

a Free Form Creole, mixing the loose spoken language, for wording the recoloring steps of an image:

make a mask for the image's orange coloring and then make it black and white then make a negative and then recolor with blue (over the mask)        

Teaser from Part 2

We had to break up the newsletter into at least two parts to give a good presentation for the 3D applications of Free Form Image Processing.

Credits: Associate Producer Tahir Gholizadeh

->: Arrow operator

Left-hand-side (rhs) a function and right-hand-side (lhs) is also a function; the -> operator serves as a function composition operator with a new argument passing/sharing Cloud Object Memory system. This Cloud-Memory system stores the results of interim and final computations of all functions in between arrows.

Moreover, there can be varied Persistence conditionals for the Chain Context, varying depending on the chain’s precedence and functions themselves.?

conditional: a boolean variable or boolean valued function which upon returning true evaluate different scripts.

In this case Persistence Conditionals determined the temporal or persistent nature of a cloud variable.?

Chain: a -> b -> ...-> c

the order of evaluation in time is from the left to right and all functions need to finish their evaluation before their next in chain functions could be evaluated, hence chain.

a->b->c ? c[b[a]]        

? : Equivalence in definition and function

Why Arrows?

1. Arrows avoid tedious bracketing and far easier on the eyes
2. Arrows are natural to render patterns of composing objects
3. Arrows lend themselves better to allow for context between their target objects

Apropos of #2 see the following oft-repeated pattern for segmenting an image into sub-regions (mask):

image -> blur [ ] -> binaries [ ] -> delete small components [ ] -> mask;        

This most useful pattern readily recognized and cognized by Free Form programmers, while the procedural counterpart not as much:

 deletesmallcomponents[binaries[blur[image] ] ] ;        

Make this chain longer and while the Arrows preserve their useful rendering of patterns, the procedural counterpart suffers from unintuitive tedium.

Note: Unless otherwise stated, in most Free Form naming conventions blank spaces are allowed e.g. ‘delete small components [ ] ‘.

Moreover, the procedural programming languages require adherence to bracket-matching rules, e.g. ... blur]image]]], which are nearly absent in Free Form expressions, you do not use parentheses when writing to someone!??

Chain Cloud-Memory

Remarkably, the functions down the Arrows chain could share variables not just their return values but any other interim-computed or external variables they have accessed to compute their functionalities.?

For example, the 5th function down the chain could share and access the variables from the 3rd function earlier in the chain! This sharing of the cloud variables is called Chain Context.?

But our original motivation to add Chain Cloud-Memory was not so much to better the evaluation functions of the chains nor to help in debugging, but mostly as an educational tool to allow the programmers to train their eyes and thoughts with new skills dealing with digital processing of images. That education was of paramount importance and the very value proposition that justified the required painstaking R&D.?

Cloud Function

• Each Cloud Function is contained with a Cloud Object and can easily be called from anywhere on the net by its URI(Universal Resource Identifier).
• While the URIs remain unchanged their functions can be replaced or updated without any alterations or recompilations required at the remote application level.

[ ] : reserved for functions bracketing and strictly imposed for the operator ->?

() : for mathematical bracketing and so no

my img ->  color distance [orange] -> binarize [ ] -> 
negate color [ ] -> recolor image[ rgb[0.06, 0.65, 0.8] ] -> recolored img;        

colors near [d, dfun]

colors near[ ] can be used to represent a region around a specific color.

The distance function dfun can be any valid Color Distance metric. Possible settings include:

orange region <- colors near [orange, 0.5];

my img -> recolor image [ orange region, rgb[0.06, 0.65, 0.8] ] -> recolored img;        

L*a*b* Color Metric Space

The CIELAB color space, also referred to as L*a*b*, is a color space defined by the International Commission on Illumination (abbreviated CIE) in 1976. It expresses color as three values: L* for perceptual lightness and a* and b* for the four unique colors of human vision: red, green, blue and yellow. CIELAB was intended as a perceptually uniform space, where a given numerical change corresponds to a similar perceived change in color. While the LAB space is not truly perceptually uniform, it nevertheless is useful in industry for detecting small differences in color.

Source: https://en.wikipedia.org/wiki/CIELAB_color_space

3D Chromaticity Plot

Notice the shift towards more blue-ish Chromaticity region, for the image processed cyan carrots:

Coming Soon: Part 2

3D Case Studies: ImageMesh, MeshAnim, Cloth Physics

Credits: Associate Producer Tahir Gholizadeh

MeshAnim

Cloth Physics