Splash Dancing 2D

In this newest piece made with Houdini, I use rotoscoped videos to generate particles and fluids, simulate with collisions all in 3D. I knew it was not as straightforward, hence I made this into a piece in itself.

This is not the first time I've derived a simulation from an image. When I was teaching effects classes in 3dsense I taught my students to emit from an image applied to geometry. The emission criteria can take from any attribute, but commonly from the brightness or luminance of each point or pixel.

Now the hardest part with this workflow comes when the image is moving. In all simulation software, the emission direction and magnitude contribute a lot to the realism of the look, as they transfer the energy of the moving surface in the right direction when they emit and fly away from the surface. These factors usually depend on surface normals of geometry, the speed (velocity) and direction of the surface in motion. With regular geometry, the vertices and faces are indexed, allowing the software to track the object down to each vertex and face, to know where it has been the frame before, and where it will go the frame after, as it animates its position, rotation and size in time.

In my case the image is not a geometry. it does not have surface relief. It is projected on a planar grid, with all the normals pointing in the same direction. Furthermore, as the character's silhouette "moves" and animates, what happens is just pixels or points getting "lighted up" from black to white, or "dimmed down" from white to black values. We have no way to track the movement or direction (unless we resort to computer vision techniques, which at present I am totally clueless with). As humans, we are able to recognise where the head is by interpreting the image, but there isn't a trivial way I can extract or derive at that information. In Houdini there is a way to extract the outline of an image into a curve. But even with this, we cannot make the software understand what each part of the curve represents, or which part of a curve represents a shoulder, for example.

In the end, I worked around it rather than attempting to tackle and solve it. First, I extracted the edges. With some intentional adjustments to noise values that affect these velocities, they can be broken up to look quite natural.

Next I was able to create collision geometry in 3D by giving some depth to the planar shapes from the videos. These were used as colliders such that the limbs now could deflect water and flick them away as the character moves around in 2D space. This added to the accuracy and believability of the simulation, and would hopefully make up for the lack of motion inheritance of direction and velocity. The dynamic movement and relatively high speed of the people in the video clips also helped to sell the velocity noise and pulsing of the set-up. Somehow it looked like the spurts of emission and directions were the result of the movement and shakes of the body, hands and legs.

With this set-up I could use regular forces like turbulence noise and gravity to shape the particles and fluids. Once these forces were working and behaving, the rest of the challenge was to get the values of the forces to shape the simulation correctly.

It made me glad that the set-up required very minimum changes from the first video clip of a single person dancing to the second shot with two persons dancing. Given that the shot sizes were similar and the movement speeds were the same, this should have come as no surprise. But I was still glad that the set-up held up for the second shot.

One of the surprising things that stood out as a challenge to me -- I am sure you will be surprised -- was the cross fading and flashing colours used in the background. It truly taxed me when I was thinking about how to set it up in a procedurally satisfying way, without animating it with keyframes on the colours.

In the end it became almost like a project in itself.

I will write about this in more detail on this in my next article.

Credits for the source videos go to patw64 from Pixabay who generously shared his videos. The groovy and excellent music is by moodmode from Pixabay