Lower Face Rigging

Two months ago I wrote an article about the new AutoRig I'm currently building for Maya. I recently started working on the facial rig part, the lower face to be more specific and I wanted to share a bit of my process and workflow. To work on this I had to put aside my own character that I used for the body rig part. I needed something better, with a nice design, a good topology and some textures, so I borrowed Alice Lomiry's Girl.

Although I've made multiple AutoRigs over the years, I've always been a little reluctant to automate the creation of the face rig. The face of a character, more than anything else, is unique, and thus, it needs a unique attention. So instead of doing a one button thing I chose to make several modules that can be connected to each other to build a unique face rig for each character.

Since I'm trying to automate this process as much as possible, it was pretty clear from the beginning that I should forget about blendShapes. Instead I decided to use a network of different types of ribbons, with some volume preservation feature to simulate the flexibility of the skin, the tension and compression.

In this article I will try to explain how I took some real life anatomical concepts and translated them into mathematics and Maya nodes.

Disclaimer : pretty much all of the math I use today in rigging are self-taught. I will do my best to explain everything as clearly as possible, but as I wrote this article I realized that explaining math stuff I learned from the internet in a language that is not my own, is extremely difficult. So if something is unclear or if you find that something I say is wrong, please feel free to let me know. After all, we all learn from each other, and what I've done with this rig is nothing more than the combining of inspirations taken from others and their work and what I've learned from them.

At the end of the article you will find a list of links that have been useful to me to create this rig, as well as some files to download that illustrates some of the concepts I'm describing here.

I. Anatomy and Squashy Ribbons

Everything started with a lot of researches and a lot of real life observation to understand how the face works. I started by searching informations about the muscles of the face, and I realized two things. First, Google Image is full of weird stuff. Second, the muscles, like for the rest of the body, are just one piece of the puzzle. The final goal when rigging an organic asset, is to deform the skin, not to imitate the muscles (we call it "skinning" after all...). Muscles, bony parts and fat are three equally important factors that will determine how the skin is deformed.

Also, the animators who are going to use the rig aren't going to pull muscles, they are going to draw emotions on a virtual face. Thus, when designing the UX (how the animators will interact with your rig), it is important to keep in mind that the intentions and emotions of animation matters as much (if not more) as the anatomical correctness.

a. What is a ribbon ?

A ribbon setup is a commonly used rigging system that uses a number of joints constrained to a nurbsSurface with the help of a rivet (or a follicle). Then, the surface is deformed by several clusters, or a skinCluster. That kind of setup allows to drive consequent amount of joints with just a few controls, and adds a lot of flexibility to the rigs.

The joints are attached to the surface by extracting a position from a [U,V] coordinate on the surface, as well as a normal, and a tangent. Which direction the surface faces, and which tangent you choose to use to constrain the joints are two determining factors that will affect the deformation.

You can see on the picture on the right how the tangent you choose to use affects the final result.

The ribbon setup I'm using also offers a customizable volume preservation feature. I'm simply dividing the current length by the default length of the surface to get a stretch value, and I'm passing this value in a multiplyDivide to apply a power of -0.5, and that gives me a volume value. Now for each scale axis of each joint, I can set a minimum and maximum volume value, a weight value for stretch volume and a weight value for squash volume. The GIF below shows an example of the different shapes we can achieve by tweaking some volume attributes

c. Placing the ribbons

To place my ribbons I tried to decompose the lower face into several major areas as you can see on the picture on the right. (note : the area I call Depressor Anguli isn't anatomically correct, I just couldn't find a better name for it. It's basically the area where the skin is stretched between the chin and the Zigomaticus Major).

The surfaces themselves and their controllers are placed directly on the skin. The joints however, are offseted from those surfaces to be placed inside the skin. Doing this in combination with the volume preservation system simulates the thickness of the skin and the fat layer, and will determine how it will react to tension and compression.

Each surface is connected on both sides either to another surface, or the controllers of the Nose, Chin, Jaw or Lips Corners (this last one is a bit special and I'm talking about it in details further)

d. The lips

The lips are a bit of an exception. They are not place like the other surfaces, it's quite the opposite actually. The surfaces are placed inside the skin, and the joints are offseted on the edge of the lips, where both lips are in contact. This is to make sure that the volume preservation system maintains that contact.

Also, the lips aren't made of one single periodic nurbsSurface, but two separate surfaces connected at their extremities, to fully isolate one lip from the other. Also, in order to make the corners more stable, their joints are actually directly controlled by their respective controllers, and not by the surfaces, and they don't have any volume preservation.

II. Opening the Jaw

a. Rotating & Translating

The opening of the jaw is something that is often handled with a simple rotation, but the reality is a bit more complex than that. When you open the jaw, it actually starts with just a rotation, but if you want to open more then the jaw must be pulled out of the Mandibular Fossa, and slide forward, down the Articular Eminence (source: youtu.be/mB468Jh9aAY)

Some time ago, I was wandering on twitter and I stumbled upon a video from Greg Hendrix where he explains this and shows his way of rigging it. I loved the idea and decided to add this to my rig, although with a totally different, vector-based approach, to better fit my needs and habits.

As you can see on this GIF, the opening of the mouth on the right feels more natural with this setup. More organic and less mechanical like the one on the left.

However, as you know, everybody is unique, from an anatomical point of view, and so we need to be able to tweak some parameters to adjust the behavior of this setup, to match the different morphology of our characters. The thing with this setup is that it's kinda hard to guess the right parameters values before doing any skinning. That's why I chose to drive these parameters with a bunch of attributes on the Jaw controller. These attributes are non-keyable and hidden from the channel box because by default they're meant to be used in the rigging stage only. Animators shouldn't have to use them since I'm trying to keep things as simple as possible for them. If they need to open a jaw, they should only have to deal with a rotation axis on a controller and that's it, I don't want to confuse them with a bunch of obscure attributes during the animation stage.

Here's the list of attributes I'm using to drive this setup (file available for download below) :

• AngleMin : the angle at which the translation of the jaw begins
• AngleMax : the angle at which the translation ends
• TranslateVector : name says it all. This is the vector along which the jaw will be translated. [0,0,1] by default, and will probably rarely be anything else.
• Magnitude : determines the maximum translation along the TranslateVector
• PeakPosition : a value from 0 to 1 that determines when, in the [AngleMin:AngleMax] input span, we reach the tip of the Articular Eminence
• EndFactor : a value from 0 to 1 that simulate how the jaw slides up along the frontal part of the Articular Eminence. (No effect if the PeakPosition is 1.0)

On the right you can see the controllers groups hierarchy I use in my rigs. Jaw_Ctr is the actual control curve, Jaw_Ctr_Offset is the transform node used for group freezing, as well as the reference space for most of the operations of the jaw translate setup. Jaw_Ctr_Sub is the output of this setup, the transform node that is actually translated. It drives the Jaw joints, and other things like the chin, lips, etc... whatever the jaw needs to drive.

So, basically, as you can imagine, since I need to drive translation with a rotation, I need to start by getting an angle value. I don't like using the rotateX, Y & Z attributes directly, so I'm using vectors, which offer much more freedom and far less restrictions. The first vector represent the closed position of the mouth, it's Jaw_Ctr_Offset's positive Z axis. In other word, using a vectorProduct node to multiply [0,0,1] by Jaw_Ctr_Offset.worldMatrix.

The second vector represent the open (or current) position of the jaw. To get this vector I'm projecting Jaw_Ctr's positive Z axis on the jaw's side (the red "open" plane). The goal of the projection is to completely separate the opening of the jaw from any swinging from side to side. We don't want this to interfere in our calculations.

Now that I have two vectors, I can get a rotation axis and an angle by using an angleBetween node, and make two axisAngleToQuat nodes out of it that represent the closed and current rotation of the Jaw. Then I simply plug those two quaternions into a quatSlerp node. That node lets you interpolate between two quaternions by using an inputT value that goes from 0 to 1. That inputT value is driven by a remapValue node that remaps the Jaw_Ctr.AngleMin to Jaw_Ctr.AngleMax span into a 0 to 1 value. I also use Jaw_Ctr.PeakPosition and Jaw_Ctr.EndFactor to control the shape of the remap curve.

The output of the quatSlerp node is a quaternion rotation that is used to rotate Jaw_Ctr.TranslateVector. All there is left to do now is to apply a length to this rotated vector, and this is pretty simple. We're just using a remap value to remap the current angle in the Jaw_Ctr.AngleMin to Jaw_Ctr.AngleMax span, into a 0 to Jaw_Ctr.Magnitude value, and we use this output to multiply each component of the rotated vector. The result is used to drive Jaw_Ctr_Sub's translation.

b. Opening the lips

The base of my lips rig is quite simple, it's just two ribbons (upper and lower lip) connected to each other at the extremities, and each controller has a 0 to 1 non-keyable attribute called followJaw that determines, as you may have guessed, how the said controller follows the jaw. It's based on a wtAddMatrix node, but you can think of it as a simple multi-target parentConstraint, the result is roughly similar.

Instead of plugging the followJaw directly in the wtAddMatrix node though, I'm adding a blendTwoAttr in between, to make the seal lips setup. These nodes are just progressively switching the weights from whatever the user defined in followJaw to 0.5, progressively placing all the controllers halfway between the jaw and their initial position, sealing the lips.

c. Pulling the corners in

Once I had that done though, one thing that struck me was how the corners of the lips moved so linearly. As a matter of fact, in real life, when the mouth is closed, the lips are in collision with the teeth, but when the jaw opens, the lips are sort of stretched and the corners of the lips aren't colliding with anything anymore, so they're slightly pulled inside of the mouth, hanging somewhere between the upper and lower teeth. That is the behavior I tried to replicate.

Once again, every character is different so we need to be able to easily adjust how far the corners are pulled in from one character to another. This is done with another non-keyable attribute on Jaw_Ctr called OpenVolumeFactor.

For each lips controller, I start by calculating the vector that goes from the Jaw's origin to the initial position of the controller, and then I project it on the jaw's horizontal plane (green one in the picture above). Now I have vector that gives me a direction to pull the lips corners in, but I need to find its magnitude (length).

For that, I reuse the two vectors I use at the beginning of the jaw translate setup, the closed (default) vector, and the open (current) vector, and I pass them into a blendColors node to have a linear transition between both. The blender attribute of the blendColors node is controlled by the followJaw weight of the current controller. The point of having a linear transition between two normalized vectors is that the length of the output vector will diminish as we go toward the 0.5 blender value.

In other words, I'm sort of extracting a stretch value out of an angle.

On the image on the right, the pink line is the output of the blendColors node. The pink + yellow line is what the vector would be if it were normalized. What I'm interested in here, is the length of the yellow part. For that I'm simply calculating the magnitude of the pink vector, then I pass it through a reverseValue node.

All there is left to do now is to multiply that magnitude by the OpenVolumeFactor of the Jaw_Ctr, and apply this length to the vector I calculated earlier.

III. Controlling the general shape of the mouth

One thing that is very important when building a face rig, or any rig actually, is that no matter how complicated it is under the hood, it has to remain easy to use. As riggers we're building rigs for the animators. The time that they spend to figure out how the rig work and the amount of controllers they have to move to create decent looking shapes are two determining factor in a rig. Following that guideline, my goal was to make a rig that would mostly articulate around the corners of the mouth. In my opinion, they are the key element of the lower face and I wanted to achieve the greatest possible range of motion just by translating the two lips corners controllers. The image below shows the extreme positions of the corners. Beyond those limits we need to start moving other controllers to avoid issues like teeth penetrating the lips and that kind of stuff. While those extremes look weird and most likely nobody's ever going to need to reach them, when building a rig, it's important to push the deformations further than what we think the animators need. (Note : the cheekbones aren't moving cause they're part of the upper face, so they're not rigged yet)

When the lips move, they seem to follow the shape of the teeth, roughly. So, basically, they're they're moving on a path that would look like a sort of mix between a V shape and a U shape. The idea to replicate that motion in maya is to rotate a vector around the center of a plane that would be somewhere inside the mouth, and the length of the vector would vary to have an elliptical interpolation, and not spherical.

The image above is my poor attempt at trying to describe a mathematical concept (the file can be downloaded below).

1. Everything starts with a plane, which is defined by an origin point (O), and a normal vector (n). Somewhere on that plane, there is a point A, which is the default position of our controller. Then somewhere, not necessarily on that plane, there is a point B, which is the current position of the controller. The vector x is the cross product of the vectors OA and n.
2. The first step is to project the point B on the plane, to get the point B'. The goal of this projection is to separate the horizontal motion of the lips from the vertical motion, so that we can have more control over the general shape.
3. then we project B' along the vector x to find the point E, which is where it intersects with OA.
4. Now we have two perpendicular vectors EA and EB', the idea is to get a normalized vector spherically interpolated between both with a weight between 0 (= EA) and 1 (= EB'), and then to scale that vector to match the non-uniform 2D space in which it evolves. Doing so is pretty simple, we just need to get the dot product of this vector by the EA vector (normalized !), and multiply the EA vector (not normalized !) by the result. Repeat the process with EB' instead of EA, and add both results with a plusMinusAverage. From here, if we add this vector to the E point, we have a new point somewhere on the plane that is "elliptically" interpolated between A and B.
5. That's all for the horizontal motion calculation. The vertical motion is much more simple as we just need to get a portion (determined by a 0 to 1 value) of the B'B vector, and add it to the result of step 4. The result is a point that I use to drive each control of the lips. The horizontal and vertical weight is determined by an attribute with a falloff setup to progressively increase the influence of each corner on the lips controls. Note : I have not yet decided if those attributes should remain unkeyable & hidden or if I should allow the animators to use them. Their primary purpose is to allow the rigger to determine the general shape of the mouth. It could be useful for animators too, but it could also be confusing.

Below is a graph that summarize the operations that affects the position of each lip controller

The End

Thank you for reading me. I hope my explainations were clear enough. There is obviously much more behind the hood than what I've described here but for clarity's sake I tried to stick to the main concepts and features. Below you'll find a non-exhaustive list of links that have been useful or inspiring to me during the creation of this rig, as well as some sample scenes for the Jaw Translate setup and the Lips Corners Slerp, if you feel like doing a bit of reverse engineering.

Please note that everything might be subject to change, for optimisation, stability or bug fixing reasons. If you have any questions or comment I'd love to hear them

Useful Links

• Thiago Paulino's face dissection : Detailed analysis of the anatomy of the face, including bones, fat and muscles
• Melinda Ozel's Face the FACS : Lots of stuff related to the Facial Action Coding System, and how to build expressions
• Chris Lesage's blog : various inspiring rigTips
• Sample files for the Jaw Translate setup (II.a) and the lips corners Slerp setup (III)