Facial Performance and Lip Sync Animation

Ideation, Brainstorming, References

For my Facial Performance and Lip Sync Project, I had a gazillion ideas racing through my head. I wanted to work with a dialogue that would showcase a transition of emotion in my scene and would help me experiment better with the facial controllers on my rig. There were some dialogues that I came up with myself and shot the reference for using multiple tones, moods, emotions, styles, voice levels to see what would work better and would open doors of experimentation and a challenge for me during the course of this particular project.?

I shot nearly 80 references using different dialogues, which are the following?

• "Everytime I come home after work, I love to chew a piece of gum and stick it in my coffee table"
• "What part of dont touch my pizza don't you get?"
• "That's the thing about relationships, one person is like blah blah blah blah blah and the other is like what are you talking about and the other person's like blueh blueh blueh blueh blueh"
• "You can't possibly be that stupid"
• Ah, nothing like the scent of green tea pouring through my veins"

I shot the references for the aforementioned dialogues and then favourited a couple of them to get them approved by my professor. Once that was sorted, I drew a couple of sketches and made some pointers to use the context within the scene and set it up with a proper storyline and how the scene would play out.?

Context and Storyboarding

Sarah came home after a long day at work only to find her husband complaining about a piece of chewing gum that he found stuck under the table. Her kids are crying and screaming in the back and she is annoyed and agitated and goes on to respond to her husband?

"Cuz everytime I come home after a long day at work, I love to chew a piece of gum and stick it in my coffee table".?

Reference Analysis

A key stage in Animation is Reference Analysis. It is used to understand your key poses and highlight the principles of animation applied within the scene and create drawovers in order to create ease while animating in Maya.?

The key poses were not many, however I drew the key poses, arcs, overlapping action and follow through and the distribution of weight within my syncsketch file and downloaded it so that I could use it as my image plane within Maya.?

Here are some screen grabs from my SyncSketch File

You can view the syncsketch file here: https://syncsketch.com/sketch/NmE2ZGI2MDU4/

Since I knew that this project was mainly focused on Lip Sync and Facial Performance, I took my notes from the Lip Sync class about Phonemes and Visemes to understand mouth shapes and how I could avoid over enunciating each word and make it look like it was a natural performance. I started studying mouth shapes using pinterest and youtube and understood one thing very clearly - that I need to use the sounds for my lip sync instead of the movement of my mouth to create a more natural, dynamic lip sync animation.?

Animation

I chose to use the Neka Rig provided to us on the materials on Canvas for students by Richard Cunningham. Since the rig was really good and I thought it'd suit well with my narrative, I decided to use it. In Maya, after bringing in the rig, I set the arms and the legs in IK and then started working on the blocking of my character in Stepped mode. I also loaded in Animbot and mirrored the initial key poses for my arms and legs and then adjusted them according to my reference. While I was blocking, I also kept checking my silhouette to ensure nothing was hidden, the body was clearly seen and the posing was done correctly. When I was happy with the blocking for the body, I viewed it in Spline mode. I divided the animation into manageable chunks and then viewed them in Spline throughout the project. Simultaneously kept working on the graph editor. This video was super helpful How to Avoid a Spline Depression https://youtu.be/6CZrfGBcp-g to understand how the graphs could be better when working in Spline mode.

Simultaneously, I kept taking feedback from Richard, Constantinos and Kshitij in class and during the weekly support sessions and implemented the notes within my Animation.?

In addition to that I studied the book Richard had shared with me Anatomy of Facial Expression (https://kingston.box.com/s/myogvs9ozh0igy9b3jg42nu4p1armk16 ) which helped me understand how the muscles on the face work and how to create facial expressions. The book was a great aid for me to work with my cheek controllers and adding exaggeration using facial controls.?

I also took aid from the Animschool YouTube Channel?- How to Push your Animation Poses https://youtu.be/WWVD7pwDpJA to use it as a tool to exaggerate my key poses for my animation. I worked on getting better poses by dividing my body in three sections - the top, medium and bottom. This breakdown helped me improve my poses and helped in creating overlapping action and follow through in my animation when I started adding the in betweens. Like before, I still couldn't turn the stepped preview on on the timeline, hence, I kept working in stepped mode using the graph editor.?

For the face, I created some important notes with the help of the Animschool videos that I later used for my Animation.

Keynotes:

• To make the facial performance look more fleshy, use arcs to ease in or out of the animation for any given facial feature
• Use the sound of your dialogue instead of focusing on the mouth. Use the sound as a guiding line for your lip sync
• Eyedarts move really fast - hence to make them look more realistic and alive, they need to be animated on ones and twos and move around in either a Triangular shape or a Parallelogram/Rhombus.?
• Eyes should never move in a ping pong motion
• Eyes need to be quick, snappy and irregular
• And along with the eyes, the eyelids need to move as well (only ever so slightly)
• Always start working on the jaw first
• Donot enunciate each phoneme
• Donot oversimplify or over complicate mouth shapes
• Always focus on creating Asymmetry for a more dynamic natural animation
• Eyes are the quickest, followed by the eyebrows and then the mouth
• And in eyes, the downs are quicker than ups
• Eyebrows, compared to the eyes and move relatively slower - can be animated on a frame range of 3-10
• LipSync has an array of different speeds?
• Usually when working on mouthshapes for M, B and P, hold them for 2 frames
• Donot open and close mouth within one frame differences

Links for Research and Keynote Compilation

Character Performance Class by Alek Kolev and Luke Randall

Facial Movements by Smchwhinnie Mewsimmic

Making Eye Darts Look Alive https://youtu.be/xqmf-4SRGvQ

Importance of Mouth Acting in Animation

https://youtu.be/AwzOiRVF84M

How to Animate the Face in 8 Minutes

https://youtu.be/BBJ24kOttTU

Facial Animation in 10 Minutes

https://youtu.be/bMQkeJ1xRfw

Tips for Animating the Head

https://youtu.be/ucE27c-zgYs

Blocking Mouth Shapes in Studio Library - Garrett Shikuma

https://youtu.be/AYkjdZehzJs

Acting Through Anticipation

https://youtu.be/qDqtXhFyMEQ

I implemented the aforementioned tips and teachings from the videos and applied them for my performance.

I used Studio Library to block out the mouth shapes - it was an incredibly helpful tool that saved a lot of time and got the basic blocking for the lip sync. Hence, I continued to adjust the facial controllers on my rig to create the expressions using my reference.?

I took feedback from Will Cook - the Animation Director at BlueZoo and he suggested that I create clearer silhouettes and create a more dynamic movement in her body as my character seemed stiff. I noticed that there were several inbetweens for the spine and chest controls - therefore, I started deleting keyframes using the graph editor to establish smooth curves and removing keys that weren't needed. I also took Constantinos's feedback and Kshitij's and they suggested that I use bendy controllers to create fluid motions for my arms. And add more motion on the chest, since my reference had a lot of body movement.

For the facial performance, I kept working on creating asymmetry to create a more dynamic animation. I used the greater than and lesser than rule for animation to create exaggerated posing that didnot make the animation look monotonous and added more character to the scene. One of the references I used as a guiding tool was the following image

I implemented this through out my animation - here are some examples of how I applied asymmetry

I also took help from the following videos to help me better my overlapping action and follow through and make the character look more alive by pushing my poses and using arcs and exaggeration.?Overlapping action and follow through was mainly applied on the mouth for lip sync and the arms and hands to replicate the dynamic movement as seen in the reference.

Keep your Animation Fluid With Overlap and Drag

https://youtu.be/eICBiX6ydUE

?

Loosening Up Animation with Overlap in Body

https://youtu.be/CGSRPAWOkLA

Understanding Overlapping Action and Follow Through

https://www.dhirubhai.net/posts/arisha-lakhani-3010a216a_animation-3danimation-characteranimation-activity-7039373925079764992-GSyQ?utm_source=share&utm_medium=member_android

Acting Through Anticipation

https://youtu.be/qDqtXhFyMEQ

I kept taking feedback from both Kshitij and Constantinos and kept working on the little changes that needed to be done and adding more details using the bendy controllers and secondary controllers on the chest, neck and mid torso. At one point I had too many keys set on the chest controllers which was causing a couple of minor snaps and the animation didnot look fluid so I reduced the number of keys and smoothened the curves on the graph editor manually and in some places using the auto and ease tangents.?

While I was working on the polishing of my animation, I kept tweaking the points where I had added exaggeration and anticipation in my animation to make it more impactful. My reference had very tiny details in the body movement and I tried to add them to my animation to ensure the body mechanics were right.?

I worked on the polishing for the Animation and also started working on adding the environment for the narrative. I used the Andy's room environment from Agora Community.

For the lighting, I used the HDRI from PolyHaven on the Skydome and adjusted the exposure and intensity along with the area light and the point light in the scene.?

For a high quality playblast I adjusted my render settings according to the How to Get a PlayBlast video uploaded on Canvas.?

I know that the animation can be done way better, I was not satisfied with the way the body was moving in the animation and I wanted it to be more impactful. I also wanted the camera to be more focused on facial performance - however, in my reference I have several body and hand movements which is why I was unable to set my camera closer. Nevertheless, I thoroughly enjoyed working on this project and I will keep practicing using the concepts taught in class and apply them to make sure that my animation is flawless.?Lastly, I created a compilation of the passes of the animation passes through out the course of this project. You can view it here on SyncSketch.

https://syncsketch.com/sketch/NmE2ZGI2MDU4/