Creating Custom FaceFX using FaceFX Editor
Mass Effect 3's FaceFX are exactly what they sound like they are - facial animation effects. These control lip-sync and expression information for the character named in a given Dialogue Node. FaceFX work in conjunction with gestures and other types of animation, but as you might expect, control only the character's face.
This tutorial covers:
- The basic principles of animation necessary to understand the FaceFX window
- Adding a blink animation to a line
- How to use FaceFX to prevent the clipping of dialogue that has been overwritten or redirected
- Best practises for using FaceFX to create entirely bespoke facial effects
This tutorial assumes basic familiarity with Dialogue Editor. If it is unfamiliar to you, you may wish to visit the Dialogue Editor overview tutorial first.
Use the Dialogue Editor to open the package file containing the conversation you are modding. Open the conversation containing the line you wish to edit the FaceFX for. Click the node, and on the right-hand side under its String Reference ID, navigate to the Assets panel.
Figure 1
This particular node is a little bit different. Those familiar with the Dialogue Editor will notice straight away that although this is a Reply Node, and thus spoken by the player character, there is no associated Wwisestream asset for a male character. That is because this node can only ever be accessed by a female Shepard.
Both the Male and Female sections have their own sets of FaceFX.
Mark Meer and Jennifer Hale's line readings can differ in terms of timing, even on the same set of words, and so one set of animations may not work for both. They may need to be edited independently.
For non-player characters, both sets may also need to be edited independently; for example, when a non-player character has two versions of a line. This is typically the case for when lines refer to Shepard's gender in some way.
A non-player character uses the male FaceFX when speaking to a male Shepard, and female FaceFX when speaking to a female Shepard.
Fortunately, there is a process for exporting and importing FaceFX animation. More on that later.
In the Female section, click the button with the tragedy and comedy masks on it to bring up our friend, the FaceFX Editor.
Figure 2
This is the anatomy of the FaceFX Editor.
Under File, it shows us the conversation currently loaded, and what character it belongs to.
Under Lines, it shows us the FaceFX from within this conversation. Directly beneath that panel is a pane that shows us details about this line's audio. Because we opened FaceFX Editor through a dialogue node, the line we want to edit is already selected for us.
The Animations column lists a lot of different things, but more on that in a moment.
The large grey grid is our Timeline. The X axis shows us a selected animation's weight value. The Y axis shows us the progression of time.
Underneath the Timeline, there is a subtitle to remind us of what's being said, although if you have overwritten the dialogue node's audio, this is probably inaccurate.
The Animations column looks very similar across all FaceFX sets. The first 15 items in the list are called phonemes in animation terms. These correspond to the shapes human mouths make when they speak. Not every letter appears on the list, and this is because many sounds share the same shape. Most of these phonemes are fairly self-evident as to what they control, however note that plosives, that is p and b sounds, are controlled by m_Flap, and sounds like 'uh' are controlled by m_Open.
m_Jaw+ and m_Jaw- are special cases - these control opening and shutting the jaw, respectively, and do nothing else to the mouth. Because of the way animation works, most jaw operations can be done solely with m_Jaw+; its opposite is rarely used.
Facial animations are created by combining these effects.
Figure 3
Clicking through the list will show patterns of dots and lines. These dots are called keys, and the lines between them are called tangents.
Animations have minimum and maximum weight values. A weight value of '0' describes the shape not being present on the face at all, and 1 plays it at its maximum value. So, with something seen here like m_Jaw+, '0' means the jaw is completely closed. '1' means the jaw is wrenched all the way open, and is usually not a great value to input if you want naturalistic looking animation.
Some animations, such as Orientation_Head_Pitch, Orientation_Head_Roll, and Orientation_Head_Yaw, can go through 0 and into negative numbers. This means that the animation is playing in the "opposite direction." So, Orientation_Head_Yaw at 0 will be looking straight on. At 1, it will be to the right. At -1, it will be to the left.
Not all animations can or should be used with negative values, and an animation's first key must always be at 0.
Each key describes how strongly this shape should be present on the face at any given point in time. The tangent shows how the shape will "move" to meet where it should be at the next key. This known as in-betweening, or tweening in animation terms. Smoother curves are slower, gentler movements and sharper ones are quicker and more dramatic.
Here we see m_Jaw+'s movements, which are usually fairly sharp, but subtle. In fact, throughout this line, the strongest weight it ever has is 0.40, which corresponds roughly with the start of the word "one" in this line.
Figure 4
Continuing to click through this list will show that some things only have one or two movements, and their timeline seems to stop much sooner than the end of the audio, as seen in fig. 4 above with m_OH. Things may be keyed to remain at a value of 0, or something very subtle. This is because the effects of all of these work in conjunction with one another, and they stack.
You will note that the timeline resizes as you click through. This is because the timeline shows you only the selected animation's keys up until the last one keyed in the sequence. For something like Blink, the range for this might only be very small.
Figure 5
Right-clicking anywhere in the Timeline gives these options.
- Add Key will place a key at exactly where your cursor is, both time and weight-wise. This can be adjusted after the fact.
- Offset All Keys After This Point is used when your line's audio has its float value set to something greater than 0, ie, a delay before it starts, and you need to move the animation up to match. (If your line has a float value, you can check it in the Dialogue Node's Matinee.)
We are going to make Shepard blink during this line at 4 seconds in.
The version of this particular dialogue node being worked on has had its audio replaced. The new length of the line is 7 seconds.
I want Shepard to blink about 4 seconds in. However as seen above, Blink has no keys, and the timeline is only visible up to 2 seconds.
Right-click, Add Key as close to 0 as you can. Right-click the key itself.
- Set Time moves the key to a point in time.
- Set Value gives it weight value.
- Break Tangents is usually used to fix weird curves after deleting keys.
- Delete Key does what's on the tin.
- Flatten Tangents is a contextually available option that creates a straight line from one key across to another. Useful for holding an animation in position.
Note: Keys depend on what is ahead of them. You cannot move a key ahead in time if it overtakes another in front of it. You must delete the key, and then place another ahead, in the desired position.
Set your key's time to 0. Set its value to 0. The screen will update and the grid will change. At about 0.50 in the timeline, place a second key and set its value to 0. This defines the beginning and end of the animation. Now, add a third key as close to in the middle of them as you can, and set its value to 0.95 - this means that over the course of a half second, Shepard will almost completely close her eyes, then open them again. Your completed keys should look something like this:
Figure 6
But, the timeline shows this animation starts at 0 and completes at 0.50 - blink and you'll miss it. Right-click before the first key, and choose Offset All Keys After This Point. Set it to go at around 4. Upon confirming, FaceFX Editor will update, and more of the timeline will be visible. It's as simple as that for a blink! Setting keys at different values will make Shepard hold her eyelids lower or higher, useful for different expressions. Change the pacing of the blink to make it something slow and owlish, or something fluttery.
FaceFX has a close relationship with audio. As something of a failsafe, audio will stop playing if the animation sequence is complete and there are no more facial animations. This is meant to stop the game from giving dead air if there is a second or two of silence accidentally left on the end of an audio asset. However this can present problems for modders who have overwritten dialogue nodes with files significantly longer than their original settings. Correctly setting a dialogue node's InterpLength and its associated WwiseEvent's DurationMilliseconds values are very important to do first and are beyond the scope of this guide. However, once those steps are done, if audio still skips, adding FaceFX will solve the issue. Simply add a key to any animation in the FaceFX set and set its time to fit the DurationMilliseconds. Make sure its value is above 0, even if small. This works as a temporary measure to ensure the audio will play, but more work in FaceFX will probably be needed.
Deleting keys down to a certain point from all animations in a FaceFX set will cause an audio line to terminate early if needed.
It's important to understand that whilst FaceFX is a powerful tool, it is limited, and its use comes with several drawbacks that are not present in conventional animation environments. Unlike the original animators, you will not be able to see your changes reflected live as you do them. You will not have access to a preview of the animation. You will be animating this line through a combination of guesswork and memory. There are, fortunately, a few tips tricks and tools to help make this process a little easier, although it can still be accomplished without them.
- Audacity - If you are dealing with replacing and editing audio, you probably have this already. But, it's a free program for editing audio.
- OBS Broadcaster - For recording what things look like in-game. Also free. Pretty easy to set up.
- GIFCam - A fantastic little app for making GIFs quickly. It was used for making the GIFs in this tutorial, which were used as iterative reference.
- Have a save file as close as possible to the conversation your line is editing.
- Have a backup of your mod before you start messing with FaceFX.
- Look around in dialogue nodes at the vanilla's FaceFX. Compare what keys look like, what values are usually used.
So, you have a modded Dialogue Node with fresh audio injected into it. It's a combination of lines from all over the place and doesn't have animation data you can steal easily from other places. You have checked that the line plays properly, but the FaceFX look weird, because of course they're set up for a different line. You are looking at the line's FaceFX not know where to start.
If I'm working from scratch, I tend to delete almost all the existing FaceFX information. Sometimes, I leave in things like blinks, eyebrow movements, or head movements if I think they work. /
Because these need to be done one phoneme at a time, I like to start with m_Jaw+. The reason for this is it usually has movements that go all the way through the line from beginning to end. It's something that makes an immediate and obvious difference. So I go ahead and delete what's there.
Open your line's audio in Audacity.
"Hehe. I already figured that out. It's not a big deal. Heh. I'll look forward to our next talk."
Note the peaks and valleys in the waveform. Usually, big peaks correspond to larger jaw movements. To get a good sense of what you need to do, try saying your line aloud along with your recording. Note the way your face moves.
What's most useful about Audacity here is that it's easy to correlate the shape of a waveform with its timeline, and sync up what you're doing on yours in FaceFX. I delete all m_Jaw+'s keys individually, then start fresh.
These are all my m_Jaw+ keys, from beginning to end. By comparing the waveform to the keys, you will see a basic correlation.
Ingame, this won't look like much yet, however:
This has a head gesture left in, and a few other phonemes at the very beginning that I want to work with. The rest is all the jaw. You can see there's a lot to be done.
![](https://imgur.com/AHVYg0Q.mp4] If the above is not loading, see here
This is the same line, with just a few more phonemes added later on in the sequence, including the blink from earlier. It's beginning to look a lot more lively, like she's talking!
The secret is iteration. When I make a few significant changes, I reapply my mod, load it up and record what's happening using OBS. Then, I look at my recording and use GIFCam to make a GIF of the line in its current state. I use this for visual reference in adding more changes, because it's nice to have a looping example of what my changes have done right in front of me. ME3's phonemes are quite forgiving, so long as an effort is made to basically match sounds, it will look believable enough.
#Exporting and Importing FaceFX Animation
Once your animation is complete, you may need to duplicate it into another set.