Lip-sync animation and Extract Phonemes

The SFM can extract phonemes from a dialog clip to create a preliminary lip-sync animation that you can then refine. You can create lip-sync animations for both existing dialog clips and new ones you record yourself.

To create lip-sync animation for a dialog clip:

1. Right-click the dialog clip, and click Show Waveform. Note that only waveforms with Show Waveform selected will be used for automatic lip-sync.
2. Hide the waveforms for other sound clips by right-clicking them and clicking Hide Waveform.
3. In the Animation Set Editor, expand the character's animation set.
4. Right-click any of the character's controls, and click Extract Phonemes. (You can take the defaults until you know how you want to modify them.)
5. Click Extract.

You can now play back your movie to see your new lip-sync animation.

Tip:You can record dialog in different languages and create lip-sync animations for each.

Refining lip-sync animation

To refine lip-sync animation:

1. In the Animation Set Editor, select the Lower Face and Tongue controls for your character. As you select the controls, you can see the mouth sliders appear on the Controls panel.
2. Go into the Motion Editor. You can see the motion data for the character's mouth sliders.
3. Right-click in the Motion Editor, and click Show Waveform. Now you can see the waveform of the dialog clip behind the facial animation curves.
4. Play with the presets to adjust your lip-sync animation. For example, you can use Smooth to smooth out the movements.
5. Next to the Procedural tab, click the Phoneme tab.
6. Play with the phoneme presets to further refine your lip-sync animation.

Tip:If you don't need to hear the audio as you work, click the mute button on the upper right of the timeline.

Automatic lip-sync animation for custom models

Phoneme extraction will only work on the HWM TF2 classes, and HL2 FACS-compliant models.

For other models, here is a YouTube tutorial that explains how to create your own phoneme table.

To enable HL2 models (that are FACS-compliant) to accept phoneme mapping:

Warning:The Element Viewer is very powerful, and has very few safeguards. If you use it incorrectly, you can irretrievably corrupt your scene and crash the SFM. It's a good idea to stay out of it until you are sure you understand what you're doing.

1. Right-click on the AnimationSet of the model you wish to enable phoneme mapping on
2. Select "Show in Element Viewer > AnimationSet"
3. Switch to the Element Viewer
4. Right click on the root of the AnimationSet (ie: Alyx1) and select "Create Phoneme Presets based on HL2 FACS controls".

See also

• Facial animation