Lip-sync animation and Extract Phonemes
The SFM can extract phonemes from a dialog clip to create a preliminary lip-sync animation that you can then refine. You can create lip-sync animations for both existing dialog clips and new ones you record yourself.
To create lip-sync animation for a dialog clip:
- Right-click the dialog clip, and click Show Waveform. Note that only waveforms with Show Waveform selected will be used for automatic lip-sync.
- Hide the waveforms for other sound clips by right-clicking them and clicking Hide Waveform.
- In the Animation Set Editor, expand the character's animation set.
- Right-click any of the character's controls, and click Extract Phonemes. (You can take the defaults until you know how you want to modify them.)
- Click Extract.
You can now play back your movie to see your new lip-sync animation.
Refining lip-sync animation
To refine lip-sync animation:
- In the Animation Set Editor, select the Lower Face and Tongue controls for your character. As you select the controls, you can see the mouth sliders appear on the Controls panel.
- Go into the Motion Editor. You can see the motion data for the character's mouth sliders.
- Right-click in the Motion Editor, and click Show Waveform. Now you can see the waveform of the dialog clip behind the facial animation curves.
- Play with the presets to adjust your lip-sync animation. For example, you can use Smooth to smooth out the movements.
- Next to the Procedural tab, click the Phoneme tab.
- Play with the phoneme presets to further refine your lip-sync animation.
Automatic lip-sync animation for custom models
Phoneme extraction will only work on the HWM TF2 classes, and HL2 FACS-compliant models.
For other models, here is a YouTube tutorial that explains how to create your own phoneme table.
To enable HL2 models (that are FACS-compliant) to accept phoneme mapping: Warning: The Element Viewer is very powerful, and has very few safeguards. If you use it incorrectly, you can irretrievably corrupt your scene and crash the SFM. It's a good idea to stay out of it until you are sure you understand what you're doing.
- Right-click on the AnimationSet of the model you wish to enable phoneme mapping on
- Select "Show in Element Viewer > AnimationSet"
- Switch to the Element Viewer
- Right click on the root of the AnimationSet (ie: Alyx1) and select "Create Phoneme Presets based on HL2 FACS controls".