Flex animation

Were you looking for Faceposer help?

Flex shapes in action. TF2 characters in have two versions, pre-authored expressions (regular models) and the HWM (hardware morph) set of shapes. Half-Life 2 characters are based on the FACS, Facial Action Coding System

Flex animation (also called blend shape and shape key animation) is the direct manipulation of vertices without the involvement of a skeleton. It is generally used to create facial expressions and lip sync, but can be applied anywhere on the model; Left 4 Dead 2's infected can have their entire head shape changed through flexes.

Premise

A flex shape consists of a set of altered vertex positions for a mesh. Each vertex in a flex has exactly one alternate position. Some vertices may remain unaltered, and thus retain their original position. Together, these alternate positions change the shape of the mesh, creating a character's smile, frown, blink, etc.

The animation comes into play when the mesh interpolates between its base shape and the flex shape, which is facilitating by a flex controller. When the flex controller's weight is non-zero, the mesh performs a simple linear translation of its base vertices towards the alternate vertex positions in the flex shape. As the flex weight increases, the vertices translate further, and the flex shape becomes more pronounced. At a weight of 1.0, the vertices have translated all the way to their alternate positions, and the flex shape is at full intensity. Weights between 0.0 and 1.0 translate the vertices partially, resulting in a subdued flex shape. Weights greater than 1.0 exaggerate the flex shape by translating vertices into extrapolated territory past their alternate positions. Negative weights invert the flex shape by translating vertices in the opposite direction.

Using flexes

Achieving flex animation involves animating the weights of flex controllers to smoothly transition between various flex shapes or combinations of flex shapes. For example, a character may transition from a wide smile to a small frown, by animating the weight of its "smile" flex from 1.0 to 0.0 while also animating the weight of its "frown" flex from 0.0 to 0.5 at the same time. In most Source engine branches and games, flex controller weights are animated by special constructs, notably Faceposer-produced choreography files and $boneflexdrivers in skeletal animation. Some offshoots, like Source Filmmaker, allow direct manipulation of flex controller weights by the user.

Primitive flex shapes are typically combined to produce complex final expressions and mouth movements. For example, a character's concerned expression might be produced by increasing the weights of its "frown", "lowered eyebrows", and "thin lips" flexes. Valve's so-called hardware morph models from Team Fortress 2 are a popular example of Source's ability to do this at runtime, though Valve rarely uses them in their games and instead prefers to use pre-combined complex shapes that were produced by their character artists.

Extensions

The concept and implementation of flex animation was recycled into vertex animation, a technique in which a mesh's vertices animate through multiple alternate positions. Since a single flex only stores a single set of alternate vertex positions, vertex animation uses a series of multiple flexes to animate a mesh's vertices through complex motions, like that of cloth.

Limitations

Flex animation has some important limitations in Source:

• Each flex can only move vertices in straight lines
• Flex animation cannot alter collision models
• No more than 10,000 vertices can be flexed per mesh

SMD has these additional limits:

• Flex animation with the VTA format is not compatible with $scale ("unmatched vertex anims").
• Vertices cannot move more than 8 units (in any or all axis)

DMX has these:

• If a shape name contains underscores, that shape becomes corrective. Don't put underscores in flexes that you want to expose to animators.

VTA format

The VTA format stores the morph shape keys separate from the mesh itself. It was used by Valve until the Orange Box. In Valve's newer games, facial animation is entirely defined within the DMX file of the mesh. There is no equivalent in .QC notation for some of the rules that take place in there. Refer to the DMX section down the page to see what has been figured out so far.

Authoring

Maya, Blender, 3DS Max, and XSI can export shape keys to VTA files. See each exporter's documentation for further details.

Valve standard flex

If you are creating a new humanoid character, your best bet is to use the same flex animation rules as Valve. These implement the Facial Action Coding System, a long-standing and widely used method of describing the full range of human expressions.

Confirm:Lip synch requires a set of FACS flexes.

You can find the scripts at sourcesdk_content/hl2/modelsrc/humans_sdk/Male_sdk/. There are two: standardflex_xsi.qci and facerules_xsi.qci. (bodyrules_xsi.qci is related, but as the name implies does not affect the face.)

For the scripts to work, you must have created and exported these standard shape keys in the order given. Then use this QC template:

$definevariable expressions "MyShapes.vta"
$definevariable headBone "ValveBiped.Bip01_Head1"

$eyeposition 0 0 70

$attachment "eyes" $headBone$ 0.043 -4.2197 67.5554 absolute
$attachment "mouth" $headBone$ 1.00 -6.30 0.00 rotate 0 -80 -90


$model facs_example "MyReference.smd" {
	eyeball righteye $headBone$ -1.2711 -4.2197 67.5593 "eyeball_r" 1 4 "pupil_r" 0.63
	eyeball lefteye $headBone$ 1.3572 -4.2197 67.5514 "eyeball_l" 1 -4 "pupil_l" 0.63

	eyelid  upper_right $expressions$ lowerer 1 -0.2621 neutral 0 0.1287 raiser 2 0.2467 split 0.1 eyeball righteye
	eyelid  lower_right $expressions$ lowerer 3 -0.3409 neutral 0 -0.2156 raiser 4 -0.0736 split 0.1 eyeball righteye
	eyelid  upper_left $expressions$ lowerer 1 -0.2621 neutral 0 0.1287 raiser 2 0.2467 split -0.1 eyeball lefteye
	eyelid  lower_left $expressions$ lowerer 3 -0.3409 neutral 0 -0.2156 raiser 4 -0.0736 split -0.1 eyeball lefteye

	mouth 0 "mouth" $headBone$ 0 1 0     // mouth illumination

	flexfile $expressions$ {
		$include "../standardflex_xsi.qci"
	}

	$include "../facerules_xsi.qci"
	// $include "../bodyrules_xsi.qci"

}

See also $model and Eyeball.

Compiling

You should have a exported a VTA and a reference SMD from your modelling package.

$model flextest "myreference.smd" {		// must use $model, not $body, and "{" must be on the same line
	flexfile "myflexanim.vta" {		// source of vertex animations
		defaultflex frame 0		// relaxed position
		flex "Frame1" frame 1
		flex "Frame2" frame 2
	}
	flexcontroller my_group "Flex1" "Flex2"		// defines controllers that will appear in Faceposer etc.
	%Frame1 = Flex1		// assigns a controller to a flex
	%Frame2 = Flex2
}

This defines two flexes and maps them directly onto two controllers.

Bug:HLMV's flex slider boxes are populated from right to left. You will need to resize the window to see them all.  [todo tested in?]

Defining flexes

Raw flexes are extracted from VTA frames, and support some preprocessing. They are not exposed by the model (flex controllers, below, are).

flex <name> frame <int> [position <normal>] [split <units>] [decay <normal>]

Used within a flexfile block to define a single shape. There can be up to 1024.

name

Internal name of the flex.

frame

The VTA frame the flex refers to.

position

The flex controller position (See also flexcontroller::range) at which this flex will reach full intensity.

split

Makes the flex read only vertices on one side of the mesh's Y origin. The value is the number of units (positive or negative) over which to smooth the divide. 0 disables.

decay

How fleshy the flex looks when animating. Vertex speed is a factor of distance moved: with the default of 1 those that move the most do so instantly, while those that move the least take 0.7 seconds to fully settle.

At 0, there is no lag on even the smallest movements. At over 1, the farthest-moving vertices start to lag too.

flexpair <name> <int> frame <int> [<flex options>]

Same as flex, but automatically creates two flexes with "L" and "R" appended to their names. The unlabelled integer is the equivalent of the split command (split itself is ignored).

defaultflex frame <int> [<flex options>]

Defines the model's relaxed position. The flex created is called "default".

Defining controllers

flexcontroller <group name> [range <normal> <normal>] <controller name> [<controller name> ... ]

An input into the model, used to create animations. Takes the form of a slider. There can be up to 96.

<group name>

Seen with values like eyelid, brow, nose, mouth, and phoneme. Required, but has no apparent effect.

range

Defines the low and high slider values (default 0 and 1). This does not affect the flex itself, but can be used together with flex's position value. Reversing the values makes the slider reverse, not the flex.

<controller name>

As many display names as needed. A controller will be created for each.

NoAutoDMXRules

Although the name of this command resambles to disable flexcontroller only for DMX, it tell to compile parser to remove all flexcontroller that were previously declared until its QC block. Which means it also affects every $model, $body, and $bodygroup command that appeared before. This is useful for DMX avoid hit 96 flexcontroller limit along custom face rules.

Note:Only available where DMX format is avaliable.

Tip:Try use it at first lines of your $model block, before any Eyeball or eyelid command.

$model my_model "head.dmx" {
	NoAutoDMXRules
	flexcontroller my_group range 0 1 my_flexcontroller
	%Flex = my_flexcontroller
}

Assigning flexes to controllers

It can be very simple:

%myflex = myflexcontroller

Or it can be very complex:

%upper_right_raiser = right_lid_raiser * (1 - right_lid_droop * 0.8) * (1 - right_lid_closer) * (1 - blink)
%brow_mid_raiser = max(min(brow_left_raiser, brow_right_raiser), max(brow_left_raiser, brow_right_raiser) * 0.5)

The following operators are supported:

• Multiplication (*)
• Division (/)
• Addition (+)
• Subtraction (-)
• Taking the lower of two values (min(val1,val2))
• Taking the higher of two values (max(val1,val2))

In all cases, either static numbers or variable/flex/controller names can be used. Flexes will never exceed their position value.

Note:Assigning is done outside the flexfile block, but still inside $model.

Bug:Negative values will cause the engine to crash!  [todo tested in?]

Tip:localvar <name> can be used to store results of an equation for re-use later. Once you've defined one, just do %mylocalvar = val.

%mouth

This is a special variable that is read by the mouth shader. When it is 1, the mouth interior is fully illuminated.

LOD

To disable flex, add nofacial to an $lod block.

There is no need to create shapes for your LOD meshes; studiomdl will transfer them from the reference mesh as appropriate.

Stereo flexes (one slider with L/R control)

In order to be combined into a single "stereo" slider with left/right controls in Source Filmmaker or Faceposer, flexes must follow a particular naming convention; they must begin with left_ and right_. Here's an example:

	flexfile "this_is_an_example.vta" {
	  flex     "eyebrowClench" frame 1
	  flexpair "eyebrowRaise" 1.0 frame 2
	  flexpair "eyebrowFurrow" 1.0 frame 3
	  flexpair "outerEyebrowRaise" 1.0 frame 4
	}

	flexcontroller brow eyebrowClench "range" 0.000 1.000
	flexcontroller brow right_eyebrowRaise left_eyebrowRaise "range" 0.000 1.000
	flexcontroller brow right_eyebrowFurrow left_eyebrowFurrow "range" 0.000 1.000
	flexcontroller brow right_outerEyebrowRaise left_outerEyebrowRaise "range" 0.000 1.000
	
	%eyebrowClench = eyebrowClench	 
	%eyebrowRaiseL = left_eyebrowRaise
	%eyebrowRaiseR = right_eyebrowRaise
	%eyebrowFurrowL = left_eyebrowFurrow
	%eyebrowFurrowR = right_eyebrowFurrow
	%outerEyebrowRaiseL = left_outerEyebrowRaise
	%outerEyebrowRaiseR = right_outerEyebrowRaise

Here, eyebrowClench is a "mono" flex, while eyebrowRaise, eyebrowFurrow, and outerEyebrowRaise will be "stereo".

Warning:Your flexcontrollers need to be declared with right/left, not left/right, or else stereo flexes will be offset by one forwards in the list of all flexes, producing rather strange results.

DMX format

DMX models store both shape data and flex controller definitions in the same file as the reference mesh.

• There are currently four exporters with DMX flex support: Valve's official Maya plugin, the Blender Source Tools, Wall Worm Model Tools, and GameZombie's SMD/DMX Tools (currently the only toolset for 3DS Max with Import support).
• See DMX model#Flex controllers for a technical description of the flex controller format.
• Open one of the "_high" TF2 DMX source files in a text editor to see how Valve set them up.

There are four key components of DMX flex controllers:

DmeCombinationInputControl

A flex controller exposed to animators.

DmeCombinationDominationRule

Used to defines shapes which fade out when other shapes are active.

e.g. "open jaw" might dominate "puff cheeks", since it's impossible to do both at once.

DmeFlexRuleExpression

Shape key pre-processor that accept a simple equation. Domination rules are a simpler alternative.

DmeFlexRulePassThrough

If flex rules are used, all shapes must have one. In this scenario, this type can be used on shapes which don't need any pre-processing.

DMX supports 128 flex controllers and an unknown (but much higher) number of shapes.

Corrective shapes

Shape keys called "[shape name 1]_[shape name 2]_[etc]" will fade in when the shapes they are named after are active at the same time. For example "openJaw_openLips" will fade in whenever both the "openJaw" and "openLips" shapes are active.

Corrective shapes are applied additively as shown by this table:

Corrective shape selection

Active shapes	Corrective shapes applied	No. active shapes
a + b	a_b	3
a + b + c	a_b + a_c + b_c + a_b_c	7

When authoring corrective shapes make sure that you have all of the relevant shapes active before you start to sculpt.

Tip:Corrective shapes don't need to be made for combinations prevented by a domination rule.

Tip:Corrective shapes can have the same "master" flex specified more than once as a driver (for example "[shape1]_[shape1]"), allowing the corrective flex to have a quadratic, cubic or even quartic response to the master flex. This method can allow a single flex controller to appear to move vertices along a curved path (rather than just straight lines) by fading in several different shapes (that combine to create the overall target shape) at different rates.

TF2's HWM ("hardware morph") models use a standard set of 50 shapes and 35 controllers and about 100 (!!) corrective shapes made on an "if you see it break" basis.

Dmxedit

Dmxedit is the tool used by Valve to post-process DMX shape keys and create flex controllers.

Using

Faceposer

See Choreography creation/Creating Events/Facial expressions.

Code

^[Todo]