Shader Authoring

From Valve Developer Community
Revision as of 17:41, 8 April 2005 by Erik Johnson (talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


This document describes how to author shaders in the Source SDK.

Note: Authoring new shaders is for advanced users. It requires a working knowledge of C++, Microsoft Visual Studio, and some familiarity with DirectX. It also assumes some familiarity with the Source SDK directory layout. This document refers to filenames with shorthand, like:


Refers to "C:\Program Files\Valve\Steam\steamapps\username\sourcesdk_content\hl2\mapsrc\X.vmf", if you have installed Steam into "C:\Program Files\Valve\Steam"

Every material that is used in Source specifies which shader it uses to render itself. The shader contains the logic and equations to take the source artwork and lighting, and produce the final rendered color for every pixel that the shader is used on.

The Source SDK fully supports Microsoft DirectX HLSL (High Level Shading Language) and shader assembly languages for writing shaders. We recommend using HLSL whenever possible, and only writing shader assembly as a last resort.

Getting Started

The Source SDK relies on some external tools to build shaders. This section describes how to download and install the tools you'll need in order to compile shaders.

Installing Perl

To install Perl, go to the download page on , download, and install Perl.

Installing DirectX

To install the latest DirectX SDK, go to Microsoft's DirectX site, download, and install DirectX.

Copying Files

Now that you have Perl and DirectX installed, you can copy out the required executables so the SDK compilation tools can use them. The table below assumes that you have installed Perl into C:\Perl, DirectX into C:\DXSDK, and your mod's source code directory is in C:\MyMod\src. Replace the directory names as appropriate.

Source File Destination Directory
C:\Perl\bin\perl.exe C:\MyMod\src\devtools\bin
C:\Perl\bin\perl58.dll C:\MyMod\src\devtools\bin
C:\DXSDK\Utilities\fxc.exe C:\MyMod\src\dx9sdk\Utilities
C:\DXSDK\Utilities\psa.exe C:\MyMod\src\dx9sdk\Utilities
C:\DXSDK\Utilities\vsa.exe C:\MyMod\src\dx9sdk\Utilities

Environment Setup

You'll also need to have the nmake.exe from Microsoft Visual Studio .NET 2003 in your path. If it isn't already in your path, you can run vcvars32.bat (which would be under your VS.NET directory somewhere like C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin).

High Level Concepts

The two major pieces of code that have to be written to create a new shader are the HLSL code and the C++ code.

The HLSL code is a C-like language in which you describe what happens at each vertex and each pixel for primitives drawn with your shader. HLSL code is compiled into files that are put under your mod directory's shaders directory. The specifics of HLSL are described in the DirectX docs here.

The C++ code is compiled into a "shader DLL" that fits the pattern of either game_shader_dx*.dll or game_shader_generic*.dll. The shader DLL goes in your mod's bin directory (the same place where your client.dll and server.dll goes). The C++ code describes the high-level operation of the shader to the Source Engine. It tells the engine things like:

  • Which textures the shader wants to use (texture names usually come from the .VMT [material] file).
  • How many rendering passes the shader will use.
  • Which HLSL code the shader will use.
  • Which parameters to pass into the HLSL code from the material file.
  • What shader to fallback to if the user's system can't support the shader.

Every shader has one C++ class and one or more HLSL files that it uses to render.

Quick Start - Sample Shaders

The Source SDK ships with some sample shaders that you can compile and use right away. To compile them and use them in your game:

1. Install a copy of the source code by running the Create a Mod from the SDK launcher. If you've already installed the latest version of the source code, you can skip this step. Note: In this document, we refer to the mod's source directory as C:\MyMod\src (replace it with yours as appropriate).

2. Open a Windows command prompt, and use cd to change the C:\MyMod\src\sdkshaders directory. Then type:

build_sample_shaders.bat -game [mod directory]

This will compile the sample HLSL code and copy the compiled code into a directory called shaders inside your mod's directory.

3. Open game_shader_generic_sample.vcproj in Visual Studio .NET and build it. This will build the C++ part of each shader into a DLL and it will automatically copy the DLL into your game's bin directory (where your client.dll and server.dll are stored).

4. Now you can refer to the new shaders at the top of your material (.VMT) files. For example, you could have a material like this:

	"$basetexture" "Brick/brickwall003a"
	"$surfaceprop" "brick"

5. The SDK includes some files you can copy into your mod to see how to refer to the sample shaders you just compiled. Note: Using the shaders you just compiled is just like referring to any other shader. See Creating Materials for more information. The relevant sample files are listed below:

Filename Use
[steam]\half-life 2\hl2\maps\sdk_shader_samples.bsp Small map that refers to the shaders
[steam]\sourcesdk_content\hl2\mapsrc\sdk_shader_samples.vmf Map source
[steam]\half-life 2\hl2\materials\sdk\sdk_lightmap.vmf Material sample 1
[steam]\half-life 2\hl2\materials\sdk\sdk_particle.vmf Material sample 2

Note: These files are automatically installed by the SDK, but you will need to copy them into the same relative locations in your mod's own folder. For example, if your Steam directory was in C:\Program Files\Valve\Steam, and your mod was called MyMod, you would need to copy:

C:\Program Files\Valve\Steam\steamapps\username\half-life 2\hl2\materials\sdk\sdk_lightmap.vmf


C:\Program Files\Valve\Steam\steamapps\SourceMods\MyMod\materials\sdk\sdk_lightmap.vmf

Anatomy Of Shader DLL Code

A shader DLL should contain one .CPP file for each shader. Inside the .CPP file, a set of macros is used to define the structure of the shader. These macros are listed below. For reference, here is a sample shader's code that you can refer to while reading the macros:

#include "BaseVSShader.h"

// Note: you have to run buildshaders.bat to generate these files from the FXC code.
#include ""
#include ""

BEGIN_VS_SHADER( SDK_Lightmap, "Help for SDK_Lightmap" )

		SHADER_PARAM( BUMPMAP, SHADER_PARAM_TYPE_TEXTURE, "shadertest/BaseTexture", "base texture" )
		SHADER_PARAM( BUMPFRAME, SHADER_PARAM_TYPE_INTEGER, "0", "frame number for $bumpmap" )

	// Set up anything that is necessary to make decisions in SHADER_FALLBACK.
		if( !params[BUMPFRAME]->IsDefined() )
			params[BUMPFRAME]->SetIntValue( 0 );

		return 0;

	// Note: You can create member functions inside the class definition.
	int SomeMemberFunction()
		return 0;

		LoadTexture( BASETEXTURE );

			// Enable the texture for base texture and lightmap.
			pShaderShadow->EnableTexture( SHADER_TEXTURE_STAGE0, true );
			pShaderShadow->EnableTexture( SHADER_TEXTURE_STAGE1, true );

			sdk_lightmap_vs20_Static_Index vshIndex;
			pShaderShadow->SetVertexShader( "sdk_lightmap_vs20", vshIndex.GetIndex() );

			sdk_lightmap_ps20_Static_Index pshIndex;
			pShaderShadow->SetPixelShader( "sdk_lightmap_ps20", pshIndex.GetIndex() );

			pShaderAPI->BindLightmap( SHADER_TEXTURE_STAGE1 );
BEGIN_VS_SHADER( [shader name], [help string] ) / END_SHADER

This macro defines the name of the shader, as referenced in .VMT files. It expands to a class definition, so you can create member functions inside.


These macros define the section in which your material parameters are defined. Briefly, the material parameters are the variables that your shader can read out of a .vmt file.

SHADER_PARAM( [param name], [param type], [default value], [help string] )

Each of these defines a parameter in your shader, and the parameter values are specified inside .VMT files. Any code you write in your shader can refer to the values of these parameters (as specified in the .VMT file) by referring to params[param name] (which will be of type IMaterialVar).

There are a number of default shader parameters that are automatically present in any shader. See Default Shader Parameters for a list of these.

For example, you could have a SHADER_PARAM like this:

SHADER_PARAM( LIGHT_COLOR, SHADER_PARAM_TYPE_VEC3, "1 0 0", "This is the directional light color." )

and a .VMT material file like this:

"(your shader name here - whatever was in BEGIN_VS_SHADER)"
	"$light_color" "0 0 1"

and then you could write params[LIGHT_COLOR]->GetVecValue() anywhere in your shader code to use the color.

See src\public\materialsystem\IMaterialVar.h for the IMaterialVar interface, and see src\public\materialsystem\IMaterialSystem.h - ShaderParamType_t - for a list of the parameter types that are supported.


The code inside this block is called right after the values for the parameters are loaded from the .vmt file. It gives the shader a chance to validate and clamp the incoming parameters, and to set default values for them if needed.


The code inside this block detects what DirectX version the user is running (using g_pHardwareConfig, and based on the DirectX version and what material parameters are specified, the shader can decide to use another shader to render the current material.

This is useful if you have a high-end shader that makes use of the very latest HLSL version. If a user with an older DirectX 7 compatible machine were to try to run your shader, it wouldn't work because their machine couldn't support it. In that case, you would return the name of a "lesser" shader that will run on the person's machine.

Note: Sometimes, a material may go through a chain of SHADER_FALLBACKuntil it finds one that will support it. If all the versions of LightmappedGeneric were compiled into one shader DLL, and a user with a DirectX 6 video card were to use a LightmappedGeneric shader, it would call them in this sequence:

LightmappedGeneric -> LightmappedGeneric_DX8 -> LightmappedGeneric_DX6


The code in this block loads the shader's textures, bumpmaps, and cubemaps, and initializes its shader flags. Briefly, textures can be loaded with LoadTexture, bumpmaps with LoadBumpMap, and cubemaps with LoadCubeMap.


The code inside the SHADER_DRAW block contains all the state settings.


These macros always are specified one after the other, and they both are responsible for configuring all the rendering parameters for your shader. See Shadow And Dynamic State for more information.

Shadow And Dynamic State

The meat and potatoes of a shader is inside SHADOW_STATE and DYNAMIC_STATE blocks. Both blocks are responsible for specifying all the rendering parameters necessary to render the shader. The difference between the two is that shadow state is only called once per material, so it can not change dynamically. Dynamic state is called every time something with a specific material is rendered, so it can change its parameters over time. The material system makes a distinction between the two for optimization purposes.

Note that each SHADOW_STATE/DYNAMIC_STATE pair is always followed by a Draw() call. Each SHADOW_STATE/DYNAMIC_STATE/Draw() sequence represents one rendering pass for your shader. You can use multiple rendering passes to achieve blending effects or to use more textures than you otherwise would be able to.

Shadow State

Most shadow state code uses pShaderShadow (of type IShaderShadow) to set its state. For a comprehensive list of the things pShaderShadow supports, look at IShaderShadow in src\public\materialsystem\ishaderapi.h. The table below summarizes the primary things done the SHADOW_STATE block.

Specification Description
pShaderShadow->EnableTexture For each texture that you want to use in your shader, you need to enable that texture in shadow state. Each texture stage corresponds to a sampler inside your pixel shader's .FXC file (SHADER_TEXTURE_STAGE0 corresponds to sampler 0, and so on..)
pShaderShadow->SetVertexShader pShaderShadow->SetPixelShader This is where you specify which .FXC (or .VSH/.PSH if you're using shader assembly) files your shader will use for its vertex shader and pixel shader. Usually, you'll have one .FXC file associated with each shader C++ class, but sometimes you'll want to choose a different .FXC file, and this call gives you the flexibility to do so. See Combo Variables for a description of the second parameter to these functions.
pShaderShadow->VertexShaderVertexFormat This call specifies which vertex components your .FXC code needs in order to operate.

The first parameter is a combination of the VertexFormatFlags_t flags listed in src\public\materialsystem\imaterial.h. These correspond roughly with the functions in CMeshBuilder (see src\public\materialsystem\imesh.h) used to specify vertex data.

The second and third parameters specify how many texture coordinates your shader uses. The second parameter says how many texture coordinates you use, and the (optional) third parameter specifies how many components you'll use per texture coordinate (if you pass NULL for this parameter, then it will assume 2 components per texture coordinate). This also corresponds with your CMeshBuilder::TexCoordXXX calls. For example, if you need a 2D texture coordinate and also a 3D one, the code drawing the primitives would look like this:

meshBuilder.TexCoord2f( 0, s, t );
meshBuilder.TexCoord3f( 1, x, y, z ); 

Then your call to VertexShaderVertexFormat would look like this:

int texCoordDimensions = { 2, 3 };
int numTexCoords = 2;
pShaderShadow->VertexShaderVertexFormat( vertexFlags, numTexCoords, texCoordDimensions, 0, 0 ); 

The fourth parameter to VertexShaderVertexFormat specifies the number of bone weights. For skinned models, this should be set to 3. For all other shaders, it can be left at 0.

The last parameter specifies a user data size, and can be either 0 or 4. It corresponds to the data passed into CMeshBuilder::UserData, and can be accessed inside the VS_INPUT structure in your vertex shader like this:

float4 vUserData : TANGENT; 
DefaultFog DisableFog These functions control how your shader uses fogging.
pShaderShadow->EnableBlending pShaderShadow->BlendFunc These functions control how your shader blends with the background (ie: translucency effects). By default, blending is disabled, so whatever your shader draws is opaque. You can use these to turn on translucency and different blending modes. When you enable translucency, all pixels produced by your shader go through this equation:
[output color] = [your shader's output color] * SRC_FACTOR + [the destination pixel's current color] * DEST_FACTOR 

The two parameters to BlendFunc specify what SRC_FACTOR and DEST_FACTOR are. They are of typeShaderBlendFactor_t (see src\public\materialsystem\ishaderapi.h). The most common way to use translucency is standard alpha blending, where the alpha inside your texture specifies how opaque your texture is (an alpha of 1 means fully opaque and an alpha of 0 means fully translucent). In that case, you would want the equation to be:

[output color] = [texture color] * [texture alpha] + [destination pixel's color] * [1 - texture alpha] 

and the code to produce that would be:

pShaderShadow->EnableBlending( true );

Note that SHADER_BLEND_SRC_ALPHA refers to the alpha produced by your pixel shader, which can be any value you want. Sometimes it's as simple as the alpha that was in one of the textures.

pShaderShadow->EnableDepthWrites This function specifies whether your shader writes to the Z buffer or not. Usually, translucent things do not write to the Z buffer, because if they did and something tried to draw behind them, it would get masked out by the Z buffer.

Dynamic State

Code inside the DYNAMIC_STATE block specifies rendering parameters like the SHADOW_STATE block does, but the parameters that it sets are things that are allowed to change over time. The table below lists the most common categories of things that are done inside a DYNAMIC_STATE block. For the complete list, look at the IShaderDynamicAPI class in src\public\materialsystem\ishaderapi.h.

Specification Description

pShaderAPI->BindLightmap BindWhite BindBlack

BindGrey These functions all tell which textures are bound to which texture samplers. They all take a parameter of type TextureStage_t, which tells which texture stage (or sampler in the pixel shader) you want the specified texture assigned to.

BindTexture takes a second parameter that tells which texture to assign to the texture stage. The variable is one of your shader parameters (of type SHADER_PARAM_TYPE_TEXTURE). BindTexture also takes an optional third parameter which is another shader parameter that tells which frame of the texture you want to bind (if the texture has multiple frames built into it).



These functions are used to set the vertex and pixel shader constants.



SetVertexShaderTextureTransform dumps a texture transform into two vertex shader constants, starting at the one you specify in the first parameter. The second parameter specifies the texture transform, which should be specified in one of your shader parameters (of type SHADER_PARAM_TYPE_MATRIX).

The two vertex shader constants are setup with a 2x2 rotation/scaling matrix on the left, and the translation on the right:

constant 1: [rot] [rot] 0 [u translation] 
constant 2: [rot] [rot] 0 [v translation] 

As an example, the code below would bind the default shader parameter called BASETEXTURETRANSFORM:


Then, your .FXC code would use it like this:

const float4 cBaseTexCoordTransform[2] : register( SHADER_SPECIFIC_CONST_0 );
baseTexCoord.x = dot( i.texCoord, cBaseTexCoordTransform[0] ) + cBaseTexCoordTransform[0].w;
baseTexCoord.y = dot( i.texCoord, cBaseTexCoordTransform[1] ) + cBaseTexCoordTransform[1].w; 

SetVertexShaderTextureScaledTransform is just like SetVertexShaderTextureTransform, but it takes a third parameter of type SHADER_PARAM_TYPE_FLOAT, which is a scaling factor for the matrix.



These functions are used to set the values of the dynamic combo variables that your shader supports. The values passed to these functions come from the classes in your shader include file.

Combo Variables

The second parameter to IShaderShadow::SetVertexShader and IShaderShadow::SetPixelShader is the shader index, which is how the Source engine allows you to have conditionals inside your shaders. HLSL currently does not support branching code. It supports if() statements, but only for static conditionals (usually based on #defines - things that can't change after the shader has been compiled).

Without dynamic if() statements in shaders, it's very easy to wind up having to hand-code every variation of a shader, and you can end up with a lot of manual labor. For example, if you have a shader that has an option to multiply the color from a third texture after it multiplies the base texture and lightmap, you'd have to copy the contents of your pixel shader into a separate .FXC file, add the code to multiply the third texture, and choose that file with pShaderShadow->SetPixelShader. That's with 1 option. With 10 orthogonal options, you'd have 2^10 (1024) shader files you'd have to manage.

To simulate dynamic conditionals in HLSL code, the Source engine uses a special syntax in its .FXC files to denote combo variables which can be different across different executions of the same .FXC file. Under the hood, the engine does the dirty work of precompiling 2^n variations of the .FXC file, each with different #defines used for the if() statements.

To make a combo variable, add a line like this to the top of your .FXC file:


or this:


where VERTEXCOLOR is replaced with the name of the variable. Use STATIC whenever possible - static combo variables can only be set once per material (at load time). The values of the static combo variables are set inside the SHADOW_STATE block in your shader's C++ class.

If you need to dynamically change the value of the combo variables (based on a proxy), use DYNAMIC. The values of DYNAMIC combo variables are set inside the DYNAMIC_STATE part of your shader's C++ class.

Note: The '0..1' part following the shader can be any range of values (like '5..10'). Be aware that the number of permutations (and thus memory usage and runtime overhead) rapidly increases as you add more possible dynamic shader variable values. For example, these lines:

// STATIC: "STATIC_VAR_1" "0..1"
// STATIC: "STATIC_VAR_2" "0..1"
// STATIC: "STATIC_VAR_3" "0..1"
// STATIC: "STATIC_VAR_4" "0..1" 

would cause 16 different versions of your .FXC file to be compiled (2^4 or one for each possible combination of valid values).

This, however:

// STATIC: "STATIC_VAR_1" "0..9"
// STATIC: "STATIC_VAR_2" "0..9"
// STATIC: "STATIC_VAR_3" "0..9"
// STATIC: "STATIC_VAR_4" "0..9" 

would cause 10,000 (10^4) different versions of your .FXC file to be compiled, which will not only take a long time to compile, it will also eat up lots of memory. You can use SKIP statements to optimize this.

Important - shader include files

When you compile your .FXC code, a file called fxctmp9\[FXC filename].inc is created. This file contains classes that you use to access the combo variables. For an example, if you look at src\sdkshaders\sdk_lightmap.cpp, it has this include statement at the top:

#include "" 

That include file defines two classes. One is for the combo variables inside a SHADOW_STATE block (combo variables denoted with // STATIC in the .FXC file), and the other is for combo variables specified inside a DYNAMIC_STATE block (denoted with // DYNAMIC inside the .FXC file).

The names of the two classes defined inside a shader include (.inc) file are [shader name]_Static_Index and [shader name]_Dynamic_Index. To tie this all together, let's say you were writing a vertex shader in a file called MyVertexShader.fxc, and it had some STATIC and DYNAMIC combo variables at the top of your .FXC file that looked like this:

// STATIC: "STATIC_VAR_1" "0..1"
// STATIC: "STATIC_VAR_2" "0..1"

// STATIC: "DYNAMIC_VAR_1" "0..1"
// STATIC: "DYNAMIC_VAR_2" "0..1" 

Then, at the top of your .cpp file for your shader, you would have this line:

#include "" 

Then, inside your SHADOW_STATE block, where you would say pShaderShadow->SetVertexShader, you would write this code:

MyVertexShader_Static_Index vshIndex;

vshIndex.SetSTATIC_VAR_1( 0 or 1 );
vshIndex.SetSTATIC_VAR_2( 0 or 1 );
pShaderShadow->SetVertexShader( "MyVertexShader", vshIndex.GetIndex() );

and inside your DYNAMIC_STATE block, you would write this code.

MyVertexShader_Dynamic_Index vshIndex;

vshIndex.SetDYNAMIC_VAR_1( 0 or 1 );
vshIndex.SetDYNAMIC_VAR_2( 0 or 1 );
pShaderAPI->SetVertexShaderIndex( vshIndex.GetIndex() ); 

==SKIP Statements==

SKIP statements can be used to tell Source's shader compiler about values of combo variables that will never be used. For example, if you had 3 combo variables: 

// STATIC: "STATIC_VAR_1" "0..1"
// STATIC: "STATIC_VAR_2" "0..1"
// STATIC: "STATIC_VAR_3" "0..1" 

but you knew that STATIC_VAR_1 and STATIC_VAR_2 could never both have a value of 1 at the same time, you would add a SKIP statement like this:


Then the compiler wouldn't have to bother compiling and storing the two extra combinations of variables.

Valve's LightmappedGeneric shader saves thousands of unnecessary versions of its .FXC file by using SKIP statements.

Default Shader Parameters

In addition to the SHADER_PARAM definitions in your shader's C++ file, there are a bunch of parameters that are automatically defined because they are so common. For example, most shaders have a 'main texture' that they use, so there is a shader parameter called BASETEXTURE. The list of default shader parameters is:

Name Parameter Type Default value
BASETEXTURETRANSFORM SHADER_PARAM_TYPE_MATRIX center .5 .5 scale 1 1 rotate 0 translate 0 0

Compiling .FXC Files

To compile .FXC files, list them in a text file (one per line), then run src\materialsystem\stdshaders\BuildShaders.bat on the text file. It will go process your .FXC code and compile all the combinations into your mod's directory. The syntax of the BuildShaders.bat command line is:

BuildShaders.bat [text filename] -game [mod directory] -source [source directory]


BuildShaders.bat MyShaders.txt -game C:\Steam\SteamApps\SourceMods\MyMod -source C:\MyMod\Src

Note: The text file that lists the .FXC files should be in the same directory as your shader DLL source code, because your shader DLL source code will need to refer to files that are generated by BuildShaders.bat.

Before building your shader DLL, you need to run BuildShaders.bat, so it can generate include files that your shader DLL will need.

You can look in src\sdkshaders\build_sample_shaders.bat for an example of how it's setup. sdk_shaders.txt specifies all the .FXC files, all the source code for the shader DLL is in the same directory, and the .INC files are placd in the fxctmp9 directory. After BuildShaders.bat runs, you can look in the fxctmp9 directory to see the include files generated by teh shader compiler.

To Be Continued...

The remainder of this document is not complete yet. Keep watching it for updates.

For More Information

You can get more information about HLSL and shader assembly programming in the online MSDN docs.

There are more documents describing how the material system works under the Material System section of the Source SDK Documentation Index.