Source Shader Editor - Basic usage

This article will show you the basic functionality of the editor with the help of a tutorial for a simple desaturation post processing effect. Start off with an empty canvas that only contains the semantic nodes. If you need to clean up you can do so by opening the File menu and selecting New and Scratch.

Vertex shader

The vertex shader (VS) for a post processing effect can be very simple. Although it's generally a good idea to relocate as many operations as possible into a VS, because it's usually being executed far less often than the pixel shader (PS). In this example the VS will be called 4 times for drawing a quad primitive, while the number of times the PS will be executed varies by your resolution (width * height). The results sent to the pixel shader input will be linearly interpolated on the current triangle based on the position of the output pixel.

For a post processing effect it is sufficient to use a passthrough VS without any vector transformations (which you'd need to project geometry into the scene for example). Create a node with swizzle operation by clicking with the right mouse button on the canvas and selecting New node -> Math (Vectors) -> Swizzle. Double click the newly created node to open its properties. Enter the string "x y z 1" and click Ok.

Note:make sure that you use spaces when swizzling channels with constants).

Left click and drag the jack with the label Position of the VS Input node to begin a new bridge. While still holding down left mouse button, move the cursor above the In jack of the swizzle node you just created. Release the left mouse button to connect the bridge. Create another bridge between the swizzle output jack (which should by now be labeled "x y z 1") and the Position jack of the VS Output node. Build the next bridge between the jacks named TexCoords 0 of the VS Input and VS Output nodes.

The compiler should now start and silently create a VS preview shader if the VS was connected properly.

Tip:A rotating image in the top right corner is shown when the compiler is busy.

How the respective graph of this tutorial should look like.

Pixel shader

The goal is to read the current framebuffer, calculate the luminosity value for each pixel and desaturate the output based on the luminosity scale.

To begin, create a Texture sample node which you can find in the context submenu Textures. In its properties dialog select Framebuffer from the Texture type drop down menu and click Ok. Create a bridge between the UV jack of this node and the TexCoords 0 jack of the PS Input node.

Tip:Spawn a map to make the preview on the framebuffer texture sample node working.

The luminosity value can be calculated with a dot product of the scene color RGB values and the constants "0.3 0.59 0.11". Create a Dotproduct node from the menu item New node -> Math (Vectors) -> Dot and a local constant through New node -> Constants -> Local constant. Double click the Constant node you just created and enter the string "0.3 0.59 0.11". Connect its output to the jack labeled B on the Dotproduct node and create a bridge between the RGB jack of the Texture sample node and the jack labeled A of the Dotproduct node.

Note:The preview will be red because the dotproduct will render to a single channel, which is the red channel.

Create a Lerp node through New node -> Math -> Lerp. Connect two bridges from your Dotproduct output to each, the Lerp input B and frac. Add another bridge which originates from your RGB jack of the Texture sample node to the Lerp input labeled A. This will blend between full color and grayscale based on the luminosity.

The color output of the PS is a four component float, so you can go ahead and use another swizzle node like you did before in the VS to convert the Lerp output or create an Append node from New node -> Math (Vectors) -> Append and add the original alpha value back. For the latter, connect the Lerp output to the first jack of the Append node and build a bridge between the jack labeled A of your Texture sample node and the second jack of the Append node.

Now connect the output of your last created node to the jack named Color 0 of the PS Output node. The shader is now finished and can be previewed.

Testing the shader

If the compile was successful, spawn a map if you haven't already. Click on Toggle preview on the editor root panel and select Post process from the dropdown menu. You can now either hide the editor with F3 and test the post processing effect fullscreen or use the preview window.

You should now save the shader with a custom name, for example desaturate, via Save as from the File menu. Now select Full compile from the Shader menu to create shader sources, compiled vcs files and implementation information that persists after you closed the game.

Note:Full compile works respective to the current file name, so you should save before using Full compile.

Implementing the shader

To make a shader usable through vmts you have to flag it for precaching. This is only possible if you did a full compile for your shader as described above. Open the precache dialog by selecting Shader precache from the Shader menu. Click on Add shader and select the implementation info of your shader, for example desaturate.dump and close the dialog.

Note:The preload list will only be read on startup, so make sure to restart your game before trying to use shaders from the list that you just added.

Note:Only a full compile will affect already precached shaders.

Creating the vmt

Referencing a shader that you created through the editor in a material is slightly different from the normal scheme. The actual game shader used in the vmt file has to be "editor_shader" and a vmt parameter defines which shader implementation and associated files to load. Accessing the desaturate shader used above would look as follows:

"editor_shader"
{
	"$shadername"	"desaturate"
}

Warning:For shaders that are supposed to be on brushes, make sure that you use a default shader in the vmt when compiling the map (e.g. LightmappedGeneric), otherwise vrad will not generate any lightmaps! Furthermore make sure that this shader uses or does not use $bumpmap respective to the editor shader.

Invoking shaders

After you have assigned your shader for preloading and created a vmt you can now simply render this post processing example as any other.

For example, open viewpostprocess.cpp and navigate to this function:

void DoEnginePostProcessing( int x, int y, int w, int h, bool bFlashlightIsOn, bool bPostVGui )

Now add this snippet to the very end of said function, assuming your vmt is located at "materials/desaturate.vmt":

static IMaterial *pMat = materials->FindMaterial( "desaturate", TEXTURE_GROUP_OTHER );
if ( pMat )
{
	pMat->AddRef();
	UpdateScreenEffectTexture();
	pRenderContext->DrawScreenSpaceRectangle( pMat, 0, 0, w, h,
						0, 0, w - 1, h - 1,
						w, h );
}

Invoking post-processing effects

The editor interface will allow you to manipulate any precached post processing effects through various functions. All of them will use an index for faster access, so you have to call the following function first to get the index:

int IVShaderEditor::GetPPEIndex( const char *pszName );

If this function returns -1 the effect could not be found, make sure that you have it in your precache list.

By default, the editor will draw all effects when you call 'g_ShaderEditorSystem->CustomPostRender(); in viewrender.cpp. Since this might not be the ideal place for whatever you are trying to accomplish, disable the effect in the editor first (either call void IVShaderEditor::SetPPEEnabled( const int &index, const bool &bEnabled ); or uncheck start enabled in the precache dialog) and then draw the effect manually by calling the following function:

void IVShaderEditor::DrawPPEOnDemand( const int &index, const bool bInScene = false );

Error handling

There are 3 general types of errors you may encoutner when creating shaders:

• Flow graph errors

These errors are easy to spot since nodes with conflicts will visualize their state with their current color. Make sure that all your inputs are valid and the properties of the node in question are not incomplete.

• Capacity errors

Certain operations will use up shader registers, samplers or other limited resources. Should you pass these limits, a red warning will be shown in the top left corner of the canvas. Remove the nodes that caused the error and either change the design of your shader or find another workaround. These errors will hinder the compiler from starting.

Tip:You can make all limited capacity information permanently visible by selecting Editor config in the File menu and checking Additional info.

• Compile errors

Compile errors will be visualized by a red blinking outline of the whole canvas. Open the console and read the compile output to get more information on the issue. You may need to open the temporary source file of your shader and locate the error by hand, which can be a bit difficult; the file can be found in ../mod/shadereditorui/shader_src.

Screenshots

You can create a highres image of the canvas by selecting Take screenshot in the File menu. This operation may fail depending on your system and environment. The file will be saved as nodeview.tga in your mods folder, the old file will be overridden.

Example shaders

Some example files can be found in the default canvas directory. You can easily tell by their prefix how they are meant to be used (postproc_ for post processing effects, model_ for models and lightmap_ for world brushes, make sure to apply lightmap shaders with bumpmapping only to surfaces that have been compiled with bumpmapping). However they are merely meant for demonstration and some of them must not be used in a mod straight away (postproc_sunrays, postproc_flare_anamorphic), you should to draw them onto an RT that has one quarter of the backbuffer resolution, apply post blur and combine them additively. Furthermore some may rely on an env_sun entity on the map and won't work properly without one.