Shader Authoring: Difference between revisions

From Valve Developer Community
Jump to navigation Jump to search
No edit summary
 
(67 intermediate revisions by 22 users not shown)
Line 1: Line 1:
{{shadertut}}
{{LanguageBar}}{{shadertut}}


This document describes how to author shaders in the Source SDK, for further information on what a shader is and how they can be used, please see [[Shader]].
This document describes how to author shaders in the Source SDK, for further information on what a shader is and how they can be used, please see [[Shader]].


==Getting Started==
==Getting Started==
Line 7: Line 7:
Every material that is used in Source specifies which shader it uses to render itself. The shader contains the logic and equations to take the source artwork and lighting, and produce the final rendered color for every pixel that the shader is used on.
Every material that is used in Source specifies which shader it uses to render itself. The shader contains the logic and equations to take the source artwork and lighting, and produce the final rendered color for every pixel that the shader is used on.


The Source SDK fully supports Microsoft DirectX [[HLSL]] (High Level Shading Language) and shader assembly languages for writing shaders. It is recommended to use HLSL whenever possible and to only write shader assembly as a last resort.
The Source SDK fully supports Microsoft DirectX [[HLSL]] (High-Level Shading Language) and shader assembly languages for writing shaders. It is recommended to use HLSL whenever possible and to only write shader assembly as a last resort.


There are a number of external tools required to compile shaders for use in the Source SDK, the following sections detail the tools and provides links for download along with brief instructions on installation and usage.
There are a number of external tools required to compile shaders for use in the Source SDK, the following sections detail the tools and provide links for download along with brief instructions on installation and usage.


The tools used are Perl, the DirectX SDK and make, the following processes, in general, only have to be performed once before the process of compiling shaders can begin.
The tools used are Perl, the DirectX SDK and make, the following processes, in general, only have to be performed once before the process of compiling shaders can begin.


===Installing Perl and DirectX===
==Options for Building Shaders==
To install Perl, go to the [http://www.perl.com/download.csp#win32 download page] on http://www.perl.com , download, and install Perl - a recommended version of Perl for windows is [http://www.activeperl.com ActivePerl].
{{important| After the 2013MP SDK got updated with TF2's code, a modified version of SCell555's shader compiler is included by default, more precisely [https://github.com/misyltoad/ShaderCompile2 this] version.}}
===Option 1 - Perl & DirectX SDK===


The Orange Box SDK also relies on the non-standard perl package, String-CRC32. If you are using Active Perl, you can install it through the Perl Package Manager which can be launched from <PerlDir>\bin\ppm.bat
====Perl====
To install Perl, go to the [https://www.perl.com/download.csp#win32 download page] on https://www.perl.com/, download, and install Perl - a recommended version of Perl for Windows is [https://strawberryperl.com/ Strawberry Perl].
 
The Orange Box SDK also relies on the non-standard Perl package, String::CRC32. If you are using Strawberry Perl, you can install it through the CPAN Client which can be launched from <code><StrawberryPerlInstallDir>\perl\bin\cpan.bat</code>
 
Enter the following command to install the package: <code>install String::CRC32</code>
 
{{note|If you have problems with cpan.bat, try to open PowerShell in <code><StrawberryPerlInstallDir>\perl\bin\</code> and run command: <code>cpan String::CRC32</code>}}
 
 
====DirectX SDK====
To install the November 2008 DirectX SDK, go to [https://archive.org/details/dxsdk_nov08 https://archive.org/details/dxsdk_nov08], download, and install the DirectX SDK (The old Microsoft link is dead, so we need to use the Wayback Machine).


To install the latest DirectX SDK, go to [http://www.microsoft.com/windows/directx Microsoft's DirectX site], download, and install the DirectX SDK. Be sure to get the latest possible version of the SDK, if a previous version is installed, remove it and download a more recent version.


{{note|Just having DirectX 9.x installed is '''not''' sufficient, the <code>DirectX SDK</code> must be installed!}}
{{note|Just having DirectX 9.x installed is '''not''' sufficient, the <code>DirectX SDK</code> must be installed!}}
{{note|<b>It is recommended that you use the November 2008 SDK as needed binaries have been removed from newer releases</b>}}
{{note|'''It is recommended that you use the November 2008 SDK as needed binaries (psa.exe, vsa.exe) have been removed from newer releases.'''}}
 


===Setting The Path===
====Setting The Path====
With Perl and the DirectX SDK installed, make sure your "Path" environmental variable has been updated appropriately.  In Windows, you can check this by right clicking on properties, going to the Advanced tab, and clicking on "Environment Variables".  Assuming that Perl is installed into <code>C:\Perl</code>, and DirectX into <code>C:\DXSDK</code>, your "Path" variable should contain both <code>C:\Perl\bin</code> and <code>C:\DXSDK\Utilities\bin\x86</code> in a semicolon delimited list.   
With Perl and the DirectX SDK installed, make sure your "Path" environmental variable has been updated appropriately.  In Windows, you can check this by right clicking on My Computer -> properties, going to the Advanced tab, and clicking on "Environment Variables".  Assuming that Perl is installed into <code>C:\Perl</code>, and DirectX into <code>C:\DXSDK</code>, your "Path" variable should contain both <code>C:\Perl\bin</code> and <code>C:\DXSDK\Utilities\bin\x86</code> in a semicolon delimited list.   


By setting up the Environmental Variable in this way, you can update the Perl and DirectX sdk as you please without having to copy files.  If for some reason you do not wish to do this, you can simply copy the required files into a different directory and add that to your path instead.
By setting up the Environmental Variable in this way, you can update the Perl and DirectX SDK as you please without having to copy files.  If for some reason you do not wish to do this, you can simply copy the required files into a different directory and add that to your path instead.


Note that you may need to restart your computer for these changes to the environment to take full effect.
{{note|You may need to restart your computer for these changes to the environment to take full effect.}}


===Shader compiling with newer DirectX SDKs===
Compiling shaders with newer DirectX SDKs from February 2007 and onwards requires a special flag to be set for the fxc.exe file in order to successfully compile shaders.(This is fixes the out of range errors received) And modification to a shader header file.


In the mod's code directory
===Option 2 - SCell555's Shader Compiler===
To install SCell555's Shader Compile tools, you must download the zip file which they provide with every [https://github.com/SCell555/ShaderCompile/releases/latest release].


open \src\devtools\bin\fxc_prep.pl
Once you have downloaded the zip (or .7z) file, open it up and you should see the following folders:
Find:
*bin
<pre>
*headers
$cmd .= "/nologo ";
*stdshaders
</pre>


Add Below:
<pre>
$cmd .= "/LD ";
</pre>
Save the file and it should be all set.


===Fix shadercompile.exe path===
Place the <code>bin</code> folder into the <code>devtools</code> folder.
As of May 08 there is an error introduced in runvmpi.pl and buildsdkshaders.bat by having two copies shadercompile.exe: one for Episode One, and one for the Orange Box.


Episode One is the only engine version that currently supports custom shaders, so you will most likely want to redirect runvmpi.pl to use that version.
Place the <code>stdshaders</code> folder into the <code>materialsytem</code> folder. If it asks you to replace the files, say yes.
Open <moddir>/src/materialsystem/stdshaders/runvmpi.pl and make the following change


<code>
Navigate into the <code>headers</code> folder in the zip. You should see a <code>cshader.h</code> file and a <code>VS2013</code> folder (which also contains another <code>cshader.h</code> file). If you are using Visual Studio 2013 or above, place the <code>cshader.h</code> in the <code>VS2013</code> folder into <code>public\shaderlib</code> and replace itIf you are using a Visual Studio version before 2013, use the {{path|cshader|h}} file that isn't in the <code>VS2013</code> folder instead.
  -$cmdToRun = "\"$ENV{\"sourcesdk\"}\\bin\\shadercompile.exe\" ...
  +$cmdToRun = "\"$ENV{\"sourcesdk\"}\\bin\\ep1\\bin\\shadercompile.exe\" ...
</code>


Then open buildsdkshaders.bat and make the following change
==Completing the Setup==


====Modify stdshader_dx9_20b.txt====
Open up {{path|materialsystem/stdshaders/stdshader_dx9_20b|txt}} and comment out all of the lines which compile shaders like so:
<code>
<code>
  -if not exist "%sourcesdk%\bin\shadercompile.exe" goto NoShaderCompile
  //
  -set ChangeToDir=%sourcesdk%\bin
// Standard shaders collection
  +if not exist "%sourcesdk%\bin\ep1\bin\shadercompile.exe" goto NoShaderCompile
//
  +set ChangeToDir=%sourcesdk%\bin\ep1\bin
//  These shaders are compiled as the following shader models:
//        _ps20.vcs
  //        _ps20b.vcs
  //        _vs20.vcs
//
  //example_model_ps20b.fxc
//example_model_vs20.fxc
//SDK_Bloom_ps2x.fxc
//SDK_screenspaceeffect_vs20.fxc
//SDK_bloomadd_ps2x.fxc
</code>
</code>


You can also update the error message in that file if you like, but it's not strictly necessary.
====Modify buildepisodicshaders.bat====
Open {{path|src\materialsystem\stdshaders\buildepisodicshaders|bat}} and set the GAMEDIR AND SDKBINDIR as instructed.
 
Inside {{path|buildsdkshaders|bat}}, you might also need to modify the call to vsvars32.bat on line 7 - If you are using Visual Studio 2013 it should point to %VS120COMNTOOLS% instead.


If for some reason you want to use the Orange Box version, replace "ep1" with "orangebox" above, and place mysql_wrapper.dll in <account>\sourcesdk\bin\orangebox\binIt is avaliable [http://files.filefront.com/mysql+wrapperdll/;11464432;/fileinfo.html here]
Note: GAMEDIR AND SDKBINDIR must be kept to the 8.3 directory name standard.
<code>
rem == Set the absolute path to your mod's game directory here ==
rem == Note that this path needs does not support long file/directory names ==
rem == So instead of a path such as "C:\Program Files\Steam\steamapps\mymod" ==
rem == you need to find the 8.3 abbreviation for the directory name using 'dir /x' ==
  rem == and set the directory to something like C:\PROGRA~2\Steam\steamapps\sourcemods\mymod ==
set GAMEDIR= YOUR PATH HERE


===Fix common_fxc.h===
rem == Set the relative path to SourceSDK\bin\orangebox\bin ==
As of the July 2008 release, compiling any shader with fxc.exe that uses the include file "common_fxc.h" will result in the following compile error.
rem == As above, this path does not support long directory names or spaces ==
<pre>
rem == e.g. ..\..\..\..\..\PROGRA~2\Steam\steamapps\<USER NAME>\sourcesdk\bin\orangebox\bin ==
common_fxc.h<156,36>: error X3000: syntax error: unexpected token 'linear'
set SDKBINDIR= YOUR PATH HERE
</pre>
</code>


To fix this, simply change all instances of the variable "linear" to any other valid HLSL variable name such as "inLinear"
====Shader Compile====
[[File:compileshadersmd.png|thumb|Example of a successful compile.]]
Navigate into the {{path|materialsystem\stdshaders}} folder. Then type in the name of the bat file for compiling your shaders; it will probably be {{path|buildepisodicshaders|bat}}. If you did everything correctly, it should compile with no issues.
{{warning|You will encounter errors about <code>D3DVERTEXTEXTURESAMPLER0</code> being undefined when using SCell555's compiler.  The fix is to simply remove any references to this definition from <code>register</code> functions in the Vertex Shader files. {{todo|List which files that have <code>register( D3DVERTEXTEXTURESAMPLER0, s0 )</code> in them to make finding the references easier.}} }}
{{warning|SCell's shader compiler doesn't produce any Pixel Shader files below 2.0b, and no Vertex Shader files below 2.0.  The stock SDK shaders rely on fallbacks to lower versions of these files and, thus will need to be modified to account for this limitation.}}


== See also ==
== See also ==
* You can get more information about HLSL and shader assembly programming in [http://msdn2.microsoft.com/en-us/library/bb509561.aspx the online MSDN docs].
* You can get more information about HLSL and shader assembly programming in [https://msdn2.microsoft.com/en-us/library/bb509561.aspx the online MSDN docs].
* [[:Category:Material System]] -- for more general information on the usage of shaders in materials.
* [[:Category:Material System]] -- for more general information on the usage of shaders in materials.
* [http://www.wraiyth.com/index.php?q=node/5 Wraiyths Shader Tutorials]
{{otherlang:en}}
{{otherlang:en:ru|Shader Authoring:ru}}


[[Category: Programming]]
[[Category:Programming]]
[[Category: Shaders]]
[[Category:Shaders]]

Latest revision as of 08:01, 21 February 2025

English (en)Русский (ru)中文 (zh)Translate (Translate)

This document describes how to author shaders in the Source SDK, for further information on what a shader is and how they can be used, please see Shader.

Getting Started

Every material that is used in Source specifies which shader it uses to render itself. The shader contains the logic and equations to take the source artwork and lighting, and produce the final rendered color for every pixel that the shader is used on.

The Source SDK fully supports Microsoft DirectX HLSL (High-Level Shading Language) and shader assembly languages for writing shaders. It is recommended to use HLSL whenever possible and to only write shader assembly as a last resort.

There are a number of external tools required to compile shaders for use in the Source SDK, the following sections detail the tools and provide links for download along with brief instructions on installation and usage.

The tools used are Perl, the DirectX SDK and make, the following processes, in general, only have to be performed once before the process of compiling shaders can begin.

Options for Building Shaders

Icon-Important.pngImportant: After the 2013MP SDK got updated with TF2's code, a modified version of SCell555's shader compiler is included by default, more precisely this version.

Option 1 - Perl & DirectX SDK

Perl

To install Perl, go to the download page on https://www.perl.com/, download, and install Perl - a recommended version of Perl for Windows is Strawberry Perl.

The Orange Box SDK also relies on the non-standard Perl package, String::CRC32. If you are using Strawberry Perl, you can install it through the CPAN Client which can be launched from <StrawberryPerlInstallDir>\perl\bin\cpan.bat

Enter the following command to install the package: install String::CRC32

Note.pngNote:If you have problems with cpan.bat, try to open PowerShell in <StrawberryPerlInstallDir>\perl\bin\ and run command: cpan String::CRC32


DirectX SDK

To install the November 2008 DirectX SDK, go to https://archive.org/details/dxsdk_nov08, download, and install the DirectX SDK (The old Microsoft link is dead, so we need to use the Wayback Machine).


Note.pngNote:Just having DirectX 9.x installed is not sufficient, the DirectX SDK must be installed!
Note.pngNote:It is recommended that you use the November 2008 SDK as needed binaries (psa.exe, vsa.exe) have been removed from newer releases.


Setting The Path

With Perl and the DirectX SDK installed, make sure your "Path" environmental variable has been updated appropriately. In Windows, you can check this by right clicking on My Computer -> properties, going to the Advanced tab, and clicking on "Environment Variables". Assuming that Perl is installed into C:\Perl, and DirectX into C:\DXSDK, your "Path" variable should contain both C:\Perl\bin and C:\DXSDK\Utilities\bin\x86 in a semicolon delimited list.

By setting up the Environmental Variable in this way, you can update the Perl and DirectX SDK as you please without having to copy files. If for some reason you do not wish to do this, you can simply copy the required files into a different directory and add that to your path instead.

Note.pngNote:You may need to restart your computer for these changes to the environment to take full effect.


Option 2 - SCell555's Shader Compiler

To install SCell555's Shader Compile tools, you must download the zip file which they provide with every release.

Once you have downloaded the zip (or .7z) file, open it up and you should see the following folders:

  • bin
  • headers
  • stdshaders


Place the bin folder into the devtools folder.

Place the stdshaders folder into the materialsytem folder. If it asks you to replace the files, say yes.

Navigate into the headers folder in the zip. You should see a cshader.h file and a VS2013 folder (which also contains another cshader.h file). If you are using Visual Studio 2013 or above, place the cshader.h in the VS2013 folder into public\shaderlib and replace it. If you are using a Visual Studio version before 2013, use the 🖿cshader.h file that isn't in the VS2013 folder instead.

Completing the Setup

Modify stdshader_dx9_20b.txt

Open up 🖿materialsystem/stdshaders/stdshader_dx9_20b.txt and comment out all of the lines which compile shaders like so:

//
// Standard shaders collection
//
//  These shaders are compiled as the following shader models:
//        _ps20.vcs
//        _ps20b.vcs
//        _vs20.vcs
//
//example_model_ps20b.fxc
//example_model_vs20.fxc
//SDK_Bloom_ps2x.fxc
//SDK_screenspaceeffect_vs20.fxc
//SDK_bloomadd_ps2x.fxc

Modify buildepisodicshaders.bat

Open 🖿src\materialsystem\stdshaders\buildepisodicshaders.bat and set the GAMEDIR AND SDKBINDIR as instructed.

Inside 🖿buildsdkshaders.bat, you might also need to modify the call to vsvars32.bat on line 7 - If you are using Visual Studio 2013 it should point to %VS120COMNTOOLS% instead.

Note: GAMEDIR AND SDKBINDIR must be kept to the 8.3 directory name standard.

rem == Set the absolute path to your mod's game directory here ==
rem == Note that this path needs does not support long file/directory names ==
rem == So instead of a path such as "C:\Program Files\Steam\steamapps\mymod" ==
rem == you need to find the 8.3 abbreviation for the directory name using 'dir /x' ==
rem == and set the directory to something like C:\PROGRA~2\Steam\steamapps\sourcemods\mymod ==
set GAMEDIR= YOUR PATH HERE
rem == Set the relative path to SourceSDK\bin\orangebox\bin ==
rem == As above, this path does not support long directory names or spaces ==
rem == e.g. ..\..\..\..\..\PROGRA~2\Steam\steamapps\<USER NAME>\sourcesdk\bin\orangebox\bin ==
set SDKBINDIR= YOUR PATH HERE

Shader Compile

Example of a successful compile.

Navigate into the 🖿materialsystem\stdshaders folder. Then type in the name of the bat file for compiling your shaders; it will probably be 🖿buildepisodicshaders.bat. If you did everything correctly, it should compile with no issues.

Warning.pngWarning:You will encounter errors about D3DVERTEXTEXTURESAMPLER0 being undefined when using SCell555's compiler. The fix is to simply remove any references to this definition from register functions in the Vertex Shader files.
Todo: List which files that have register( D3DVERTEXTEXTURESAMPLER0, s0 ) in them to make finding the references easier.
Warning.pngWarning:SCell's shader compiler doesn't produce any Pixel Shader files below 2.0b, and no Vertex Shader files below 2.0. The stock SDK shaders rely on fallbacks to lower versions of these files and, thus will need to be modified to account for this limitation.

See also