Parallax Corrected Cubemaps

This article or section should be converted to third person to conform to wiki standards.

Note:Some areas of this article use personal pronouns (e.g. "I", "we've") and in general could use a little bit more professionalism

For the third-party Source brush entity responsible for setting cubemap boundaries, see parallax_obb.

For the Source 2 point entity responsible for both creating both the cubemap and parallax bounding box, see env_cubemap_box.

Overview

Source's native cubemap implementation does not allow them to follow the player's perspective. While it is passable in most cases, tying the reflections to player's view can increase their realism, especially when using high-resolution reflections. A way to do it is called parallax correction.

Parallax-corrected cubemaps use a bounding box brush to bake their reflection based on a specified area around them, and a custom shader to make use of it.

This tutorial is based on the work of Brian Charles. The video of its effect can be found here:  Parallax Corrected Cubemaps in the Source Engine.

The Code

First, this file

• matrixinvert.h

Needs to be added to <src code directory>/src/utils/vbsp/

Next, go to your project solution (everything.sln) and open /utils/vbsp/cubemap.cpp.

cubemap.cpp

Add somewhere below the SideHasCubemapAndWasntManuallyReferenced( int iSide ) function:

char *g_pParallaxObbStrs[MAX_MAP_CUBEMAPSAMPLES];

Then right below, change the CubeMap_InsertSample( ... ) function declaration to look like this:

void Cubemap_InsertSample( const Vector &origin, int size, char *pParallaxObbStr = "" )

In the body of the function, at the very top add:

g_pParallaxObbStrs[g_nCubemapSamples] = pParallaxObbStr;

Change the PatchEnvmapForMaterialAndDependents( ... ) function declaration to:

static bool PatchEnvmapForMaterialAndDependents( const char *pMaterialName, const PatchInfo_t &info, const char *pCubemapTexture, const char *pParallaxObbMatrix = "" )

Change the code inside the if ( pDependentMaterial ) statement to:

	bDependentMaterialPatched = PatchEnvmapForMaterialAndDependents( pDependentMaterial, info, pCubemapTexture, pParallaxObbMatrix );

Below it, change

	MaterialPatchInfo_t pPatchInfo[2];

to:

	MaterialPatchInfo_t pPatchInfo[6];

Further below, locate the line:

	char pDependentPatchedMaterialName[1024];

Add right above it:

	// Parallax cubemap matrix
	CUtlVector<char *> matRowList;
	if ( pParallaxObbMatrix[0] != '\0' )
	{
		V_SplitString( pParallaxObbMatrix, ";", matRowList );
	
		pPatchInfo[nPatchCount].m_pKey = "$envMapParallaxOBB1";
		pPatchInfo[nPatchCount].m_pValue = matRowList[0];
		++nPatchCount;
		pPatchInfo[nPatchCount].m_pKey = "$envMapParallaxOBB2";
		pPatchInfo[nPatchCount].m_pValue = matRowList[1];
		++nPatchCount;
		pPatchInfo[nPatchCount].m_pKey = "$envMapParallaxOBB3";
		pPatchInfo[nPatchCount].m_pValue = matRowList[2];
		++nPatchCount;
		pPatchInfo[nPatchCount].m_pKey = "$envMapOrigin";
		pPatchInfo[nPatchCount].m_pValue = matRowList[3];
		++nPatchCount;
	}

At the bottom of the function locate this line:

	CreateMaterialPatch( pMaterialName, pPatchedMaterialName, nPatchCount, pPatchInfo, PATCH_REPLACE );

Change it to:

	CreateMaterialPatch( pMaterialName, pPatchedMaterialName, nPatchCount, pPatchInfo, PATCH_INSERT );

Then below that add this:

	// Clean up parallax stuff
	matRowList.PurgeAndDeleteElements();

Scroll down to the Cubemap_CreateTexInfo( ... ) function declaration and change it to:

static int Cubemap_CreateTexInfo( int originalTexInfo, int origin[3], int cubemapIndex )

Inside the function, locate the line:

	GeneratePatchedName( "c", info, false, pTextureName, 1024 );

Add this below it:

	// Append origin info if this cubemap has a parallax OBB
	char originAppendedString[1024] = "";
	if ( g_pParallaxObbStrs[cubemapIndex][0] != '\0' )
	{
		Q_snprintf( originAppendedString, 1024, "%s;[%d %d %d]", g_pParallaxObbStrs[cubemapIndex], origin[0], origin[1], origin[2] );
	}

Further below, find the statement:

	if ( !PatchEnvmapForMaterialAndDependents( pMaterialName, info, pTextureName ) )

Change it to:

	if ( !PatchEnvmapForMaterialAndDependents( pMaterialName, info, pTextureName, originAppendedString ) )

Below, find the Cubemap_FixupBrushSidesMaterials( void ) function, and the line:

	pSide->texinfo = Cubemap_CreateTexInfo( pSide->texinfo, g_CubemapSamples[cubemapID].origin );

Change it to this:

	pSide->texinfo = Cubemap_CreateTexInfo( pSide->texinfo, g_CubemapSamples[cubemapID].origin, cubemapID );

Scroll down to the Cubemap_AttachDefaultCubemapToSpecularSides( void ) function, and inside it find this line:

	pSide->texinfo = Cubemap_CreateTexInfo( pSide->texinfo, g_CubemapSamples[iCubemap].origin );

Change it to this:

	pSide->texinfo = Cubemap_CreateTexInfo( pSide->texinfo, g_CubemapSamples[iCubemap].origin, iCubemap );

This is all for cubemap.cpp.

vbsp.h

Find the declaration:

void Cubemap_InsertSample( const Vector& origin, int size );

Change it to:

void Cubemap_InsertSample( const Vector &origin, int size, char *pParallaxObbStr );

Above that, add:

extern char *g_pParallaxObbStrs[MAX_MAP_CUBEMAPSAMPLES];

This is all for vbsp.h.

map.cpp

At the very top, add this include:

#include "matrixinvert.h"

That is the file downloaded earlier.

Then scroll down to the LoadEntityCallback( ... ) function. Find the if statement in it -

if( !strcmp( "env_cubemap", pClassName ) )

And inside it find this line:

	const char *pSideListStr = ValueForKey( mapent, "sides" );

Add below it:

	char *pParallaxObbStr = ValueForKey( mapent, "parallaxobb" );

Then, find the line:

	Cubemap_InsertSample( mapent->origin, size );

Replace it with:

	Cubemap_InsertSample( mapent->origin, size, pParallaxObbStr );

After the closing bracket of the if statement that code was in, add:

	//
	// parallax_obb brushes are removed after the transformation matrix is found and saved into 
	// the entity's data (ent will be removed after data transferred to patched materials)
	//
	if ( !strcmp( "parallax_obb", pClassName ) )
	{
		matrix3x4_t obbMatrix, invObbMatrix;
		SetIdentityMatrix( obbMatrix );
		SetIdentityMatrix( invObbMatrix );

		// Get corner and its 3 edges (scaled, local x, y, and z axes)
		mapbrush_t *brush = &mapbrushes[mapent->firstbrush];
		Vector corner, x, y, z;

		// Find first valid winding (with these whiles, if not enough valid windings then identity matrix is passed through to VMTs)
		int i = 0;
		while ( i < brush->numsides )
		{
			winding_t *wind = brush->original_sides[i].winding;
			if ( !wind )
			{
				i++;
				continue;
			}

			corner = wind->p[0];
			y = wind->p[1] - corner;
			z = wind->p[3] - corner;
			x = CrossProduct( y, z ).Normalized();

			i++;
			break;
		}

		// Skip second valid winding (opposite face from first, unusable for finding Z's length)
		while ( i < brush->numsides )
		{
			winding_t *wind = brush->original_sides[i].winding;
			if ( !wind )
			{
				i++;
				continue;
			}
			i++;
			break;
		}

		// Find third valid winding
		while ( i < brush->numsides )
		{
			winding_t *wind = brush->original_sides[i].winding;
			if ( !wind )
			{
				i++;
				continue;
			}

			// Find length of X
			// Start with diagonal, then scale X by the projection of diag onto X
			Vector diag = wind->p[0] - wind->p[2];
			x *= abs( DotProduct( diag, x ) );

			// Build transformation matrix (what is needed to turn a [0,0,0] - [1,1,1] cube into this brush)
			MatrixSetColumn( x, 0, obbMatrix );
			MatrixSetColumn( y, 1, obbMatrix );
			MatrixSetColumn( z, 2, obbMatrix );
			MatrixSetColumn( corner, 3, obbMatrix );

			// Find inverse (we need the world to local matrix, "transformationmatrix" is kind of a misnomer)
			MatrixInversion( obbMatrix, invObbMatrix );
			break;
		}
			
		char szMatrix[1024];
		Q_snprintf( szMatrix, 1024, "[%f %f %f %f];[%f %f %f %f];[%f %f %f %f]", invObbMatrix[0][0], invObbMatrix[0][1], invObbMatrix[0][2], invObbMatrix[0][3], invObbMatrix[1][0], invObbMatrix[1][1], invObbMatrix[1][2], invObbMatrix[1][3], invObbMatrix[2][0], invObbMatrix[2][1], invObbMatrix[2][2], invObbMatrix[2][3] );
		SetKeyValue( mapent, "transformationmatrix", szMatrix );

		return ( ChunkFile_Ok );
	}

Further down, find the LoadMapFile( ... ) function, inside it, find this line:

	g_LoadingMap->CheckForInstances( pszFileName );

Below that line, add:

	if ( g_MainMap == g_LoadingMap )
	{
		// Fill out parallax obb matrix array
		for ( int i = 0; i < g_nCubemapSamples; i++ )
		{
			if ( g_pParallaxObbStrs[i][0] != '\0' )
			{
				entity_t *obbEnt = EntityByName( g_pParallaxObbStrs[i] );
				g_pParallaxObbStrs[i] = ValueForKey( obbEnt, "transformationmatrix" );
			}
		}
		
		// Remove parallax_obb entities (in a nice slow linear search)
		for ( int i = 0; i < g_MainMap->num_entities; i++ )
		{
			entity_t *mapent = &g_MainMap->entities[i];
			const char *pClassName = ValueForKey( mapent, "classname" );
			if ( !strcmp( "parallax_obb", pClassName ) )
			{
				mapent->numbrushes = 0;
				mapent->epairs = NULL;
			}
		}
	}

That is all for map.cpp.

The VBSP project should now compile without any errors. If there are, double-check everything.

Once vbsp.exe is compiled, move it to the game's bin/ folder (not the mod's bin/; the game's - typically the same one where Hammer is located).

Warning:Steam often replaces custom vbsp.exe with its original version whenever it verifies the file integrity of the game.

Workaround:To prevent the loss of the custom vbsp.exe, rename it, then point to it in Hammer's game configuration.

The Shaders

Now, the fun stuff: shader editing! The Parallax Corrected Cubemaps that will now be patched into your materials need a custom LightmappedGeneric shader for them to be properly displayed.

Let's start simple. To edit shaders, you'll need to open your everything.sln file. Bonus points if you already had it open!

Warning:This section is the lengthiest and most tedious, just due to having to compile a custom LightmappedGeneric shader, which can take upwards of 30 minutes per compile, even on beefy computers!

Note:It's recommended to follow the Shader Authoring tutorial to get a feel for compiling shaders. This tutorial assumes you're comfortable with that.

Shader C++ Files

Firstly, let's get the LightmappedGeneric shader familiar with the Parallax Corrected bounding boxes you stored in the material.

lightmappedgeneric_dx9_helper.h

Add the following lines to the end of the LightmappedGeneric_DX9_Vars_t struct (after the int m_nOutlineEnd1; member):

	// Parallax cubemaps
	int m_nEnvmapParallaxObb1;
	int m_nEnvmapParallaxObb2;
	int m_nEnvmapParallaxObb3;
	int m_nEnvmapOrigin;

lightmappedgeneric_dx9.cpp

Now we need the implementation file to populate these shader parameters. Add the following lines inside of the

BEGIN_SHADER_PARAMS

block at the top of this file, after the

	SHADER_PARAM( OUTLINEEND1, SHADER_PARAM_TYPE_FLOAT, "0.0", "outer end value for outline")

line:

	// Parallax cubemaps
	SHADER_PARAM( ENVMAPPARALLAXOBB1, SHADER_PARAM_TYPE_VEC4, "[1 0 0 0]", "The first line of the parallax correction OBB matrix" )
	SHADER_PARAM( ENVMAPPARALLAXOBB2, SHADER_PARAM_TYPE_VEC4, "[0 1 0 0]", "The second line of the parallax correction OBB matrix" )
	SHADER_PARAM( ENVMAPPARALLAXOBB3, SHADER_PARAM_TYPE_VEC4, "[0 0 1 0]", "The third line of the parallax correction OBB matrix" )
	SHADER_PARAM( ENVMAPORIGIN, SHADER_PARAM_TYPE_VEC3, "[0 0 0]", "The world space position of the env_cubemap being corrected" )

If you did your homework in the Shader Authoring section, you'll know that this is defining the shader parameters inside of the material file. Now let's assign them to the struct's new members. In the same file, inside of the

void SetupVars( LightmappedGeneric_DX9_Vars_t& info )

function, add the following lines at the end of it (after the info.m_nOutlineEnd1 = OUTLINEEND1; line):

	// Parallax cubemaps
	info.m_nEnvmapParallaxObb1 = ENVMAPPARALLAXOBB1;
	info.m_nEnvmapParallaxObb2 = ENVMAPPARALLAXOBB2;
	info.m_nEnvmapParallaxObb3 = ENVMAPPARALLAXOBB3;
	info.m_nEnvmapOrigin = ENVMAPORIGIN;

Great, now our shader's code knows about our patched parameters. But we still have to do stuff with them!

lightmappedgeneric_dx9_helper.cpp

Firstly, let's do a safety check. We need to ensure we have all the required components for the Parallax Corrected Cubemaps to work properly. Inside of

void InitParamsLightmappedGeneric_DX9( CBaseVSShader *pShader, IMaterialVar** params, const char *pMaterialName, LightmappedGeneric_DX9_Vars_t &info )

near the top of this file, add the following lines of code to the (you guessed it) end of it, after the InitFloatParam( info.m_nOutlineAlpha, params, 1.0 ); line:

	// Parallax cubemaps
	// Cubemap parallax correction requires all 4 lines (if the 2nd, 3rd, or 4th are undef, undef the first one (checking done on first var)
	if ( !( params[info.m_nEnvmapParallaxObb2]->IsDefined() && params[info.m_nEnvmapParallaxObb3]->IsDefined() && params[info.m_nEnvmapOrigin]->IsDefined() ) )
	{
		params[info.m_nEnvmapParallaxObb1]->SetIntValue( 0 );
	}

This allows the parallax-correcting code to not run if there's some issue with your patched materials or something.

Next, we need to tell the shader itself whether the above code we added returns true, and if so, to render the parallax corrected cubemap. Inside the

void DrawLightmappedGeneric_DX9_Internal( ... )

function, after the line

	bool hasNormalMapAlphaEnvmapMask = IS_FLAG_SET( MATERIAL_VAR_NORMALMAPALPHAENVMAPMASK );

add the following boolean declaration:

	// Parallax cubemaps
	bool hasParallaxCorrection = params[info.m_nEnvmapParallaxObb1]->GetType() == MATERIAL_VAR_TYPE_VECTOR;

Now let's check this boolean. Further down this gargantuan function, we need to set the combo on the pixel shader. Inside of the

	if ( g_pHardwareConfig->SupportsPixelShaders_2_b() )

block, add the following line BEFORE the

	SET_STATIC_PIXEL_SHADER( lightmappedgeneric_ps20b );

line:

	// Parallax cubemaps enabled for 2_0b and onwards
	SET_STATIC_PIXEL_SHADER_COMBO( PARALLAXCORRECT, hasParallaxCorrection );

If you're able to read, you can start to realize what we're going to do next: since Shader Model 2.0 was all the rage for the GoldSource days, and something like parallax corrected cubemaps is a little high-tech for it, we need to enable it only if the graphics card running the engine supports Shader Model 2.0b (and higher). So a little further in this mega-function, inside of the else statement for the code above, RIGHT BEFORE the line SET_STATIC_PIXEL_SHADER( lightmappedgeneric_ps20 );, add the following:

	// Parallax cubemaps
	SET_STATIC_PIXEL_SHADER_COMBO( PARALLAXCORRECT, 0 ); // No parallax cubemaps with ps_2_0 :(

So now our pixel shader (which we're editing next section) knows whether or not it should handle the parallax corrected cubemaps.

Lastly, but certainly not least-ly (I mean come on we still haven't even edited the shader yet!), we need to send the pixel shader the proper value to use to display our parallax-corrected cubemap. So, even further down in this hulking hunk of hot garbage function, RIGHT BEFORE the pContextData->m_SemiStaticCmdsOut.End(); line, add the following block of code:

	// Parallax cubemaps
	if ( hasParallaxCorrection )
	{
		pContextData->m_SemiStaticCmdsOut.SetPixelShaderConstant( 21, params[info.m_nEnvmapOrigin]->GetVecValue() );

		float *vecs[3];
		vecs[0] = const_cast<float *>( params[info.m_nEnvmapParallaxObb1]->GetVecValue() );
		vecs[1] = const_cast<float *>( params[info.m_nEnvmapParallaxObb2]->GetVecValue() );
		vecs[2] = const_cast<float *>( params[info.m_nEnvmapParallaxObb3]->GetVecValue() );
		float matrix[4][4];
		for ( int i = 0; i < 3; i++ )
		{
			for ( int j = 0; j < 4; j++ )
			{
				matrix[i][j] = vecs[i][j];
			}
		}
		matrix[3][0] = matrix[3][1] = matrix[3][2] = 0;
		matrix[3][3] = 1;
		pContextData->m_SemiStaticCmdsOut.SetPixelShaderConstant( 22, &matrix[0][0], 4 );
	}

Woo! Now our shader has the right data passed into it. Alright, time to put on your DirectX 8 hats, we're going in!

The Shader Files

lightmappedgeneric_ps2_3_x.h

Now, I know what you're thinking. This isn't anywhere close to a shader file! It's a gosh dang C++ header file! But worry not, this file is included inside of the next file we edit, the actual shader source (FXC) file. So all of this code will actually get compiled into the shader, making it a shader source in disguise!

First things first, let's declare us some registers to snag up. If you're wondering why this shader doesn't work with Shader Model 2.0, here is why. Inside this file, right below

const float4 g_ShadowTweaks : register( c19 );

add:

// Parallax cubemaps
#if ( PARALLAXCORRECT )
const float3 g_CubemapPos : register( c21 );
const float4x4 g_ObbMatrix : register( c22 ); // Through c25
#endif

Don't worry about that PARALLAXCORRECT, it'll make sense in a bit.

Now, let's do the main shader math we need! We can skip further down the file, and find where cubemaps are processed, inside of the if( bCubemap ) block, add the following block of code after the fresnel = fresnel * g_OneMinusFresnelReflection + g_FresnelReflection; line:

	// Parallax correction (2_0b and beyond)
	// Adapted from http://seblagarde.wordpress.com/2012/09/29/image-based-lighting-approaches-and-parallax-corrected-cubemap/
#if !( defined( SHADER_MODEL_PS_1_1 ) || defined( SHADER_MODEL_PS_1_4 ) || defined( SHADER_MODEL_PS_2_0 ) )
#if ( PARALLAXCORRECT )
	float3 worldPos = i.worldPos_projPosZ.xyz;
	float3 positionLS = mul( float4( worldPos, 1 ), g_ObbMatrix );
	float3 rayLS = mul( reflectVect, (float3x3)g_ObbMatrix );

	float3 firstPlaneIntersect = ( float3( 1.0f, 1.0f, 1.0f ) - positionLS ) / rayLS;
	float3 secondPlaneIntersect = ( -positionLS ) / rayLS;
	float3 furthestPlane = max( firstPlaneIntersect, secondPlaneIntersect );
	float distance = min( furthestPlane.x, min( furthestPlane.y, furthestPlane.z ) );

	// Use distance in WS directly to recover intersection
	float3 intersectPositionWS = worldPos + reflectVect * distance;
	reflectVect = intersectPositionWS - g_CubemapPos;
#endif
#endif

Wonderful. Our shader will now properly do fancy maths to parallax the cubemap based on the player's position. Onwards to the FXC file!

lightmappedgeneric_ps2x.fxc

Heck yeah, here we go, shader file editing! Told you we'd get here! Alright, you ready? Brace yourself, these edits are such a doozy.

Add the following line, after // STATIC: "FLASHLIGHT" "0..1" [ps20b] [XBOX]:

// STATIC: "PARALLAXCORRECT" "0..1"

This is declaring the static combo that our code from the previous section declares. But hold on tight, we still got more!

We need to optimize the shader to not compile under certain conditions. For example, when shadercompile is working on the Shader Model 2.0 version, we can tell it not to build the version with the Parallax Corrected Cubemaps inside of it. Similarly, if there isn't even a dang cubemap to begin with, we don't want the parallax-correcting code to run! So after the line // SKIP ($DETAIL_BLEND_MODE == 11 ) && ($BUMPMAP != 0 ), add:

// SKIP: $PARALLAXCORRECT && !$CUBEMAP
// SKIP: $PARALLAXCORRECT [ps20]

That's all there is to it! You can open your eyes now. Actually, hopefully, they were open for all of this tutorial, because otherwise, you will be getting very interesting results.

Compile the Shader(s)

Note:Psst, if you need extra help with this section, look at the Your First Shader page, it's a pretty good resource for compiling shaders!

Now we need to compile the shader. Got some coffee to go make? Some meal to go prepare? Some arbitrary 30-minute task? Good! Compiling a custom LightmappedGeneric shader is going to take roughly that time. Any time you need to change anything with the shader, it'll take 30 minutes to fully recompile it! So, hopefully, this works on the first try.

Inside of your stdshader_dx9_20b.txt file (or your custom game one, I don't judge), make sure that the following shaders are added:

lightmappedgeneric_ps2x.fxc
lightmappedgeneric_ps11.fxc
lightmappedgeneric_vs20.fxc

Now, compile. Go do that task I was asking you about. It's gonna take a while because it's a Perl script compiling one of the more complex shaders, so it has a lot of combinations to go through. It's going to use all of your resources, too, so don't get spooked. Is it winter? Good, it'll warm up your room.

When it completes, it will generate the proper lightmappedgeneric_*.inc files that your code uses, and you should have your lightmappedgeneric_*.vcs files inside of the shaders/fxc/ subdirectory. You can copy these over to your game's shaders/fxc/ folder.

With your new .inc files, you should be able to compile the game_shader_dx9 project. With the project compiled, you will have a nifty game_shader_dx9.dll to copy over to your game's bin/ folder, alongside your server and client DLLs.

What Next?

The FGD File

parallaxcubes.fgd:

@include "YOUR FGD HERE. THIS ONE WILL OVERRIDE THE REGULAR ENV_CUBEMAP ENTITY AND ALSO ADD THE PARALLAX_OBB ENTITY. ONLY INCLUDE THIS ONE IN HAMMER."

@PointClass color(0 0 255) sidelist(sides) iconsprite("editor/env_cubemap.vmt") = env_cubemap : 
	"An entity that creates a sample point for the Cubic Environment Map."
[
	cubemapsize(choices) : "Cubemap Size" : 0 =
	[
		0 : "Default"
		1 : "1x1"
		2 : "2x2"
		3 : "4x4"
		4 : "8x8"
		5 : "16x16"
		6 : "32x32"
		7 : "64x64"
		8 : "128x128"
		9 : "256x256"
	]
	sides(sidelist) : "Brush faces": : "(Optional) Brushes faces to directly attach to the env_cubemap. Press Pick then click on faces in the 3D View to select them. Use CTRL while clicking to add or remove from the selection."
	parallaxobb(target_destination) : "Cubemap Bounds" : : "(Optional) assigns this cubemap a bounding box for parallax correction (brush entity tied to parallax_obb)."
]

@SolidClass = parallax_obb
[
	targetname(target_source) : "Name" : : "The name that other entities refer to this entity by."
]

(Optional) Test Map

Creation of Parallax Corrected Cubemaps can be found in Parallax Corrected Cubemaps/Creation article.

Todo: Test Map coming soon

Conclusion

And that's it, try launching your mod to see if it works!