Difference between revisions of "Scenes.image"

From Valve Developer Community
Jump to: navigation, search
m
m (Removed div style from code blocks for easier readability)
 
(8 intermediate revisions by 4 users not shown)
Line 3: Line 3:
 
'''''To build a scenes.image, load Faceposer and choose <code>File > Rebuild scenes.image...</code>'''''
 
'''''To build a scenes.image, load Faceposer and choose <code>File > Rebuild scenes.image...</code>'''''
  
{{Warning|Only the VCD files in '''YOUR''' <code>scenes/</code> folder will be built into the image. This means that you will need to extract Valve's scenes from your base game's [[Game Cache File|GCF]]s before compiling if you want to use them. Remember that a lot of scenes used automatically by NPCs are in the original <code>source engine.gcf</code>. (Yes, doing all this sucks!) You ''don't'' need to distribute the extracted VCD files though.}}
+
This will collect all <code>.vcd</code> files located in your [[Game Directory|game directory]]'s <code>scenes</code> folder and compile them to <code>scenes/scenes.image</code>, overwriting the file if it already exists. The raw <code>.vcd</code> files do not need to be present for a compiled <code>scenes.image</code> to work in-game. Image rebuilding gathers scenes in all directories, meaning scenes in a separate path (i.e. any path in gameinfo.txt that isn't the main directory {{confirm}}) will be built into <code>scenes.image</code>.
 +
 
 +
{{Warning|Only the VCD files in the game directory's <code>scenes/</code> folder will be built into the image and it will completely overwrite the existing image, meaning any scenes in the image but not the rebuild would be lost. The raw <code>.vcd</code> files are no longer available in the current versions of most Valve titles. For [[Half-Life 2]], this means that you will need to extract Valve's scenes for Half-Life 2, its episodes, and [[Half-Life 2: Lost Coast|Lost Coast]] from [[Source SDK 2013]]'s <code>sp/game</code> folders.}}
 +
 
 +
{{bug|The file system used to gather VCDs for recompiling <code>scenes.image</code> doesn't seem to react well when a <code>/*</code> token is in {{ent|gameinfo.txt}}<code>'s</code> search paths (e.g. <code><nowiki>|gameinfo_path|custom/*</nowiki></code>), using that as a central game folder and not compiling any of the other directories. This bug is not well understood, but you can work around it by either compiling your scenes in a new <code>scenes</code> folder in the <code>/*</code> directory (e.g. <code>custom/scenes</code>) or temporarily removing said paths from your <code>gameinfo.txt</code>.}}
 +
 
 +
 
  
 
== Bypassing ==
 
== Bypassing ==
  
If you are producing your own binaries, the old method of loading loose files can be reinstated:
+
If you are producing your own binaries, the old method of loading loose files can be reinstated. The following code is based off of existing methods that can be found in the Source SDK. When you find them, copy and paste the code present here. You can paste the whole modified function or measure the differences yourself.
 +
 
 +
Note that this code attempts to load loose files ''before'' searching the image cache, which can be changed.
  
<div style="max-height:20em;overflow:auto;">
 
 
<source lang=cpp>
 
<source lang=cpp>
 
CChoreoScene *CSceneEntity::LoadScene( const char *filename, IChoreoEventCallback *pCallback )
 
CChoreoScene *CSceneEntity::LoadScene( const char *filename, IChoreoEventCallback *pCallback )
Line 53: Line 60:
 
FreeSceneFileMemory( pBuffer );
 
FreeSceneFileMemory( pBuffer );
 
return pScene;
 
return pScene;
}
 
 
}
 
}
 
</source>
 
</source>
Line 110: Line 116:
 
}
 
}
 
</source>
 
</source>
</div>
+
<source lang=cpp>
 +
void PrecacheInstancedScene( char const *pszScene )
 +
{
 +
static int nMakingReslists = -1;
 +
 +
if ( nMakingReslists == -1 )
 +
{
 +
nMakingReslists = CommandLine()->FindParm( "-makereslists" ) > 0 ? 1 : 0;
 +
}
 +
 
 +
if ( nMakingReslists == 1 )
 +
{
 +
// Just stat the file to add to reslist
 +
g_pFullFileSystem->Size( pszScene );
 +
}
 +
 
 +
SceneCachedData_t sceneData;
 +
 
 +
char loadfile[MAX_PATH];
 +
Q_strncpy( loadfile, pszScene, sizeof( loadfile ) );
 +
Q_SetExtension( loadfile, ".vcd", sizeof( loadfile ) );
 +
Q_FixSlashes( loadfile );
 +
 
 +
// Attempt to precache manually
 +
void *pBuffer = NULL;
 +
if (filesystem->ReadFileEx( loadfile, "MOD", &pBuffer, false, true ))
 +
{
 +
g_TokenProcessor.SetBuffer((char*)pBuffer);
 +
CChoreoScene *pScene = ChoreoLoadScene( loadfile, NULL, &g_TokenProcessor, LocalScene_Printf );
 +
if (pScene)
 +
{
 +
for ( int i = 0; i < pScene->GetNumEvents(); i++ )
 +
{
 +
CChoreoEvent *pEvent = pScene->GetEvent(i);
 +
if (pEvent && pEvent->GetType() == CChoreoEvent::SPEAK)
 +
{
 +
CBaseEntity::PrecacheScriptSound( pEvent->GetParameters() );
 +
 
 +
// Precache CC token
 +
if ( pEvent->GetCloseCaptionType() == CChoreoEvent::CC_MASTER &&
 +
pEvent->GetNumSlaves() > 0 )
 +
{
 +
char tok[ CChoreoEvent::MAX_CCTOKEN_STRING ];
 +
if ( pEvent->GetPlaybackCloseCaptionToken( tok, sizeof( tok ) ) )
 +
{
 +
CBaseEntity::PrecacheScriptSound( tok );
 +
}
 +
}
 +
}
 +
}
 +
}
 +
}
 +
else if ( !scenefilecache->GetSceneCachedData( pszScene, &sceneData ) )
 +
{
 +
// This warning was meant for when scenes.image was supposed to be the sole method of loading scenes.
 +
// It's been deactivated as part of the raw file support, as even if this was somehow on the Xbox 360,
 +
// it would never trip anyway because if the file existed, it would've been read earlier.
 +
//if ( developer.GetInt() && ( IsX360() && ( g_pFullFileSystem->GetDVDMode() != DVDMODE_STRICT ) && g_pFullFileSystem->FileExists( pszScene, "GAME" ) ) )
 +
//{
 +
// Warning( "PrecacheInstancedScene: Missing scene '%s' from scene image cache.\nRebuild scene image cache!\n", pszScene );
 +
//}
 +
}
 +
else
 +
{
 +
for ( int i = 0; i < sceneData.numSounds; ++i )
 +
{
 +
short stringId = scenefilecache->GetSceneCachedSound( sceneData.sceneId, i );
 +
CBaseEntity::PrecacheScriptSound( scenefilecache->GetSceneString( stringId ) );
 +
}
 +
}
 +
 
 +
g_pStringTableClientSideChoreoScenes->AddString( CBaseEntity::IsServer(), pszScene );
 +
}
 +
</source>
  
 
This code also fixes a bug where the maximum path length for a scene was longer on the server than on the client.
 
This code also fixes a bug where the maximum path length for a scene was longer on the server than on the client.
 +
 
== Technical information ==
 
== Technical information ==
 
Scenes.image files are little-endian on all platforms, with all offsets being absolute to the beginning.
 
Scenes.image files are little-endian on all platforms, with all offsets being absolute to the beginning.

Latest revision as of 08:59, 19 September 2020

scenes/scenes.image is a binary data blob from which Source 2007 (i.e. The Orange Box engine) loads choreography scenes. It was created to improve performance on consoles: it is loaded into memory in its entirety when the engine starts, which eliminates the need for consoles' slow optical media to seek around on a disk for hundreds of loose .VCD files. It helps on PC too however, so has been ported across.

To build a scenes.image, load Faceposer and choose File > Rebuild scenes.image...

This will collect all .vcd files located in your game directory's scenes folder and compile them to scenes/scenes.image, overwriting the file if it already exists. The raw .vcd files do not need to be present for a compiled scenes.image to work in-game. Image rebuilding gathers scenes in all directories, meaning scenes in a separate path (i.e. any path in gameinfo.txt that isn't the main directory [confirm]) will be built into scenes.image.

Warning.png Warning: Only the VCD files in the game directory's scenes/ folder will be built into the image and it will completely overwrite the existing image, meaning any scenes in the image but not the rebuild would be lost. The raw .vcd files are no longer available in the current versions of most Valve titles. For Half-Life 2, this means that you will need to extract Valve's scenes for Half-Life 2, its episodes, and Lost Coast from Source SDK 2013's sp/game folders.
Bug.png Bug: The file system used to gather VCDs for recompiling scenes.image doesn't seem to react well when a /* token is in gameinfo.txt's search paths (e.g. |gameinfo_path|custom/*), using that as a central game folder and not compiling any of the other directories. This bug is not well understood, but you can work around it by either compiling your scenes in a new scenes folder in the /* directory (e.g. custom/scenes) or temporarily removing said paths from your gameinfo.txt.


Bypassing

If you are producing your own binaries, the old method of loading loose files can be reinstated. The following code is based off of existing methods that can be found in the Source SDK. When you find them, copy and paste the code present here. You can paste the whole modified function or measure the differences yourself.

Note that this code attempts to load loose files before searching the image cache, which can be changed.

CChoreoScene *CSceneEntity::LoadScene( const char *filename, IChoreoEventCallback *pCallback )
{
	char loadfile[MAX_PATH];
	Q_strncpy( loadfile, filename, sizeof( loadfile ) );
	Q_SetExtension( loadfile, ".vcd", sizeof( loadfile ) );
	Q_FixSlashes( loadfile );
	
	void *pBuffer = 0;
	CChoreoScene *pScene;

	int fileSize = filesystem->ReadFileEx( loadfile, "MOD", &pBuffer, true );
	if (fileSize)
	{
		g_TokenProcessor.SetBuffer((char*)pBuffer);
		pScene = ChoreoLoadScene( loadfile, NULL, &g_TokenProcessor, LocalScene_Printf );
	}
	else
	{
		// binary compiled vcd
		pScene = new CChoreoScene( NULL );
		if ( !CopySceneFileIntoMemory( loadfile, &pBuffer, &fileSize ) )
		{
			MissingSceneWarning( loadfile );
			return NULL;
		}
		CUtlBuffer buf( pBuffer, fileSize, CUtlBuffer::READ_ONLY );
		if ( !pScene->RestoreFromBinaryBuffer( buf, loadfile, &g_ChoreoStringPool ) )
		{
			Warning( "CSceneEntity::LoadScene: Unable to load scene '%s'\n", loadfile );
			delete pScene;
			pScene = NULL;
		}
	}
	
	if(pScene)
	{
		pScene->SetPrintFunc( LocalScene_Printf );
		pScene->SetEventCallbackInterface( pCallback );
	}

	FreeSceneFileMemory( pBuffer );
	return pScene;
}
CChoreoScene *C_SceneEntity::LoadScene( const char *filename )
{
	char loadfile[MAX_PATH];
	Q_strncpy( loadfile, filename, sizeof( loadfile ) );
	Q_SetExtension( loadfile, ".vcd", sizeof( loadfile ) );
	Q_FixSlashes( loadfile );

	void *pBuffer = 0;
	CChoreoScene *pScene;

	int fileSize = filesystem->ReadFileEx( loadfile, "MOD", &pBuffer, true );
	if (fileSize)
	{
		g_TokenProcessor.SetBuffer((char*)pBuffer);
		pScene = ChoreoLoadScene( loadfile, this, &g_TokenProcessor, Scene_Printf );
	}
	else
	{
		fileSize = scenefilecache->GetSceneBufferSize( loadfile );
		if ( fileSize <= 0 )
			return NULL;

		pBuffer = new char[ fileSize ];
		if ( !scenefilecache->GetSceneData( filename, (byte *)pBuffer, fileSize ) )
		{
			delete[] pBuffer;
			return NULL;
		}

	
		if ( IsBufferBinaryVCD( (char*)pBuffer, fileSize ) )
		{
			pScene = new CChoreoScene( this );
			CUtlBuffer buf( pBuffer, fileSize, CUtlBuffer::READ_ONLY );
			if ( !pScene->RestoreFromBinaryBuffer( buf, loadfile, &g_ChoreoStringPool ) )
			{
				Warning( "Unable to restore scene '%s'\n", loadfile );
				delete pScene;
				pScene = NULL;
			}
		}
	}

	if(pScene)
	{
		pScene->SetPrintFunc( Scene_Printf );
		pScene->SetEventCallbackInterface( this );
	}

	delete[] pBuffer;
	return pScene;
}
void PrecacheInstancedScene( char const *pszScene )
{
	static int nMakingReslists = -1;
	
	if ( nMakingReslists == -1 )
	{
		nMakingReslists = CommandLine()->FindParm( "-makereslists" ) > 0 ? 1 : 0;
	}

	if ( nMakingReslists == 1 )
	{
		// Just stat the file to add to reslist
		g_pFullFileSystem->Size( pszScene );
	}

	SceneCachedData_t sceneData;

	char loadfile[MAX_PATH];
	Q_strncpy( loadfile, pszScene, sizeof( loadfile ) );
	Q_SetExtension( loadfile, ".vcd", sizeof( loadfile ) );
	Q_FixSlashes( loadfile );

	// Attempt to precache manually
	void *pBuffer = NULL;
	if (filesystem->ReadFileEx( loadfile, "MOD", &pBuffer, false, true ))
	{
		g_TokenProcessor.SetBuffer((char*)pBuffer);
		CChoreoScene *pScene = ChoreoLoadScene( loadfile, NULL, &g_TokenProcessor, LocalScene_Printf );
		if (pScene)
		{
			for ( int i = 0; i < pScene->GetNumEvents(); i++ )
			{
				CChoreoEvent *pEvent = pScene->GetEvent(i);
				if (pEvent && pEvent->GetType() == CChoreoEvent::SPEAK)
				{
					CBaseEntity::PrecacheScriptSound( pEvent->GetParameters() );

					// Precache CC token
					if ( pEvent->GetCloseCaptionType() == CChoreoEvent::CC_MASTER && 
						 pEvent->GetNumSlaves() > 0 )
					{
						char tok[ CChoreoEvent::MAX_CCTOKEN_STRING ];
						if ( pEvent->GetPlaybackCloseCaptionToken( tok, sizeof( tok ) ) )
						{
							CBaseEntity::PrecacheScriptSound( tok );
						}
					}
				}
			}
		}
	}
	else if ( !scenefilecache->GetSceneCachedData( pszScene, &sceneData ) )
	{
		// This warning was meant for when scenes.image was supposed to be the sole method of loading scenes.
		// It's been deactivated as part of the raw file support, as even if this was somehow on the Xbox 360,
		// it would never trip anyway because if the file existed, it would've been read earlier.
		//if ( developer.GetInt() && ( IsX360() && ( g_pFullFileSystem->GetDVDMode() != DVDMODE_STRICT ) && g_pFullFileSystem->FileExists( pszScene, "GAME" ) ) )
		//{
		//	Warning( "PrecacheInstancedScene: Missing scene '%s' from scene image cache.\nRebuild scene image cache!\n", pszScene );
		//}
	}
	else
	{
		for ( int i = 0; i < sceneData.numSounds; ++i )
		{
			short stringId = scenefilecache->GetSceneCachedSound( sceneData.sceneId, i );
			CBaseEntity::PrecacheScriptSound( scenefilecache->GetSceneString( stringId ) );
		}
	}

	g_pStringTableClientSideChoreoScenes->AddString( CBaseEntity::IsServer(), pszScene );
}

This code also fixes a bug where the maximum path length for a scene was longer on the server than on the client.

Technical information

Scenes.image files are little-endian on all platforms, with all offsets being absolute to the beginning.

Header

The scenes.image file has the following header:

Type Meaning Description
UInt32 ID 4-byte identified used for file identification. Always big-endian "VSIF".
UInt32 Version scenes.image file version. 2 is used.
UInt32 Scenes number Total number of scene files.
UInt32 Strings number Number of strings in the string pool (explained later).
UInt32 Scene entry offset Offset to the scene entries (explained later).

String pool

After the header, there are offsets to strings in the string pool.

The string pool stores all strings used in binary VCD (BVCD) files. BVCDs use signed 16-bit integers for string indices, so there cannot be more than 32767 strings in an image.

All strings in the pool are null-terminated.

Scene entries

The scene entries are placed at "Scene entry offset", and have the following structure:

Type Meaning Description
UInt32 CRC file name CRC32 hash of scene file name, in scenes\???.vcd format. All file names are lower-case and have \ as path separator. Valve CRC code should be used for best compatibility.
UInt32 Data offset Offset to the scene file.
UInt32 Data length Size of the scene file as stored in scenes.image (not real size of BVCD and/or LZMA compressed/uncompressed data).
UInt32 Scene summary offset Offset to the scene summary (explained later).

Scene data

Scenes are stored in either uncompressed or compressed format at the offsets specified in the scene entries.

Uncompressed scenes start with "bvcd" 4-byte ID. Their size is specified in the scene entries.

Compressed scenes have the following header:

Type Meaning Description
UInt32 ID Always "LZMA".
UInt32 Actual size Size of uncompressed data.
UInt32 LZMA size Size of compressed data.
Byte[5] Properties Metadata used internally by LZMA module for uncompressing.

After the header, there is compressed data, with the size being specified in the "LZMA size" field of the header.

Scenes are stored in Binary VCD format, and must be decompiled (either using CChoreoScene::RestoreFromBinaryBuffer and CChoreoScene::SaveToFile functions or a tool such as VSIF2VCD) in order to be viewable in Faceposer.

Scene summary

Scene summary stores scene information used to optimize scene playback.

Scene summaries have the following header:

Type Meaning Description
UInt32 Milliseconds Scene playback length, in milliseconds.
UInt32 Sounds number Number of sounds used in the scene.

After the header, there are 32-bit indices of strings in the string pool. The strings are sound names. They are used for precaching the scene.