Difference between revisions of "Soundcache"

From Valve Developer Community
Jump to: navigation, search
m (mentioned that csgo no longer uses soundcaches)
 
(48 intermediate revisions by 13 users not shown)
Line 1: Line 1:
 +
{{otherlang2
 +
| ru = Soundcache:ru}}
 +
 +
After an update in 2018, {{csgo}} '''[[Counter-Strike: Global Offensive]]''' no longer uses soundcaches, and loads all the audio files in real-time instead. This also means that any console commands that used to be associated with it (such as '''snd_rebuildaudiocache''') have been removed.
 
==Outline==
 
==Outline==
A '''Soundcache''' is a file with contains the first .125 seconds (125 milliseconds) of each sound required in a map. At level load, the Source engine loads certain sound .cache files instead of the full .wavs themselves in order to save memory and load time.  During gameplay when a sound is requested, the sound begins to play immediately (consuming the 125 milliseconds of preload data in the .cache files) while the full sound is asynchronously loaded from the hard drive. In most cases, there is enough data loaded by the time the sound preload data is exhausted such that the async i/o never needs to "block" to finish loading.  If the async layer, however, does need to block, then a noticeable framerate hitch can occur.  These hitches can be displayed to the developer console by enabling the cvar <code>snd_async_spew_blocking 1</code>.
+
A '''soundcache''' is a file that contains the first .125 seconds (125 milliseconds) of each sound required in a map. At level load, Source loads the relevant cache files instead of the sound data itself in order to save memory and decrease waiting times.  During gameplay, when a sound is requested, it begins to play immediately while the full sound is asynchronously loaded from the hard drive. In most cases, there is enough data loaded by the time the 125 milliseconds of cache data is exhausted such that the async I/O never needs to "block" gameplay to finish loading.  If the async layer, however, does need to block, then a noticeable framerate hitch can occur.  These hitches can be displayed to the developer console with the cvar <code>snd_async_spew_blocking 1</code>.
  
 
==Re-building the .manifest Files==
 
==Re-building the .manifest Files==
  
These instructions apply to MODs trying to generate the best possible .cache files:
+
{{note|(From Valve employee [[User:Yahnbernier|Yahn Bernier]]) I haven't actually tried to run this from the Steam cmd line, so if anyone runs into issues with the command line instructions below, please email me at [mailto:[email protected] [email protected]] and I'll get it sorted out}}
 +
 
 +
{{note|If you are short on patience, you can run through the following procedure using only one map in the maplist and still receive a proficient soundcache.}}
 +
 
 +
{{note|Starting with the [[Left 4 Dead]] series add-on custom content, a sound cache is built for every '''sound''' directory. Please see [[L4D2 Custom Sound and Music Tutorial]] for an example of what may apply to future Source engine-based games.}}
  
1. Create a text file called maplist.txt and place it into your game directory. For example, in the case of Counter-Strike, it would live in the \cstrike directory.
+
These instructions apply to mods trying to generate the best possible .cache files:
2. Add in all of the map names to maplist.txt. Just add the map name without .bsp on the end of them (e.g. d1_trainstation_01, not d1_trainstation_01.bsp). Seperate each map name with a carriage return.
 
3. Create a new .bat file and copy in this line:
 
Steam.exe -applaunch nnn -w 640 -sw -console -condebug -nocrashdialog -makereslists -usereslistfile maplist.txt +mat_picmip 2 +r_lod 3 -autoconfig %1 %2 %3 %4 %5
 
where nnn is the appropriate steam App Id for the game which you are modding.
 
4. Save and run the batch file.  Running this process can take a very long time and if the engine crashes at any point then the data will not be valid.  The engine must make it through the entire maplist.txt file cleanly to generate valid .cache files.
 
  
This will run the engine and start each map, wait a few seconds for things to settle, and write out a reslist to the \<gamedir>\reslists folder.  These reslists are then used to build the actual sound .cache files.  At exit, after running with the above command line, the engine should rebuild the meta-cache files and all map specific cache files based on the maplist.txt file.
+
#Create a text file called <code>maplist.txt</code> and place it into the game directory.
 +
#Add in all of the map names to <code>maplist.txt</code>. Just add the map names, e.g. d1_trainstation_01, not d1_trainstation_01.bsp. Separate each map name with a line break.
 +
#In Steam "My Games" Page, right click your mod then select '''Properties'''. Open '''Set Launch options...''' and enter following line:<br /><code>-sw -console -condebug -nocrashdialog -makereslists -usereslistfile maplist.txt +mat_picmip 2 +r_lod 3 -autoconfig</code>{{note|If you are using the Source SDK Base, enter the line into the properties of the SDK Base, not your mod.}}
 +
#Click OK and run the game.  Running this process can take a very long time, and if the engine crashes at any point, the data will not be valid.  The engine must make it through the entire <code>maplist.txt</code> file cleanly to generate valid .cache files. Don't forget to revert your command line option once it finished.
 +
#Once complete, reset mat_picmip to 0, or its previous value
 +
The engine will run and start each map, wait a few seconds for things to settle, and write out a reslist to the <code>reslists</code> folder.  These reslists are then used to build the actual sound .cache files.  At exit, after running with the above command line, the engine should rebuild the meta-cache files and all map specific cache files based on the <code>maplist.txt</code> file.
  
Each map's <code>mapname.cache</code> is built using the accompanying .manifest, which contains a list of sound assets Precached by that map.
+
Each map's <code>mapname.cache</code> is built using the accompanying .manifest files, which contains a list of sounds Precached by that map, as well as the reslist files mentioned above which contain the names of all files accessed during level load, not just those mentioned as Precached.
  
 
==Extraction==
 
==Extraction==
 
*.cache files can be viewed with Game Extractor [http://www.watto.org/extract/]
 
*.cache files can be viewed with Game Extractor [http://www.watto.org/extract/]
*.manifest files can be viewed with any text editor
+
*.manifest files can be viewed with any [[text editor]].
  
 
==Caches==
 
==Caches==
*<code>_sharedprecache.cache</code>: contains data for precached sounds (see the Precaching assets section for info about precaching) common to over 50% of maps (e.g., sounds referenced by the response system which can occor in any map are always precached).
+
*<code>_sharedprecache.cache</code>: contains data for precached sounds (see the [[Precaching Assets]] section for info about precaching) common to over 50% of maps (e.g., sounds referenced by the response system which can occur in any map are always precached).
 
*<code>_other.cache</code>: contains the file header, but no preload data, for any sound files that were not encountered in any other precache lists. Sounds in this cache will never cause I/O-related blocking as they play back fully asynchronously, that is, they only begin to mix data to the sound hardware after the sound data has completed asynchronous loading.
 
*<code>_other.cache</code>: contains the file header, but no preload data, for any sound files that were not encountered in any other precache lists. Sounds in this cache will never cause I/O-related blocking as they play back fully asynchronously, that is, they only begin to mix data to the sound hardware after the sound data has completed asynchronous loading.
 
*<code><mapname>.cache</code>: Contains the remaining files that were Precached by the engine for the specified .bsp, but are not already a part of <code>_sharedprecache.cache</code>.
 
*<code><mapname>.cache</code>: Contains the remaining files that were Precached by the engine for the specified .bsp, but are not already a part of <code>_sharedprecache.cache</code>.
 +
*<code>_other_rebuild.cache</code>: This is a temporary file used as a way to quickly rebuild the runtime .cache files above.
 +
 +
==Hand-editing==
 +
Although the engine-generated <code>.manifest</code> files help somewhat, editing the files by hand after their generation can increase performance by some 400%. Look out for '\VO\' files in particular: in mods based on Half-Life 2, there will be three VO entries followed by three Combine soldier death sounds that you will want to keep, followed by hundreds of further, probably unneccesary, VOs that can all be deleted. After deleting all VO entries your manifests (and thus caches) should be under '''half''' their original size! This effect can then cascade down and produce more optimised <code>_sharedprecache</code> files, reducing duplication (in the author's experiment, the engine-generated caches were 30MB while his own were 7MB).
 +
 +
There are further entries that can be deleted, for example weapon sounds for weapons your map does not use, but care should be taken to avoid the deletion of files that ''will'' be used, as absent cache entries will likely cause hitches.
 +
 +
==Shipping==
 +
You don't have to ship your mod with above caches. If you include reslist/.manifest files, the game will build sound caches when you first run it. However, this process can take a signifigant amount of time and users may be confused and/or get impatient. If your caches are small, you may want to consider shipping them anyway.
  
 
==Console Commands==
 
==Console Commands==
Line 30: Line 47:
 
:Writes the .manifest file. Possibly buggy and probably not necessary based on using the above-described command line to automatically rebuild the .cache files.
 
:Writes the .manifest file. Possibly buggy and probably not necessary based on using the above-described command line to automatically rebuild the .cache files.
 
*<code>snd_rebuildaudiocache</code>
 
*<code>snd_rebuildaudiocache</code>
:Writes the .cache file, containing soundfile snippets. Appears to have no effect.
+
:Rebuilds the .cache files. Only has effect if valid reslists and .manifest files exist.
 
 
 
==Limitations & Bugs==
 
==Limitations & Bugs==
The contents of the .cache files could be biased by hand-editing the .manifest files and requesting the engine to <code>snd_rebuildaudiocache</code>.  However, this is generally not advised.
+
{|style="background: black;"
 +
|style="color: yellow;background-color: black;font-size: 90px;line-height:90px;"|!
 +
|style="color: white;background-color: black;font-size: 25px;line-height:30px;"|It's important to note that a mod without a shared precache soundcache built can force the engine to create '''large''' map soundcaches; this large soundcache can usually be in the range of 20 MB for Half-Life 2 based mods.
 +
|}
  
 +
[[Category:File formats]]
 +
[[Category:Publicity & Publication]]
 
[[Category:Sound System]]
 
[[Category:Sound System]]
 
[[Category:Technical]]
 
[[Category:Technical]]

Latest revision as of 09:22, 3 April 2019

Русский

After an update in 2018, <Counter-Strike: Global Offensive> Counter-Strike: Global Offensive no longer uses soundcaches, and loads all the audio files in real-time instead. This also means that any console commands that used to be associated with it (such as snd_rebuildaudiocache) have been removed.

Outline

A soundcache is a file that contains the first .125 seconds (125 milliseconds) of each sound required in a map. At level load, Source loads the relevant cache files instead of the sound data itself in order to save memory and decrease waiting times. During gameplay, when a sound is requested, it begins to play immediately while the full sound is asynchronously loaded from the hard drive. In most cases, there is enough data loaded by the time the 125 milliseconds of cache data is exhausted such that the async I/O never needs to "block" gameplay to finish loading. If the async layer, however, does need to block, then a noticeable framerate hitch can occur. These hitches can be displayed to the developer console with the cvar snd_async_spew_blocking 1.

Re-building the .manifest Files

Note.png Note: (From Valve employee Yahn Bernier) I haven't actually tried to run this from the Steam cmd line, so if anyone runs into issues with the command line instructions below, please email me at [email protected] and I'll get it sorted out
Note.png Note: If you are short on patience, you can run through the following procedure using only one map in the maplist and still receive a proficient soundcache.
Note.png Note: Starting with the Left 4 Dead series add-on custom content, a sound cache is built for every sound directory. Please see L4D2 Custom Sound and Music Tutorial for an example of what may apply to future Source engine-based games.

These instructions apply to mods trying to generate the best possible .cache files:

  1. Create a text file called maplist.txt and place it into the game directory.
  2. Add in all of the map names to maplist.txt. Just add the map names, e.g. d1_trainstation_01, not d1_trainstation_01.bsp. Separate each map name with a line break.
  3. In Steam "My Games" Page, right click your mod then select Properties. Open Set Launch options... and enter following line:
    -sw -console -condebug -nocrashdialog -makereslists -usereslistfile maplist.txt +mat_picmip 2 +r_lod 3 -autoconfig
    Note.png Note: If you are using the Source SDK Base, enter the line into the properties of the SDK Base, not your mod.
  4. Click OK and run the game. Running this process can take a very long time, and if the engine crashes at any point, the data will not be valid. The engine must make it through the entire maplist.txt file cleanly to generate valid .cache files. Don't forget to revert your command line option once it finished.
  5. Once complete, reset mat_picmip to 0, or its previous value

The engine will run and start each map, wait a few seconds for things to settle, and write out a reslist to the reslists folder. These reslists are then used to build the actual sound .cache files. At exit, after running with the above command line, the engine should rebuild the meta-cache files and all map specific cache files based on the maplist.txt file.

Each map's mapname.cache is built using the accompanying .manifest files, which contains a list of sounds Precached by that map, as well as the reslist files mentioned above which contain the names of all files accessed during level load, not just those mentioned as Precached.

Extraction

  • .cache files can be viewed with Game Extractor [1]
  • .manifest files can be viewed with any text editor.

Caches

  • _sharedprecache.cache: contains data for precached sounds (see the Precaching Assets section for info about precaching) common to over 50% of maps (e.g., sounds referenced by the response system which can occur in any map are always precached).
  • _other.cache: contains the file header, but no preload data, for any sound files that were not encountered in any other precache lists. Sounds in this cache will never cause I/O-related blocking as they play back fully asynchronously, that is, they only begin to mix data to the sound hardware after the sound data has completed asynchronous loading.
  • <mapname>.cache: Contains the remaining files that were Precached by the engine for the specified .bsp, but are not already a part of _sharedprecache.cache.
  • _other_rebuild.cache: This is a temporary file used as a way to quickly rebuild the runtime .cache files above.

Hand-editing

Although the engine-generated .manifest files help somewhat, editing the files by hand after their generation can increase performance by some 400%. Look out for '\VO\' files in particular: in mods based on Half-Life 2, there will be three VO entries followed by three Combine soldier death sounds that you will want to keep, followed by hundreds of further, probably unneccesary, VOs that can all be deleted. After deleting all VO entries your manifests (and thus caches) should be under half their original size! This effect can then cascade down and produce more optimised _sharedprecache files, reducing duplication (in the author's experiment, the engine-generated caches were 30MB while his own were 7MB).

There are further entries that can be deleted, for example weapon sounds for weapons your map does not use, but care should be taken to avoid the deletion of files that will be used, as absent cache entries will likely cause hitches.

Shipping

You don't have to ship your mod with above caches. If you include reslist/.manifest files, the game will build sound caches when you first run it. However, this process can take a signifigant amount of time and users may be confused and/or get impatient. If your caches are small, you may want to consider shipping them anyway.

Console Commands

  • snd_writemanifest
Writes the .manifest file. Possibly buggy and probably not necessary based on using the above-described command line to automatically rebuild the .cache files.
  • snd_rebuildaudiocache
Rebuilds the .cache files. Only has effect if valid reslists and .manifest files exist.

Limitations & Bugs

! It's important to note that a mod without a shared precache soundcache built can force the engine to create large map soundcaches; this large soundcache can usually be in the range of 20 MB for Half-Life 2 based mods.