Compression (Source 1)/en

From Valve Developer Community
Jump to: navigation, search
Abstract Mapping series Discuss your thoughts - Help us develop the articles or ideas you want

Ammunition | Animals & Creatures | Antlions | Beams & Lasers | Cables & Ropes | Clouds & Sky | Color Theory | Combat | Combine | Compression | Doors | Dust, Fog & Smoke | Elevators | Level Transitions | Environmental Lighting & Weather | Explosions | Fire | Foliage | Glass & Windows | Headcrab | Health | Ladders | Lighting | Optimization (Level Design) | Physics | Retinal scanners | Sound & Music | Special effects | Terrain | Trains | Turrets | Water | Weapons | Zombie

Unlike engines such as id Tech 3 id Tech 3, game archives in Source Source are uncompressed. While this reduced potential for fragmentation on hard drives back in 2004, modern storage drives and CPUs are able to load the data much faster, while ever-increasing game sizes means storage space is still a luxury. Additionally, multiplayer maps need to be kept under certain sizes for a server to automatically send it to clients. These factors mean that it is wise to seek out methods to reduce avoidable file size bloat, through the use of various compression techniques, both lossless and lossy. This page lists some ways that file size can be optimized in a Source Engine game or mod.

Tip.pngTip:Max file download sizes for multiplayer servers (multiplayer map files should be smaller than this):
  • 64 MB for servers without FastDL or Workshop downloads (requires editing net_maxfilesize)
  • 150 MB for servers using FastDL (256 MB in Jabroni Brawl: Episode 3)
  • Game-dependent for games using Workshop downloads (525 MB for Counter-Strike: Global Offensive)
Confirm.pngConfirm: Might actually be up to 2 GB for FastDL outside of CS:GO engine branch CS:GO engine branch; it's not a great idea to reach anywhere near that, though.
Tip.pngTip:Smart usage of compression can actually reduce load times, as less data needs to be read from the storage drive!

Texture Compression

Textures are one of the biggest potential sources of file size bloat. Additionally, higher quality textures can result in increased VRAM usage, which will increase a game or mod's minimum GPU requirements.

  • Most textures should use DXT1 or DXT5 (depending on if they have an alpha channel). While older DXTn compressors introduced a green tint to textures, the modern compressors used in VTFEdit Reloaded and Vtex can reliably compress to these formats without noticeable color shift.
  • If an opaque texture is low-res enough that DXT1 compression artefacts are problematic, consider using 16-bit BGR565 instead of 24-bit BGR888. The quality difference can be often negligible for brush and model diffuse textures, while still being much higher quality than DXT1.
Note.pngNote:BGR565, BGR888, and RGB888 are all converted to BGRA8888 before being uploaded to the GPU; they will save space on the storage drive, but will take up the same amount of video memory.
  • Formats with more than 8 bits per channel should be reserved for HDR skyboxes and cubemaps only; nobody is going to be able to tell a 48-bit normal map apart from a 24-bit one!
  • HDR skybox textures can use a Compressed HDR format that stores a 48-bit RGB image in a 32-bit BGRA8888 VTF file. Vtex uses this format when converting HDR textures to VTF, unless nocompress is specified.
Note.pngNote:Since uncompressed HDR textures are required to have a (usually unused) alpha channel, which adds 16 extra bits, the effective compression ratio of compressed HDR is actually 2:1 instead of 1.5:1. Of course, compressed HDR can't have translucency, since the alpha channel is already used for RGB data, but the sky shader doesn't support translucency anyway.

Strata Source Strata Source, notably used by Portal 2: Community Edition and Momentum Mod, supports some additional texture formats and features:

  • BC7 is much higher quality than DXT1 or DXT5, while maintaining the same GPU footprint as DXT5. This means it can be used where other engine branches would require a BGR565 or BGRA8888 texture. Additionally, it can be used instead of DXT5 at negligible cost for textures where the color data is more important than the alpha channel data.
  • ATI1N is a compressed greyscale format which is half the size of I8. While this is the same size as DXT1, it only has a single greyscale channel, making it superior for greyscale textures like decals using the DecalModulate shader, or specular masks for materials without a normal map.
    Confirm:Make sure the DecalModulate shader actually supports greyscale textures.
  • VTF 7.6 supports deflate compression. While this does not reduce GPU footprint, it can greatly reduce filesize on the user's storage device.

Audio Compression

Audio is another major source of file size bloat.

  • The most obvious way to reduce audio file size is to use lossy audio compression, with MP3 being the most common. This works well for non-looping music or incidental sounds in a soundscape. This introduces latency, however, and it can't be used for certain Source Engine features that are built with WAV files in mind, such as looping sounds and lip synching.
Tip.pngTip:When making a standalone game or single-player mod, it is possible to add support for other similar lossy formats, such as Ogg Vorbis or Opus, and bypass some of the limitations of the default MP3 implementation.
  • WAV files don't need to be uncompressed. Source Engine also supports 4-bit Microsoft ADPCM, which can be used for almost anything that uncompressed 16-bit PCM-encoded WAVs can be used for. They can be looped in their entirety, and can be used for lip-synched dialogue. They are natively supported by sound hardware, making them less CPU-demanding than MP3 files.
    They have some disadvantages, though, mostly related to looping. The format requires fixed sample counts, meaning that shorter looped sound effects will have a noticeable "pop" when looping (this isn't usually noticeable for longer music tracks). The starting loop cue will also sometimes be ignored, making the format unsuitable for music that doesn't loop the entire file. Therefore, ADPCM is best suited for dialogue, non-looping sound effects, and some looping music.
    • If ADPCM and MP3 are unsuitable for a given sound, but it absolutely necessary to reduce the sound's file size, it is also possible to use an uncompressed 8-bit WAV instead of 16-bit. This will cut file size in half, but will also raise the noise floor to a noticeable level, resulting in an effect similar to tape hiss, which may be undesirable and obnoxious.
      Warning.pngWarning:Many audio editors, such as Audacity, use dithering by default when converting between bit depths. While a proper shaped dither is imperceivable at 16-bit or higher, it makes the quantization noise worse at 8-bit, and should be disabled.
      Tip.pngTip:Disabling dithering at 16-bit can also improve the compression of a zipped WAV file.
  • Regardless of the chosen format, reducing the sample rate of a sound effect will result in lower potential file sizes (see the MP3 and WAV pages for examples). This will result in a reduction in quality, but for certain applications such as music on a multiplayer map and explosions, the drop in sample rate from 44100 Hz to 22050 Hz may be an acceptable compromise in exchange for a lower file size.
Format Recommended usages
16-bit WAV
  • Looping sound effects
  • Looping music that only loops part of the track
  • Looping soundscape ambience with an intro
8-bit WAV
  • Sounds that are already 8-bit (ex: Half-Life ports)
  • Last resort for saving space compared to 16-bit
  • Lip-synced dialog
  • Non-looping sound effects
  • Looping music that loops the entire track
  • Looping soundscape ambience without an intro
MP3, Ogg, etc
  • Non-looping music
  • Non-looping soundscape ambience
  • Off-screen dialog

Map Compression

In some engine branches, such as Source 2013 Multiplayer Source 2013 Multiplayer, BSP files can be losslessly compressed with LZMA compression via a BSPZIP command. This can greatly reduce the file size of a map with embedded content and/or high resolution lightmaps (especially prop lightmaps) and cubemaps.

There is usually no disadvantage to doing this, although some caveats are listed on the BSPZIP page.

Dedicated Server FastDL Compression

Source supports FastDL downloads to be compressed using Wikipedia icon BZip2 (BZ2). The files will be decompressed by the user client after downloading. It is recommended to compress most loose files that are larger than a few kilobytes (VTF, WAV, MDL, etc). BSP files should be compressed with BZ2 if not already compressed with BSPZIP. MP3 files are already heavily compressed and do not substantially reduce in size from BZ2 compression; BZ2 compression of MP3s or BSPZIP-compressed BSPs may unnecessarily increase user client load times with negligible bandwidth savings.

File System Compression

An end user can also choose to losslessly and transparently compress a game's files via file system compression options in the user's operating system, such as Wikipedia icon LZX compression on Windows Windows and Wikipedia icon ZSTD compression on Linux Linux. This can reliably cut the size of many Source Engine games in half or smaller, but this is a broad topic that expands far beyond the scope of this wiki.