View source for VPK (file format)

The [[VPK]] file format is a new package format used by [[Left 4 Dead]] to store game related content.

== Conception ==

Prior to [[Left 4 Dead]], typical Source engine games stored their content in [[GCF]] files.  Executable files, modifiable files (e.g. configuration files) and custom content were copied and stored locally on the user's hard drive.  Possibly brought on by [http://nemesis.thewavelength.net/index.php?c=216 poor performance], the [[NCF]] file format was introduced and all game content was copied entirely to the hard drive.  This, however, introduced a new problem.  Source engine materials and models are stored in thousands of small files and it would be expensive to continuously open and close these files.  The solution was the conception of the VPK file format which is used to store Left 4 Dead materials, models and particles in a handful of files which can be quickly accessed.

== Design Decisions ==

=== Preload Data ===

In order to efficiently access small or critical files, the beginning of each file can optionally be stored in the VPK directory.  In practice, this seems to be limited to the first 1000 bytes of Source engine materials ([[VMT]] files) which are typically only a few hundred bytes in size.

=== Multiple Archives ===

Previous Source engine games that had been distributed by the more advanced GCF file format had the luxury of internally fragmenting new and updated files.  This meant that new and updated files could be efficiently downloaded and saved with minimal bandwidth and disk IO.  Because the new VPK files are independent of distribution (Left 4 Dead is distributed by NCF files and Steam knows nothing of the VPK file format), their content is split up over multiple archives that seem to be limited to about 32 MB in size.  Because of this, when a file in a specific file is updated, only the VPK archive that contains the file needs to be updated.  Additionally, new files can be downloaded to their own individual archives.  This is why most of the newer archives are small in size; their contents are limited the the files added in a single update.

== File Format ==

A VPK package is actually spread out over multiple files sharing the same extension.  The directory is stored in a specific file called <name>_dir.vpk and the content is spread over several additional archive files called <name>_*.vpk (where * is the zero based archive index).  Consequentially, there are two file formats:

=== Directory File Format ===

==== Header ====

Originally, the VPK file had no header or identifier.  This changed when the [http://store.steampowered.com/news/2620/ June 25, 2009 Left 4 Dead update] was released adding support for third party campaigns.  VPK directory files created after this date have the following header:

 struct VPKHeader
 {
 	unsigned int Signature;
 	unsinged int Version;
 	unsigned int DirectoryLength;
 };

{| class=standard-table
| '''Variable''' || '''Description'''
|-
| Signature || Identifying signature.  Always 0x55aa1234.
|-
| Version || Version number.  Always 1.
|-
| DirectoryLength || The length of the directory.  If the file data is not stored in archives, its offset is the offset of the start of the directory plus this number.
|}

One can check if a VPK file was created before the update by checking if the first four bytes do not match the above signature.

==== Directory ====

Todo.

=== Archive File Format ===

VPK Archives store raw file data.  They have no identifying header and know nothing of their contents.  Though not necessary, the raw file data is typically tightly packed.