View source for DEM (file format)

{{LanguageBar}}
{{src topicon}}
{{update|2020 Source code leaks reveal different information than previously known. Also information here is outdated.}}
'''DEM''' (short for demo) is the Source demo file format. Used in conjunction with the Source recorder to record demos, which contain recorded events that can be edited and played back in-game. Demos can be edited to change camera angles, speed up, slow down, play music, and other functions.

{{note|There are currently no tools available for editing demos after they are recorded.}}

For the purpose of this we will be referring to each collection of events as a frame. This may or may not add up to the number of frames given by the header.

== Limitations/Bugs ==
{{important|Newer {{l4d2|4}} versions can't play 3-year old [[.dem]] files... Demo network protocol 2042 outdated, engine version is 2100. ''Failed to read demo header.'' }}
{{workaround|Users can lower NetworkVersion&#61;2.0.4.2 in the {{path|Left 4 Dead 2\left4dead2\[[steam.inf]]}}. Users may see ''userid "83" not found'' in the console... Older versions had a different name and file...<sup>[1]</sup>|offset=1}}
{{confirm|What about accuracy?|offset=2}}
{{confirm|What about performance?|offset=2}}
{{important|It's almost impossible to get older versions of the games with {{steamicon|4}}. This makes preservation of [[.dem]] files and Workshop Items which require specific versions impossible just with {{steamicon|4}}.}}

== File format ==

===Demo Header===
{| class=standard-table
! Type || Field || Value
|-
| [[String]] || Header || 8 characters, should be "HL2DEMO"+NULL
|-
| [[Int]] || Demo Protocol || Demo protocol version (stored in little endian)
|-
| [[Int]] || Network Protocol || Network protocol version number (stored in little endian)
|-
| [[String]] || Server name || 260 characters long
|- 
| [[String]] || Client name || 260 characters long
|-
| [[String]] || Map name || 260 characters long
|-
| [[String]] || Game directory || 260 characters long
|-
| [[Float]] || Playback time || The length of the demo, in seconds
|-
| [[Int]] || Ticks || The number of ticks in the demo
|- 
| [[Int]] || Frames || The number of frames in the demo
|-
| [[Int]] || Sign on length || Length of the signon data (Init for first frame)
|}

===Frame===
Each frame begins with 0 or more of these commands:
These are described in hl2sdk\utils\demofile\demoformat.h / hl2sdk-ob\public\demofile\demoformat.h / hl2sdk-l4d\public\demofile\demoformat.h

====Network Protocols 7 and 8====
{| class=standard-table
! Type || Value
|-
| dem_signon || 1
|-
| dem_packet || 2
|-
| dem_synctick || 3
|-
| dem_consolecmd || 4
|-
| dem_usercmd || 5
|-
| dem_datatables || 6
|-
| dem_stop || 7
|-
| dem_lastcommand || dem_stop
|}

====Network Protocols 14 and 15====
{| class=standard-table
! Type || Value
|-
| dem_stringtables || 8
|-
| dem_lastcommand || dem_stringtables
|}

====Network Protocols 36 and Higher====
{| class=standard-table
! Type || Value
|-
| dem_customdata || 8
|-
| dem_stringtables || 9
|-
| dem_lastcommand || dem_stringtables
|}

Depending on the command that was received, a different action needs to be taken.

====dem_stop====
This is a signal that the demo is over, and there is no more data to be parsed.

====dem_consolecmd, dem_datatables, dem_usercmd, dem_stringtables====
Read a standard data packet, deal with it as necessary.

====dem_synctick, dem_signon, dem_packet====
Ignore.

===Standard Data "Packet"===
Any time there is more than one byte to be read in, a standard format is used.

This begins with an integer that has the number of bytes in this packet.

Example code to handle this:

<source lang="cpp">
int ReadData(char **buffer)
{
	int len;
	fread (&len,sizeof(int),1,fDemo);
	if (len > 0)
	{
		*buffer = new char[len];
		fread(*buffer,len,1,fDemo);
	}
	return len;
}
</source>

=== Frame Format ===
<source lang="cpp">
Frame {
       int ServerFrame;
       int ClientFrame; //ServerFrame and ClientFrame delta probably correspond to client ping.
       int SubPacketSize;
       *buffer = new char[SubPacketSize]; // State update message?
       Packet pkt = (rest of frame as data exists) // All demo commands are strung together in this area, structure below
       JunkData data = (unknown) // ex: 0x8f 5a b5 04 94 e6 7c 24 00 00 00 00 00 ... (40 bytes of 0x00 after the 0x24)
                                 // This could either be the end of the frame or the start of the next frame.
}
</source>

<source lang="cpp">
Packet {
       char CmdType;
       int Unknown;
       int TickCount; //This only sporadically appears.
       int SizeOfPacket;
       *buffer = new char[SizeOfPacket];
}
</source>

=== Example of use ===

It can be interesting to grab the header of the demo files in case you run a webserver which is hosting your demos / TVs.
Here is a PHP script that does the job for you, and has hability to tell you if the demo-file is a Record in Eyes or a TV.

You can easily convert it in a C++ code if you plan to use it for another need.

<source lang="php">
<?
/* Source & OrangeBox demos header reader.
By PoLaRiTy (nocheatz.com)

Help from   : https://developer.valvesoftware.com/wiki/DEM_Format#Demo_Header
              http://hg.alliedmods.net/hl2sdks/hl2sdk-css/file/1901d5b74430/public/demofile/demoformat.h

Types sizes :
Int : 4 bytes
Float : 4 bytes
String : 260 bytes
*/


class DemoInfo_s
{
	var $dem_prot;       // Demo protocol version 
	var $net_prot;       // Network protocol versio
	var $host_name;      // HOSTNAME in case of TV, and IP:PORT or localhost:PORT in case of RIE (Record In eyes).
	var $client_name;    // Client name or TV name.
	var $map_name;       // Map name
	var $gamedir;        // Root game directory
	var $time;           // Playback time (s)
	var $ticks;          // Number of ticks
	var $frames;         // Number of frames
	var $tickrate;       // Tickrate
	var $type;           // TV or RIE ? (0 = RIE, 1 = TV)
	var $status_present; // true if a status command is available in the demo.
}

function ExtOfFile($pathtofile)
{
	return end(explode('.',$pathtofile));
}

function ReadString($handle, $n = 260)
{
	$buffer = "";
	for($d = 1; ((($char = fgetc($handle)) !== false) && ($d < $n)); $d++) $buffer = $buffer.$char;
	return trim($buffer);
}

function ReadInt($handle, $n = 4)
{
	$buf = fread($handle, $n);
	$number = unpack("i", $buf);     
	return $number[1];
}

function ReadFloat($handle)
{
	$buf = fread($handle, 4);
	$number = unpack("f", $buf);     
	return $number[1];
}

function IsGoodIPPORTFormat($string)
{
	if(preg_match('/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\:[0-9]{1,5}/', $string))	return true;
	else return false;
}

function GetDemoInfo($pathtofile, $fast = false) // BOOL fast : if true, doesn't check the presence of the status 
{
	$infos = NULL;
	if(ExtOfFile($pathtofile) === "dem")
	{
		$handle = fopen($pathtofile, "r");
		if($handle)
		{
			if(ReadString($handle, 8) === "HL2DEMO")
			{
				$infos = new DemoInfo_s;
				$infos->dem_prot = ReadInt($handle);
				$infos->net_prot = ReadInt($handle);
				$infos->host_name = ReadString($handle);
				$infos->client_name = ReadString($handle);
				$infos->map_name = ReadString($handle);
				$infos->gamedir = ReadString($handle);
				$infos->time = ReadFloat($handle);
				$infos->ticks = ReadInt($handle);
				$infos->frames = ReadInt($handle);
				$infos->tickrate = intval($infos->ticks / $infos->time);
				if(IsGoodIPPORTFormat($infos->host_name)) $infos->type = 0; // RIE   TODO : Add localhost:PORT check.
				else $infos->type = 1; // TV
				$infos->status_present = false;
				if(!$fast && !($infos->type == 1)) // No status in TV records.
				{
					while(!(($l = fgets($handle)) === false))
					{
						if(stripos($l, "\x00status\x00") !== false)
						{
							$infos->status_present = true;
							break;
						}
					}
				}
			}
			else
			{
				echo "Bad file format.";
			}
			fclose($handle);
		}
		else
		{
			echo "File not found or unable to read.";
		}
	}
	else
	{
		echo "Bad file extension.";
	}
	return $infos;
}
?>
</source>

== References ==
# "Watch Old L4D2 demos" by Dr. Whoop DSc https://gamebanana.com/tuts/9534

==See also==
*[[Demo Recording Tools]]
*[[Demo Video Creation]]
*[[Demoinfo]]

{{Split|reason=split by forward compatibility}}

[[Category:File formats]]