DEM (file format)

This article or section needs to be updated to include current information regarding the subject because:

2020 Source code leaks reveal different information than previously known. Also information here is outdated.

Remember to check for any notes left by the tagger at this article's talk page.

DEM (short for demo) is the Source demo file format. Used in conjunction with the Source recorder to record demos, which contain recorded events that can be edited and played back in-game. Demos can be edited to change camera angles, speed up, slow down, play music, and other functions.

Note:There are currently no tools available for editing demos after they are recorded.

For the purpose of this we will be referring to each collection of events as a frame. This may or may not add up to the number of frames given by the header.

Limitations/Bugs

Important:Newer Left 4 Dead 2 versions can't play 3-year old .dem files... Demo network protocol 2042 outdated, engine version is 2100. Failed to read demo header.

Workaround:Users can lower NetworkVersion=2.0.4.2 in the Left 4 Dead 2\left4dead2\steam.inf. Users may see userid "83" not found in the console... Older versions had a different name and file...^[1]

Confirm:What about accuracy?

Confirm:What about performance?

Important:It's almost impossible to get older versions of the games with Steam. This makes preservation of .dem files and Workshop Items which require specific versions impossible just with Steam.

File format

Demo Header

Type	Field	Value
String	Header	8 characters, should be "HL2DEMO"+NULL
Int	Demo Protocol	Demo protocol version (stored in little endian)
Int	Network Protocol	Network protocol version number (stored in little endian)
String	Server name	260 characters long
String	Client name	260 characters long
String	Map name	260 characters long
String	Game directory	260 characters long
Float	Playback time	The length of the demo, in seconds
Int	Ticks	The number of ticks in the demo
Int	Frames	The number of frames in the demo
Int	Sign on length	Length of the signon data (Init for first frame)

Frame

Each frame begins with 0 or more of these commands: These are described in hl2sdk\utils\demofile\demoformat.h / hl2sdk-ob\public\demofile\demoformat.h / hl2sdk-l4d\public\demofile\demoformat.h

Network Protocols 7 and 8

Type	Value
dem_signon	1
dem_packet	2
dem_synctick	3
dem_consolecmd	4
dem_usercmd	5
dem_datatables	6
dem_stop	7
dem_lastcommand	dem_stop

Network Protocols 14 and 15

Type	Value
dem_stringtables	8
dem_lastcommand	dem_stringtables

Network Protocols 36 and Higher

Type	Value
dem_customdata	8
dem_stringtables	9
dem_lastcommand	dem_stringtables

Depending on the command that was received, a different action needs to be taken.

dem_stop

This is a signal that the demo is over, and there is no more data to be parsed.

dem_consolecmd, dem_datatables, dem_usercmd, dem_stringtables

Read a standard data packet, deal with it as necessary.

dem_synctick, dem_signon, dem_packet

Ignore.

Standard Data "Packet"

Any time there is more than one byte to be read in, a standard format is used.

This begins with an integer that has the number of bytes in this packet.

Example code to handle this:

int ReadData(char **buffer)
{
	int len;
	fread (&len,sizeof(int),1,fDemo);
	if (len > 0)
	{
		*buffer = new char[len];
		fread(*buffer,len,1,fDemo);
	}
	return len;
}

Frame Format

Frame {
       int ServerFrame;
       int ClientFrame; //ServerFrame and ClientFrame delta probably correspond to client ping.
       int SubPacketSize;
       *buffer = new char[SubPacketSize]; // State update message?
       Packet pkt = (rest of frame as data exists) // All demo commands are strung together in this area, structure below
       JunkData data = (unknown) // ex: 0x8f 5a b5 04 94 e6 7c 24 00 00 00 00 00 ... (40 bytes of 0x00 after the 0x24)
                                 // This could either be the end of the frame or the start of the next frame.
}

Packet {
       char CmdType;
       int Unknown;
       int TickCount; //This only sporadically appears.
       int SizeOfPacket;
       *buffer = new char[SizeOfPacket];
}

Example of use

It can be interesting to grab the header of the demo files in case you run a webserver which is hosting your demos / TVs. Here is a PHP script that does the job for you, and has hability to tell you if the demo-file is a Record in Eyes or a TV.

You can easily convert it in a C++ code if you plan to use it for another need.

<?
/* Source & OrangeBox demos header reader.
By PoLaRiTy (nocheatz.com)

Help from   : https://developer.valvesoftware.com/wiki/DEM_Format#Demo_Header
              http://hg.alliedmods.net/hl2sdks/hl2sdk-css/file/1901d5b74430/public/demofile/demoformat.h

Types sizes :
Int : 4 bytes
Float : 4 bytes
String : 260 bytes
*/


class DemoInfo_s
{
	var $dem_prot;       // Demo protocol version 
	var $net_prot;       // Network protocol versio
	var $host_name;      // HOSTNAME in case of TV, and IP:PORT or localhost:PORT in case of RIE (Record In eyes).
	var $client_name;    // Client name or TV name.
	var $map_name;       // Map name
	var $gamedir;        // Root game directory
	var $time;           // Playback time (s)
	var $ticks;          // Number of ticks
	var $frames;         // Number of frames
	var $tickrate;       // Tickrate
	var $type;           // TV or RIE ? (0 = RIE, 1 = TV)
	var $status_present; // true if a status command is available in the demo.
}

function ExtOfFile($pathtofile)
{
	return end(explode('.',$pathtofile));
}

function ReadString($handle, $n = 260)
{
	$buffer = "";
	for($d = 1; ((($char = fgetc($handle)) !== false) && ($d < $n)); $d++) $buffer = $buffer.$char;
	return trim($buffer);
}

function ReadInt($handle, $n = 4)
{
	$buf = fread($handle, $n);
	$number = unpack("i", $buf);     
	return $number[1];
}

function ReadFloat($handle)
{
	$buf = fread($handle, 4);
	$number = unpack("f", $buf);     
	return $number[1];
}

function IsGoodIPPORTFormat($string)
{
	if(preg_match('/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\:[0-9]{1,5}/', $string))	return true;
	else return false;
}

function GetDemoInfo($pathtofile, $fast = false) // BOOL fast : if true, doesn't check the presence of the status 
{
	$infos = NULL;
	if(ExtOfFile($pathtofile) === "dem")
	{
		$handle = fopen($pathtofile, "r");
		if($handle)
		{
			if(ReadString($handle, 8) === "HL2DEMO")
			{
				$infos = new DemoInfo_s;
				$infos->dem_prot = ReadInt($handle);
				$infos->net_prot = ReadInt($handle);
				$infos->host_name = ReadString($handle);
				$infos->client_name = ReadString($handle);
				$infos->map_name = ReadString($handle);
				$infos->gamedir = ReadString($handle);
				$infos->time = ReadFloat($handle);
				$infos->ticks = ReadInt($handle);
				$infos->frames = ReadInt($handle);
				$infos->tickrate = intval($infos->ticks / $infos->time);
				if(IsGoodIPPORTFormat($infos->host_name)) $infos->type = 0; // RIE   TODO : Add localhost:PORT check.
				else $infos->type = 1; // TV
				$infos->status_present = false;
				if(!$fast && !($infos->type == 1)) // No status in TV records.
				{
					while(!(($l = fgets($handle)) === false))
					{
						if(stripos($l, "\x00status\x00") !== false)
						{
							$infos->status_present = true;
							break;
						}
					}
				}
			}
			else
			{
				echo "Bad file format.";
			}
			fclose($handle);
		}
		else
		{
			echo "File not found or unable to read.";
		}
	}
	else
	{
		echo "Bad file extension.";
	}
	return $infos;
}
?>

References

1. "Watch Old L4D2 demos" by Dr. Whoop DSc https://gamebanana.com/tuts/9534

See also

• Demo Recording Tools
• Demo Video Creation
• Demoinfo

Split

It has been suggested that this article or section be split into multiple articles: split by forward compatibility. (Discuss)