KeyValues: Difference between revisions
No edit summary |
(Add sourcepp to the C++ parser list) |
||
(12 intermediate revisions by 9 users not shown) | |||
Line 1: | Line 1: | ||
{{Distinguish|$keyvalues|desc1=[[Qc]] command|Keyvalue|desc2=Entities in Hammer}} | {{hierarchy|KeyValues}}{{Distinguish|$keyvalues|desc1=[[Qc]] command|Keyvalue|desc2=Entities in Hammer}} | ||
The KeyValues format is used in the {{src|4}} engine to store [[meta data for resources]], [[script]]s, [[material]]s, [[VGUI|VGUI element]]s, and more...{{clarify}} | The KeyValues format is used in the {{src|4}} engine to store [[meta data for resources]], [[script]]s, [[material]]s, [[VGUI|VGUI element]]s, and more...{{clarify}} | ||
== Definition == | |||
'''The KeyValues class''' handles data buffer input, file input, and file output. | |||
A KeyValue is defined recursively as a named key either with a value or children. All keys have names set to them and a search can be performed by the names. If a KeyValue is a parent key, it contains other KeyValues. | |||
=== Value Types === | |||
{{Note|The KeyValue type gets set automatically when a set value function (i.e. <code>SetFloat</code>) is called.}} | |||
* <code>TYPE_NONE</code> | |||
* <code>TYPE_STRING</code> | |||
* <code>TYPE_INT</code> | |||
* <code>TYPE_FLOAT</code> | |||
* <code>TYPE_PTR</code> | |||
* <code>TYPE_WSTRING</code> | |||
* <code>TYPE_COLOR</code> | |||
=== Example === | |||
<source lang="cpp"> | |||
"ParentKey1" | |||
{ | |||
"ValueKey1" "1" | |||
"ParentKey2" | |||
{ | |||
... | |||
} | |||
} | |||
</source> | |||
== File Format == | == File Format == | ||
The KeyValues.h header in the Source SDK code defines the file format in the following manner | The [https://github.com/ValveSoftware/source-sdk-2013/blob/master/src/public/tier1/KeyValues.h KeyValues.h] header in the Source SDK code defines the file format in the following manner. | ||
=== | === Basic Rules === | ||
* It has 3 control characters '''{''', '''}''' and '''"'''. Names and values may be quoted or not. The quote '''"''' character must not be used within name or values, only for quoting whole tokens. You may use escape sequences while parsing and add within a quoted token a '''\"''' to add quotes within your name or token. | * It has 3 control characters: '''{''', '''}''' and '''"'''. Names and values may be quoted or not. The quote '''"''' character must not be used within name or values, only for quoting whole tokens. You may use escape sequences while parsing and add within a quoted token a '''\"''' to add quotes within your name or token. | ||
* When using Escape Sequences, the parser must know that by setting '''KeyValues::UsesEscapeSequences( true )''', which is off by default. | * When using Escape Sequences, the parser must know that by setting '''KeyValues::UsesEscapeSequences( true )''', which is off by default. | ||
* Non-quoted tokens ends with a ''whitespace'', '''{''', '''}''' and '''"'''. So you may use '''{''' and '''}''' within quoted tokens, but not for non-quoted tokens. | * Non-quoted tokens ends with a ''whitespace'', '''{''', '''}''' and '''"'''. So you may use '''{''' and '''}''' within quoted tokens, but not for non-quoted tokens. | ||
Line 13: | Line 40: | ||
* Whitespaces are space, return, newline and tabulator. Allowed Escape sequences are '''\n''', '''\t''', '''\\''', and '''\"'''. | * Whitespaces are space, return, newline and tabulator. Allowed Escape sequences are '''\n''', '''\t''', '''\\''', and '''\"'''. | ||
* The number character '''#''' is used for macro purposes (eg #include), don't use it as first character in key names. | * The number character '''#''' is used for macro purposes (eg #include), don't use it as first character in key names. | ||
* Each token can be up to 1024 characters long (including null-termination, and double quotes, that will leave room for 1021 actual characters). Note that some functions in the KeyValues class will not handle strings with over 256 characters correctly. Searching for a key name over 256 characters containing a forward slash '''/''' character causes a buffer overrun on the stack in the <code>KeyValues::FindKey</code> function. | |||
Please note that the above is only an informal description of the format. It is not a definitive guide for how the format should be interpreted by parsers. It has not been tested whether the KeyValues parser in the Source SDK Code parses exactly as according to the above. The KeyValues parser might not handle very long keyvalues correctly. | Please note that the above is only an informal description of the format. It is not a definitive guide for how the format should be interpreted by parsers. It has not been tested whether the KeyValues parser in the Source SDK Code parses exactly as according to the above. The KeyValues parser might not handle very long keyvalues correctly. | ||
=== Include Statements === | |||
* You can use include statements in keyvalues scripts to include other text files. <source lang="cpp">#include "file path"</source> You can also use the base statement that | |||
'''Example:''' Say you want to include ''' | * You can use include statements in keyvalues scripts to include other text files. <source lang="cpp">#include "file path"</source> | ||
According to the [https://github.com/ValveSoftware/source-sdk-2013/blob/39f6dde8fbc238727c020d13b05ecadd31bda4c0/src/tier1/KeyValues.cpp#L2049-L2066 source code], the way <code>#include</code> statements work is by parsing the included file as usual and then appending all of its keys at the end of the currently parsed list, including duplicate keys. | |||
" | |||
{ | |||
'''Example:''' Say you want to include '''extras.vdf''' in your '''main.vdf''' file. If you write '''extras.vdf''' like this: | |||
} | <source lang="cpp"> | ||
* The file format specification allows for C++ style comments. However, the implementation of this in Valve's Key-Value code is poor, and assumes anything following a single forward slash is a comment. EDIT : This does not appear to be the case (Refer [https://github.com/ValveSoftware/source-sdk-2013/blob/ | "Key1" "Extra1" | ||
<source lang="cpp">// This is a comment until the line ends | "Key2" "Extra2" | ||
"List" | |||
{ | |||
"InnerKey1" "InnerExtra1" | |||
"InnerKey2" "InnerExtra2" | |||
} | |||
</source> | |||
And '''main.vdf''' like this: | |||
<source lang="cpp"> | |||
#include "extras.vdf" | |||
"Key1" "Value1" | |||
"List" | |||
{ | |||
} | |||
</source> | |||
Then the '''main.vdf''' file will end up being loaded with the following contents: | |||
<source lang="cpp"> | |||
"Key1" "Value1" | |||
"List" | |||
{ | |||
} | |||
"Key1" "Extra1" | |||
"Key2" "Extra2" | |||
"List" | |||
{ | |||
"InnerKey1" "InnerExtra1" | |||
"InnerKey2" "InnerExtra2" | |||
} | |||
</source> | |||
Note how the contents of '''extras.vdf''' have been appended at the end, despite the <code>#include</code> statement being in the beginning of the file. That is because all include statements are executed after fully parsing the current list (Refer [https://github.com/ValveSoftware/source-sdk-2013/blob/39f6dde8fbc238727c020d13b05ecadd31bda4c0/src/tier1/KeyValues.cpp#L2365-L2385 this]). | |||
* You can also use the base statement to include other text files that has its own distinct behavior. <source lang="cpp">#base "file path"</source> | |||
According to the [https://github.com/ValveSoftware/source-sdk-2013/blob/39f6dde8fbc238727c020d13b05ecadd31bda4c0/src/tier1/KeyValues.cpp#L2132-L2180 source code], the way <code>#base</code> statements work is by parsing the included file the same way but instead of simply appending all of its keys at the end, it "merges" all keys with the current list. It does so by recursively going through every list in the current file and only appending new keys that don't exist in the file yet. | |||
'''Example:''' Say you want to include '''extras.vdf''' from the previous example in your '''main.vdf''' file. If you write '''main.vdf''' like this: | |||
<source lang="cpp"> | |||
#base "extras.vdf" | |||
"Key1" "Value1" | |||
"List" | |||
{ | |||
"InnerKey1" "InnerValue1" | |||
} | |||
</source> | |||
Then the '''main.vdf''' file will end up being loaded with the following contents: | |||
<source lang="cpp"> | |||
"Key1" "Value1" // Note how "extras.vdf" did not override the value here | |||
"List" | |||
{ | |||
"InnerKey1" "InnerValue1" // And did not override it here either | |||
"InnerKey2" "InnerExtra2" // But appended a new key to this list | |||
} | |||
"Key2" "Extra2" // Then appended another new key here | |||
</source> | |||
=== Comments === | |||
* The file format specification allows for C++ style comments. However, the implementation of this in Valve's Key-Value code is poor, and assumes anything following a single forward slash is a comment. EDIT : This does not appear to be the case (Refer [https://github.com/ValveSoftware/source-sdk-2013/blob/0d8dceea4310fde5706b3ce1c70609d72a38efdf/sp/src/tier1/utlbuffer.cpp#L406 this]). <source lang="cpp">// This is a comment until the line ends | |||
/ This is also a comment until the line ends | / This is also a comment until the line ends | ||
</source> | </source> | ||
* There is no support for block comments, and if contained in a single line, they will eliminate everything after them (due to being treated as a single-line comment). <source lang="cpp">/* This is an inline comment, which will not work in this file format. */ "this_pair_is_ignored" "true"</source> | * There is no support for block comments, and if contained in a single line, they will eliminate everything after them (due to being treated as a single-line comment). <source lang="cpp">/* This is an inline comment, which will not work in this file format. */ "this_pair_is_ignored" "true"</source> | ||
== | === Conditional Statements === | ||
* You can use conditional statements that will be ignored if non true, this will work on any key value and its children. If you prepend a key value statement with a <code>[$X360]</code> token or a <code>[$WIN32]</code> token it will be ignored depending on the platform in use. Since we have the source code for the keyvalues parser, you might be able to create your own conditional statements. | |||
==<font color="red">Important Notes</font>== | ==<font color="red">Important Notes</font>== | ||
Line 73: | Line 142: | ||
~CLeakTrack() | ~CLeakTrack() | ||
{ | { | ||
<span style="background-color: | <span style="background-color:#ff00005c;color:white;padding:2px 2px 2px 2px;">AssertMsg ( keys.Count() == 0, VarArgs("keys.Count() is %i",keys.Count()) );</span> | ||
for(int x=0;x<keys.Count();x++) | for(int x=0;x<keys.Count();x++) | ||
Line 122: | Line 191: | ||
:It is also valid to use the KeyValues member function <code>GetDataType</code> in the loop to do a type specific traverse. | :It is also valid to use the KeyValues member function <code>GetDataType</code> in the loop to do a type specific traverse. | ||
==Other | ==Other implementations== | ||
This section lists libraries in other programming languages that implements the Valve Data Format (VDF)/Key Values. | This section lists libraries in other programming languages that implements the Valve Data Format (VDF)/Key Values. | ||
===C=== | ===C=== | ||
* https://github.com/Jan200101/libofdf | * https://github.com/Jan200101/libofdf | ||
* https://github.com/DreamyCecil/VDF | |||
===C#=== | ===C#=== | ||
Line 138: | Line 208: | ||
* https://github.com/TinyTinni/ValveFileVDF | * https://github.com/TinyTinni/ValveFileVDF | ||
* https://github.com/OpenVicProject/lexy-vdf | * https://github.com/OpenVicProject/lexy-vdf | ||
* https://github.com/craftablescience/sourcepp | |||
===D=== | |||
* https://gitlab.com/midnaw/resmond | |||
* https://github.com/Yoplitein/keyvalues | |||
===Dart=== | ===Dart=== | ||
Line 171: | Line 246: | ||
* https://github.com/lukezbihlyj/vdf-parser | * https://github.com/lukezbihlyj/vdf-parser | ||
* https://github.com/EXayer/vdf-converter | * https://github.com/EXayer/vdf-converter | ||
===PowerShell=== | |||
* https://github.com/constup/vdf-converter-powershell | |||
===Rust=== | ===Rust=== | ||
Line 178: | Line 256: | ||
===Other=== | ===Other=== | ||
* VS-Code Extension: https://github.com/cooolbros/vscode-vdf | * VS-Code Extension: https://github.com/cooolbros/vscode-vdf | ||
* JetBrains Extension: https://plugins.jetbrains.com/plugin/22668-valve-data-format | |||
[[Category:Plain text formats]] | [[Category:Plain text formats]] |
Latest revision as of 03:18, 14 July 2025
The KeyValues format is used in the Source engine to store meta data for resources, scripts, materials, VGUI elements, and more...[Clarify]
Definition
The KeyValues class handles data buffer input, file input, and file output.
A KeyValue is defined recursively as a named key either with a value or children. All keys have names set to them and a search can be performed by the names. If a KeyValue is a parent key, it contains other KeyValues.
Value Types

SetFloat
) is called.TYPE_NONE
TYPE_STRING
TYPE_INT
TYPE_FLOAT
TYPE_PTR
TYPE_WSTRING
TYPE_COLOR
Example
"ParentKey1"
{
"ValueKey1" "1"
"ParentKey2"
{
...
}
}
File Format
The KeyValues.h header in the Source SDK code defines the file format in the following manner.
Basic Rules
- It has 3 control characters: {, } and ". Names and values may be quoted or not. The quote " character must not be used within name or values, only for quoting whole tokens. You may use escape sequences while parsing and add within a quoted token a \" to add quotes within your name or token.
- When using Escape Sequences, the parser must know that by setting KeyValues::UsesEscapeSequences( true ), which is off by default.
- Non-quoted tokens ends with a whitespace, {, } and ". So you may use { and } within quoted tokens, but not for non-quoted tokens.
- An open bracket { after a key name indicates a list of subkeys which is finished with a closing bracket }. Subkeys use the same definitions recursively.
- Whitespaces are space, return, newline and tabulator. Allowed Escape sequences are \n, \t, \\, and \".
- The number character # is used for macro purposes (eg #include), don't use it as first character in key names.
- Each token can be up to 1024 characters long (including null-termination, and double quotes, that will leave room for 1021 actual characters). Note that some functions in the KeyValues class will not handle strings with over 256 characters correctly. Searching for a key name over 256 characters containing a forward slash / character causes a buffer overrun on the stack in the
KeyValues::FindKey
function.
Please note that the above is only an informal description of the format. It is not a definitive guide for how the format should be interpreted by parsers. It has not been tested whether the KeyValues parser in the Source SDK Code parses exactly as according to the above. The KeyValues parser might not handle very long keyvalues correctly.
Include Statements
- You can use include statements in keyvalues scripts to include other text files.
#include "file path"
According to the source code, the way #include
statements work is by parsing the included file as usual and then appending all of its keys at the end of the currently parsed list, including duplicate keys.
Example: Say you want to include extras.vdf in your main.vdf file. If you write extras.vdf like this:
"Key1" "Extra1"
"Key2" "Extra2"
"List"
{
"InnerKey1" "InnerExtra1"
"InnerKey2" "InnerExtra2"
}
And main.vdf like this:
#include "extras.vdf"
"Key1" "Value1"
"List"
{
}
Then the main.vdf file will end up being loaded with the following contents:
"Key1" "Value1"
"List"
{
}
"Key1" "Extra1"
"Key2" "Extra2"
"List"
{
"InnerKey1" "InnerExtra1"
"InnerKey2" "InnerExtra2"
}
Note how the contents of extras.vdf have been appended at the end, despite the #include
statement being in the beginning of the file. That is because all include statements are executed after fully parsing the current list (Refer this).
- You can also use the base statement to include other text files that has its own distinct behavior.
#base "file path"
According to the source code, the way #base
statements work is by parsing the included file the same way but instead of simply appending all of its keys at the end, it "merges" all keys with the current list. It does so by recursively going through every list in the current file and only appending new keys that don't exist in the file yet.
Example: Say you want to include extras.vdf from the previous example in your main.vdf file. If you write main.vdf like this:
#base "extras.vdf"
"Key1" "Value1"
"List"
{
"InnerKey1" "InnerValue1"
}
Then the main.vdf file will end up being loaded with the following contents:
"Key1" "Value1" // Note how "extras.vdf" did not override the value here
"List"
{
"InnerKey1" "InnerValue1" // And did not override it here either
"InnerKey2" "InnerExtra2" // But appended a new key to this list
}
"Key2" "Extra2" // Then appended another new key here
Comments
- The file format specification allows for C++ style comments. However, the implementation of this in Valve's Key-Value code is poor, and assumes anything following a single forward slash is a comment. EDIT : This does not appear to be the case (Refer this).
// This is a comment until the line ends / This is also a comment until the line ends
- There is no support for block comments, and if contained in a single line, they will eliminate everything after them (due to being treated as a single-line comment).
/* This is an inline comment, which will not work in this file format. */ "this_pair_is_ignored" "true"
Conditional Statements
- You can use conditional statements that will be ignored if non true, this will work on any key value and its children. If you prepend a key value statement with a
[$X360]
token or a[$WIN32]
token it will be ignored depending on the platform in use. Since we have the source code for the keyvalues parser, you might be able to create your own conditional statements.
Important Notes
- It is important to note that you must include filesystem.h. Not doing this will result in the inability to reliably call LoadFromFile and SaveToFile. Symptoms are crashes in datastore.dll or a fail-state.
- SaveToFile will not save any keyvalues whose value string is empty, although these can be loaded without any problems.
- In
tier1/KeyValues.cpp
, uncomment this line to hit the ~CLeakTrack assert to see what's looking like it's leaking
// #define LEAKTRACK
- When you have completed using a KeyValues object, if you are unsure, check if is NULL and if not, call the object's
deleteThis
function. - If you, or any code utilizing the KeyValues object, fail to delete your KeyValues objects, you will end up with memory leaks.
- Many instances of the usage of the KeyValues in the SDK do not call their
deleteThis
function, therefore making memory leaks.
CLeakTrack
This class is in tier1/KeyValues.cpp
.
Replacing the deconstructor for the CLeakTrack class can help you stomp out KeyValues leaks when the engine shuts down.
~CLeakTrack()
{
AssertMsg ( keys.Count() == 0, VarArgs("keys.Count() is %i",keys.Count()) );
for(int x=0;x<keys.Count();x++)
{
keys[x].kv->deleteThis();
keys.Remove(x);
}
}
When you do this in debug build, you should get told how many leaks you get when you shutdown the game. This number may vary for the amount of activity done within the game.
Although this modification can remove all leaked memory, it should not be relied on to take care of them. Some memory leaks are caused by the engine, so this is a last resort for those kinds of leaks.
Example
ComboBox *pCombo = new ComboBox(...); pCombo->AddItem("Text", new KeyValues("UserData", "Key", "Value"));
It's important to note that if you were to follow what is called by the AddItem function, you would see that the UserData KeyValues are copied, rather than used directly. Copying KeyValues is a common source of confusion of how your KeyValues objects are used, so make sure to follow where they go. If you are unsure, you could try calling the deleteThis
function and see if the same functionality as before still exists.
Fix
ComboBox *pCombo = new ComboBox(...); KeyValues *kv = new KeyValues("UserData", "Key", "Value"); pCombo->AddItem("Text", kv); kv->deleteThis();
What not to do
ComboBox *pCombo = new ComboBox(...); pCombo->AddItem("Text", new KeyValues("UserData", "Key", "Value"));
Button *pButton = new Button(...); KeyValues *kv = new KeyValues("ButtonCommand"); pButton->SetCommand(kv); kv->deleteThis();
Useful Traverses
The following loops are contained in the following function: void Traverse(KeyValues *pKV);
All Subkeys
for ( KeyValues *sub = pKV->GetFirstSubKey(); sub; sub = sub->GetNextKey() )
- This loop steps through all subkeys contained in pKV.
- To handle subkeys with their own subkeys in pKV, you may want use a recursive loop. To check if a subkey has subkeys, do the following:
if(sub->GetFirstSubKey()) { Traverse(sub); }
All Values
for ( KeyValues *sub = pKV->GetFirstValue(); sub; sub = sub->GetNextValue() )
- This loop steps through all subkeys contained in a pKV that have values.
- It is also valid to use the KeyValues member function
GetDataType
in the loop to do a type specific traverse.
Other implementations
This section lists libraries in other programming languages that implements the Valve Data Format (VDF)/Key Values.
C
C#
- https://github.com/sanmadjack/VDF
- https://github.com/shravan2x/Gameloop.Vdf
- https://github.com/Indieteur/Steam-Apps-Management-API
- https://github.com/GerhardvanZyl/Steam-VDF-Converter
C++
- https://github.com/ozxybox/SpeedyKeyV
- https://github.com/TinyTinni/ValveFileVDF
- https://github.com/OpenVicProject/lexy-vdf
- https://github.com/craftablescience/sourcepp
D
Dart
Go
- https://github.com/andygrunwald/vdf
- https://github.com/marshauf/keyvalues
- https://github.com/Jleagle/steam-go
- https://github.com/Wakeful-Cloud/vdf
Java
JavaScript
- https://github.com/rossengeorgiev/vdf-parser
- https://github.com/node-steam/vdf
- https://github.com/Corecii/steam-binary-vdf-ts
- https://github.com/RoyalBingBong/vdfplus
- https://github.com/key-values/key-values-ts
Lua
Python
- https://github.com/ValvePython/vdf
- https://github.com/gorgitko/valve-keyvalues-python
- https://github.com/TeamSpen210/srctools, also on PyPI
PHP
- https://github.com/rossengeorgiev/vdf-parser
- https://github.com/devinwl/keyvalues-php
- https://github.com/lukezbihlyj/vdf-parser
- https://github.com/EXayer/vdf-converter
PowerShell
Rust
Other
- VS-Code Extension: https://github.com/cooolbros/vscode-vdf
- JetBrains Extension: https://plugins.jetbrains.com/plugin/22668-valve-data-format