Decompiling Maps: Difference between revisions

From Valve Developer Community
Jump to navigation Jump to search
 
(152 intermediate revisions by 64 users not shown)
Line 1: Line 1:
[[Category:Level Design]] [[Category:Tutorials]]
{{LanguageBar}}
Before you read on make sure you know that decompiling maps can be a very sticky business.  It is widly frowned upon by the majority of professional level designers.  For good reason, when you decompile a map you are taking their hard work and using it for yourself.  So before you get the idea to go around decompiling every map and throwing the vmf's up on the internet, please take the time and think about something you have dedicated a small portion of your life to, and then imagine someone breaking it into a million pieces.
[[Category:Level Design]]
[[Category:Tutorials]]


When you decompile a map you should be thinking one of two things:
Decompiling is the process of taking a compiled map file and generating a file that Hammer can open. Decompiled maps are not perfect recreations, because of format differences{{clarify}}, loss of data (such as missing [[Func viscluster]], tools textures such as NPC Clip or Player Clip all replaced with either Nodraw or entirely replaced with NPC Clip/Player Clip which cause lighting issues or NPCs/players to get stuck in some area, and imprecise vertex position on complex brushes). This makes them useful for learning, measurement, and inspection, but not direct recompilation. The original source is ideal for reproductions or modifications.
* You are curious how a certain part of his/her map works
* You are going to greatly change the map and only use the skeleton, and of course give the author of the original map the full rights to your work.


Releasing reproductions, modifications, or approximations of another's work without permission nor credit may be considered rude to the author, is largely frowned upon, and is in fact copyright infringement, which may have legal consequences depending on the source of the map. There are methods of preventing maps from being fully decompiled (obfuscating textures, keeping entities or VScript server-side, etc.), but these are rarely used because most people who decompile maps are doing so to learn from them rather than to steal.


== The Hacker's Toolbox ==
==Decompilers==
To decompile maps I suggest the following tools:
==={{source2|4}}===
* [[EntSpy]]
*{{vrf|4}} (used to view the entities and other lumps on a map, decompiling to vmap, and exporting to glTF)
* [[Vmex]]
==={{src|4}}===
*[[EntSpy]] (used to view and edit the entities of a BSP without recompiling)
*[[BSPSource]] (open source map decompiler for Source engine maps, based on VMEX)
*[[VMEX]] (decompiler for Source engine maps)
*[[MapFool]] (helps porting HL1 maps to Source)
*[[xwad]] (command-line tool to convert texture formats)
==={{gldsrc|4}}===
* {{bspviewer|4}} requires .NET Framework v1.1. (not recommended)
:{{Warning|In {{crafty|4}} the function of bsp decompilation was cut out.}}
*{{hlusmd|4}} requires .NET Desktop Runtime 6
*{{mbspc|4}} (not recommended)
*{{b2m|4}} (not recommended)
*{{winbspc|4}} (not recommended)
{{warning|[[BSP2MAP]], [[MBSPC]], [[005]] and [[WinBSPC]] are unstable and may not compile correctly in the end; {{hlusmd|1}} instead. It will still require manual cleanup, but is significantly more accurate.}}
*{{005|4}} ({{hl}} not recommended; {{nightfire}} recommended)


'''Optional'''
==Files==
* [[BSP2MAP]]
Most maps do not need to be extracted, as they are located in the file system rather than VPK. The map files are normally located in a ''maps'' folder, for example <code>common\Team Fortress 2\tf\maps\</code> holds all the official Team Fortress 2 maps.
* [[MapFool]]
{{note|A few games may have extra maps sectioned into other mod folders, for example <code>Portal 2\portal2_dlc1\maps\</code>.}}
* [[xwad]]
{{note|A few games use [[Patching_levels_with_lump_files|lump file edits]] to edit existing entities,  and [[Commentary_System|comment files]] to add new entities without recompiling maps. for example: {{l4d2}} adds [[info_remarkable]] entities and [[env_player_blocker]] entities this way.<br>
* [[WinBSPC]]
BSPSource is capable of reading the .lmp files and incorporate its data, but it will not load entities from the commentary files. Luckily you can convert commentary files back to VMF files by following [[Commentary_System#Converting_Commentary_back_to_VMF|these steps]]. and merging those entities into the decompiled map.}}
* [[BSP Viewer]]


== Getting Dirty ==
==Before you go==
To start off, you are required to have installed the JRE, which can be downloaded from [http://java.sun.com/j2se/1.5.0/download.jsp http://java.sun.com/j2se/1.5.0/download.jsp].  Once that is installed you are ready to decompile your first map.  Download VMEX and copy it to the Source SDK bin folder.  You'll notice VMEX comes with two bat files which can be run to start up VMEX.
Before you take this jump to run off and decompile a map take a second or two to think, "What am I trying to learn/do?".
 
Read the included TXT files, you may just find the creator's e-mail, and that could be more helpful than you imagine.
To decompile a map run VMEX and browse to the map, then hit decompile.
 
== Playing in the Mud ==
Through your adventures into the world of open source mapping, you might come across some words that will make you scratch your head, "**** This map is protected from decompiling."  This output is produced by VMEX when you try to decompile a map that someone really did not want decompiled.  If you are a map maker and you want your map to produce this kind of output for those who try, just add a keyvalue called '''"no_decomp"''', with a value of 1, to any entity.  But do not jump up and down yet, because as easy as that process is, its even easier to undo it and safely decompile the map again.
 
For those of you who do find one of these rare maps, this is where [[EntSpy]] comes into play.  Open the map in EntSpy and search for the light_environment entity, when you find it, go to the window directly to the right and find a box that says "no_decomp 1" and simply remove it.  Then save the map in EntSpy and go back to VMEX to decompile the map. 
If you cannot find the key word in light_environment, you might have to go searching.  Look in the common entities first, the ones that usually are only created once and will not break the map if they fail.


For those who are more pro, what you can do is load the bsp into wordpad and search for no_decomp yourself.  Then make a mental notes of which entity it's in and go back to EntSpy to finish the job.
Some mappers will offer help and advice, they may even go so far as to provide examples of what you wanted to borrow from the map.


== Porting Half Life Maps ==
==Porting newer maps to older Source games==
{{note|If the map you want to port is open-source, use the map file provided to load into Hammer. If an rmf file is provided, it has to be saved as a map file using the old Hammer.}}
Decompiled maps from [[Left 4 Dead 2]], [[Portal 2]], [[Counter-Strike: Global Offensive]], and [[Alien Swarm]] will crash when loaded in the Hammer for [[Half-Life 2]], [[Team Fortress 2]], [[Counter-Strike: Source]], [[Day of Defeat: Source]] and other games based on {{src13|1}}, [[Orange_Box_(engine_branch)|Orange Box branches]] ({{src07}} - {{src09}}/{{srcmp}}) or earlier. Opening the map in [[Left 4 Dead]] Hammer and re-saving it will automatically convert it to be compatible.
 
{{Note|This VMF incompatibility issue does not affect {{gmod|1}}, as it ran on heavily modified version of {{src13|1}} that support assets from {{l4dbranch|1}} and later. Garry's Mod ships with modified version of Hammer which fixed the issue, allowing you to load VMFs (or decompiled maps) from Left 4 Dead 2 and later branches.}}
This is what had the most amount of attention right after Source came out.  People wanted to port their old 1.3 maps to source, or, people wanted to "help" someone else by doing it for them.  To do this little hack you will need [[BSP2MAP]] or [[BSP Veiwer]] and [[MapFool]], as well as all the wads that the map uses (texture packs). 
If you use [[BSP2MAP]] then go into the command prompt (Start->Run->type cmd) and navigating to the folder you hold BSP2MAP in.  Run the program with the argument being the filename of the map you want to decompile.  If it succeeds, your in.
 
The other option is to use [[BSP Viewer]].  Run the program and load the map you want.  Then goto File->Export->Map File, and the program will generate a map file for you.  This program will generate a map file for all maps, so you do not have to worry about a windows error like in [[BSP2MAP]], however, this program does not produce the best quality map files.  More solids are lost and even worse, it creates irregular spacing in the file that neither hammer nor [[MapFool]] can read.  Once you get the map file you will need to go into the file and replace all the spaces with nothing.  You can do this by opening notepad and replacing all <nowiki>"&nbsp;&nbsp;"</nowiki>(double space) with nothing.  Notepad does this really slow so you will probably want to do it overnight, or you can get a better program that does search and replace faster. 
 
So now that you have a brand new .map you can load it into hammer and look around.  Pretty devistating isn't it?  All those grey walls...  But this is where MapFool steps up to the plate.  Close hammer and open MapFool.  Load your map into the program and you are confronted by a big text box with all the raw data.  There is only one line in all this mess you should be worried about.  Look just below '''mapversion''' and you will see '''wad''' and a list of wad files.  Hopefully you know where all these are because now is when you need them.  Click File->Options and hit the "Add" button to add your wad files.  After you think you got them all, go just below that list and enter the vtex path and the texture's folder path. 
 
Next you are going to want to use the Tools->Upgrade Obsolete Entities program to do just that, update all the func_walls and such that source does not use. 
 
Now the fun part, go to Tools->Export Textures to tga... and export them to a folder with a bit of space and memerable.  Next go to the item below that called "Make Materials..." in the textbox labeled tga files, navigate to the folder you just exported all those textures to and you should see a list of files appear in the list box below.  You'll notice some textures are red and have some text saying they are not powers of two.  This is because source requires all textures to have sides that are exactly 2, 4, 8, 16, 32, 64, 128, 256, 512, or 1024.  A bit annoying really because if you are lucky, about half of the textures you need are that size. 
 
[[Image:Ava d.jpg|thumb|A snap shot of ''avalanche'' from DoD 1.3, decompiled and loaded into Hammer]]
 
If you want these textures to show up in hammer, just navigate to that folder containing all the tga's and open the ones that are not powers of two in photoshop.  Then its just a simple matter of making both sides a power of two.
 
 
 
After thats all cleared up go ahead and click proceed in the options box and watch as MapFool converts all the textures in the map into materials that will show up in hammer. 
 
If you open your map in hammer now, (hopefully you exported those textures into a folder that hammer will look in (eg, materials, in the source cstrike folder), the map should look a bit more complete.  If everything is still grey then you might have to go back into MapFool and use Tools->Texture Folder name->Prepend to prepend the character "/" onto each texture name.  What usually happens is Hammer looks for the textures in <gamedir>/materials and not <gamedir>/materials/ a small difference, but one that will come back to bite you.
 
When all is said and done, you should have an almost perfect replica of your favorite HL1 map.  Do not try and compile it right off the bat, you might have gotten a warning message from Hammer saying it wasnt able to load a X amount of solids.  That is because BSP2MAP had to reconstruct all the brushes and no program can do that big of a task and not get something wrong.  You might notice in your map there are vertices that strech all the way around your map and there might be a box that is sitting 200 units below the level.  You must fix these first, then you can have all the fun you want.
 
== Levels of Security for Map makers ==
The method mentioned above is one of the simplest ways to protect your map from decompilation.  VMEX offers three levels of security.  A slightly more secure method is to make one brush in your map and texture it with '''"tools/locked"''', this is not a real texture so you will have to create it. The most secure way you can stop the hackers is by downloading VMEX's custom prefab from [http://www.geocities.com/cofrdrbob/files/protector.zip here], add it anywhere to your map, but do not rotate, scale, or retexture it.  All three of these can still be bypassed of course, but only by those who absolutly know what they are doing.  And when someone does eventually decompile your map, who better to have the vmf then a pro? 
 
Map designers need to know that people will get ahold of your vmf in some version or another, whether it be by your hand or not.  Adding protection like this only delays the inevitable.  For the most part, those who get a vmf only look, but if you do eventually login to a server that is running your map + sniper tower in the mid, yelling at the person will not help anyone.  Just feel confident that your original design will win over some poor guys attempt to create his own map.  What does one server running his bad version of your map affect anyway.
 
Also, unlike HL1 maps, the source format stores practically all the information for the map in the bsp.  Nothing is interpreted by the engine.  Thus, all that someone would need is a $10 copy of DJ Java Decompiler, a bit of free time, and a drive to hack your map and they can take VMEX and edit out all of the protections.  The main point being, if you want to create a map that no one else will be able to change, you better do it for another engine because source might as well be called open source.
 
== Before you go ==
Before you take this jump to run off and decompile a map take a second or two to think, "What am I trying to learn/do?".
Read the included txt files, you may just find the creators e-mail, and that could be more helpful than you imagine.


Some mappers will offer up help and advice, they may even go so far as to provide perfect examples of what you were trying to "borrow" from the map.
{{todo|Document how to fix this manually. It's caused by L4D2 changing the character used to separate output values, but I'm not sure what character it changed to.}}


What's better, copying someone else's test and getting the answers then, or having someone teach you what to do so you can do it yourself next time.
==See also==
*[[Porting GoldSrc content (maps, models, etc.) to Source]]
[[Category:Source]]

Latest revision as of 04:59, 9 July 2025

English (en)한국어 (ko)Русский (ru)中文 (zh)Translate (Translate)

Decompiling is the process of taking a compiled map file and generating a file that Hammer can open. Decompiled maps are not perfect recreations, because of format differences[Clarify], loss of data (such as missing Func viscluster, tools textures such as NPC Clip or Player Clip all replaced with either Nodraw or entirely replaced with NPC Clip/Player Clip which cause lighting issues or NPCs/players to get stuck in some area, and imprecise vertex position on complex brushes). This makes them useful for learning, measurement, and inspection, but not direct recompilation. The original source is ideal for reproductions or modifications.

Releasing reproductions, modifications, or approximations of another's work without permission nor credit may be considered rude to the author, is largely frowned upon, and is in fact copyright infringement, which may have legal consequences depending on the source of the map. There are methods of preventing maps from being fully decompiled (obfuscating textures, keeping entities or VScript server-side, etc.), but these are rarely used because most people who decompile maps are doing so to learn from them rather than to steal.

Decompilers

Source 2 Source 2

  • Source 2 Viewer Source 2 Viewer (used to view the entities and other lumps on a map, decompiling to vmap, and exporting to glTF)

Source Source

  • EntSpy (used to view and edit the entities of a BSP without recompiling)
  • BSPSource (open source map decompiler for Source engine maps, based on VMEX)
  • VMEX (decompiler for Source engine maps)
  • MapFool (helps porting HL1 maps to Source)
  • xwad (command-line tool to convert texture formats)

GoldSrc GoldSrc

  • BSP Viewer BSP Viewer requires .NET Framework v1.1. (not recommended)
Warning.pngWarning:In Crafty Crafty the function of bsp decompilation was cut out.
Warning.pngWarning:BSP2MAP, MBSPC, 005 and WinBSPC are unstable and may not compile correctly in the end; Half-Life Unified SDK Map Decompiler instead. It will still require manual cleanup, but is significantly more accurate.

Files

Most maps do not need to be extracted, as they are located in the file system rather than VPK. The map files are normally located in a maps folder, for example common\Team Fortress 2\tf\maps\ holds all the official Team Fortress 2 maps.

Note.pngNote:A few games may have extra maps sectioned into other mod folders, for example Portal 2\portal2_dlc1\maps\.
Note.pngNote:A few games use lump file edits to edit existing entities, and comment files to add new entities without recompiling maps. for example: Left 4 Dead 2 adds info_remarkable entities and env_player_blocker entities this way.
BSPSource is capable of reading the .lmp files and incorporate its data, but it will not load entities from the commentary files. Luckily you can convert commentary files back to VMF files by following these steps. and merging those entities into the decompiled map.

Before you go

Before you take this jump to run off and decompile a map take a second or two to think, "What am I trying to learn/do?". Read the included TXT files, you may just find the creator's e-mail, and that could be more helpful than you imagine.

Some mappers will offer help and advice, they may even go so far as to provide examples of what you wanted to borrow from the map.

Porting newer maps to older Source games

Decompiled maps from Left 4 Dead 2, Portal 2, Counter-Strike: Global Offensive, and Alien Swarm will crash when loaded in the Hammer for Half-Life 2, Team Fortress 2, Counter-Strike: Source, Day of Defeat: Source and other games based on Source 2013, Orange Box branches (Source 2007 - Source 2009/Source Multiplayer) or earlier. Opening the map in Left 4 Dead Hammer and re-saving it will automatically convert it to be compatible.

Note.pngNote:This VMF incompatibility issue does not affect Garry's Mod, as it ran on heavily modified version of Source 2013 that support assets from Left 4 Dead engine branch and later. Garry's Mod ships with modified version of Hammer which fixed the issue, allowing you to load VMFs (or decompiled maps) from Left 4 Dead 2 and later branches.
Todo: Document how to fix this manually. It's caused by L4D2 changing the character used to separate output values, but I'm not sure what character it changed to.

See also

Retrieved from "https://developer.valvesoftware.com/w/index.php?title=Decompiling_Maps&oldid=484956"