Decompiling Maps: Difference between revisions

From Valve Developer Community
Jump to navigation Jump to search
 
(121 intermediate revisions by 44 users not shown)
Line 1: Line 1:
Before you read on you should be aware that using (parts or full copies of) decompiled maps is widely frowned upon by the majority of professional level designers, for good reason. When you decompile a map you are taking a designer's hard work and using it for yourself. Before you decompile maps as you please and upload the [[VMF]]s to the internet, please take a moment to think about your dedicating a small portion of your life to something and then someone breaking it into a million pieces.  Remember also that map decompilation is imperfect, and can cause quite a few issues in the decompiled maps.  For this reason, in addition to moral reasons, it is usually a bad idea to recompile a decompiled map with the intent to release it.
{{LanguageBar}}
[[Category:Level Design]]
[[Category:Tutorials]]


When you decompile a map it should be for one of four reasons:
Decompiling is the process of taking a compiled map file and generating a file that Hammer can open. Decompiled maps are not perfect recreations, because of format differences{{clarify}}, loss of data (such as missing [[Func viscluster]], tools textures such as NPC Clip or Player Clip all replaced with either Nodraw or entirely replaced with NPC Clip/Player Clip which cause lighting issues or NPCs/players to get stuck in some area, and imprecise vertex position on complex brushes). This makes them useful for learning, measurement, and inspection, but not direct recompilation. The original source is ideal for reproductions or modifications.
* You are curious about how a certain part of that person's map works.
* You are going to greatly change the map and only use the skeleton, and of course give the author of the original map credit to your work.
* You are trying to create an 'Advanced map', such as the ones in [[Portal]]. Again, full credit should be given to the original author.
* Your VMF is somehow corrupt and you don't have a recent back up.


If you intend to use someone else's work in your project (whether it be a map, a model, etc...) always try to '''get that person's permission''' and, at the very least, '''always''' give them credit.
Releasing reproductions, modifications, or approximations of another's work without permission nor credit may be considered rude to the author, is largely frowned upon, and is in fact copyright infringement, which may have legal consequences depending on the source of the map. There are methods of preventing maps from being fully decompiled (obfuscating textures, keeping entities or VScript server-side, etc.), but these are rarely used because most people who decompile maps are doing so to learn from them rather than to steal.


== The hacker's toolbox ==
==Decompilers==
For decompiling maps the following tools have been proven to work:
==={{source2|4}}===
* [[Entspy|EntSpy]]
*{{vrf|4}} (used to view the entities and other lumps on a map, decompiling to vmap, and exporting to glTF)
* [[BSPSource]] ([[Vmex]] for old maps)
==={{src|4}}===
*[[EntSpy]] (used to view and edit the entities of a BSP without recompiling)
*[[BSPSource]] (open source map decompiler for Source engine maps, based on VMEX)
*[[VMEX]] (decompiler for Source engine maps)
*[[MapFool]] (helps porting HL1 maps to Source)
*[[xwad]] (command-line tool to convert texture formats)
==={{gldsrc|4}}===
* {{bspviewer|4}} requires .NET Framework v1.1. (not recommended)
:{{Warning|In {{crafty|4}} the function of bsp decompilation was cut out.}}
*{{hlusmd|4}} requires .NET Desktop Runtime 6
*{{mbspc|4}} (not recommended)
*{{b2m|4}} (not recommended)
*{{winbspc|4}} (not recommended)
{{warning|[[BSP2MAP]], [[MBSPC]], [[005]] and [[WinBSPC]] are unstable and may not compile correctly in the end; {{hlusmd|1}} instead. It will still require manual cleanup, but is significantly more accurate.}}
*{{005|4}} ({{hl}} not recommended; {{nightfire}} recommended)


'''Optional'''
==Files==
* [[BSP2MAP]]
Most maps do not need to be extracted, as they are located in the file system rather than VPK. The map files are normally located in a ''maps'' folder, for example <code>common\Team Fortress 2\tf\maps\</code> holds all the official Team Fortress 2 maps.
* [[MapFool]]
{{note|A few games may have extra maps sectioned into other mod folders, for example <code>Portal 2\portal2_dlc1\maps\</code>.}}
* [[xwad]]
{{note|A few games use [[Patching_levels_with_lump_files|lump file edits]] to edit existing entities,  and [[Commentary_System|comment files]] to add new entities without recompiling maps. for example: {{l4d2}} adds [[info_remarkable]] entities and [[env_player_blocker]] entities this way.<br>
* [[WinBSPC]]
BSPSource is capable of reading the .lmp files and incorporate its data, but it will not load entities from the commentary files. Luckily you can convert commentary files back to VMF files by following [[Commentary_System#Converting_Commentary_back_to_VMF|these steps]]. and merging those entities into the decompiled map.}}
* [[BSP Viewer]]


== Getting dirty ==
==Before you go==
To start off, you are required to have installed the Java Runtime Environment (JRE), which can be downloaded from [http://java.sun.com/javase/downloads/ http://java.sun.com/javase/downloads/]. It allows your computer to run Java applications. Once installed, you are ready to decompile your first map. Download VMEX and copy it to the Source SDK bin folder. You'll notice VMEX comes with two .bat files which can be run to start up VMEX.
Before you take this jump to run off and decompile a map take a second or two to think, "What am I trying to learn/do?".
 
Read the included TXT files, you may just find the creator's e-mail, and that could be more helpful than you imagine.
To decompile a map run VMEX and browse to the map, then hit decompile.
 
{{note|VMEX now supports Episode Two maps too.}}
 
== Playing in the mud ==
Through your adventures into the world of open source mapping, you might come across some words that will make you scratch your head, "**** This map is protected from decompiling."  This output is produced by VMEX when you try to decompile a map that someone really did not want decompiled.  If you are a map maker and you want your map to produce this kind of output for those who try, just add a keyvalue called '''"no_decomp"''', with a value of 1, to any entity.  But do not jump up and down yet, because as easy as that process is, its even easier to undo it and safely decompile the map again.
 
For those of you who do find one of these rare maps, this is where [[Entspy|EntSpy]] comes into play.  Open the map in EntSpy and search for the light_environment entity, when you find it, go to the window directly to the right and find a box that says "no_decomp 1" and simply remove it.  Then save the map in EntSpy and go back to VMEX to decompile the map. 
If you cannot find the key word in light_environment, you might have to go searching.  Look in the common entities first, the ones that usually are only created once and will not break the map if they fail.
 
For those who are more pro, what you can do is load the bsp into WordPad and search for "no_decomp" yourself.  Then make a mental notes of which entity it's in and go back to EntSpy to finish the job.
 
== Porting [[Half-Life]] maps ==
{{note|If the map you want to port is open-source, use the map file provided to load into Hammer. If an RMF file is provided, it has to be saved as a map file using the old Hammer.}}
 
This is what had the most amount of attention right after Source came out.  People wanted to port their old 1.3 maps to source, or, people wanted to "help" someone else by doing it for them.  To do this little hack you will need [[BSP2MAP]] or [[BSP Viewer]] and [[MapFool]], as well as all the wads that the map uses (texture packs). 
If you use [[BSP2MAP]] then go into the command prompt (Start->Run->type cmd) and navigating to the folder BSP2MAP is in.  Run the program with the argument being the filename of the map (including .bsp) you want to decompile.  If it succeeds, you're in.
 
The other option is to use [[BSP Viewer]].  Run the program and load the map you want.  Then goto File->Export->Map File, and the program will generate a map file for you.  This program will generate a map file for all maps, so you do not have to worry about a windows error like in [[BSP2MAP]], however, this program does not produce the best quality map files.  More solids are lost and even worse, it creates irregular spacing in the file that neither hammer nor [[MapFool]] can read.  Once you get the map file you will need to go into the file and replace all the spaces with nothing.  You can do this by opening notepad and replacing all <nowiki>"&nbsp;&nbsp;"</nowiki>(double space) with nothing.  Notepad does this really slow so you will probably want to do it overnight, or you can get a better program that does search and replace faster. 
 
So now that you have a brand new .map you can load it into hammer and look around. Pretty devastating isn't it? All those grey walls...  But this is where MapFool steps up to the plate.  Close hammer and open MapFool.  Load your map into the program and you are confronted by a big text box with all the raw data.  There is only one line in all this mess you should be worried about.  Look just below '''mapversion''' and you will see '''wad''' and a list of wad files.  Hopefully you know where all these are because now is when you need them.  Click File->Options and hit the "Add" button to add your wad files.  After you think you got them all, go just below that list and enter the vtex path and the texture's folder path.
 
Next you are going to want to use the Tools->Upgrade Obsolete Entities program to do just that, update all the func_walls and such that source does not use. 
 
Now the fun part, go to Tools->Export Textures to TGA and export them to a folder with a bit of space and memerable <!--memerable: can't understand what this means!-->.  Next go to the item below that called "Make Materials..." in the text box labeled TGA files, navigate to the folder you just exported all those textures to and you should see a list of files appear in the list box below.  You'll notice some textures are red and have some text saying they are not powers of two.  This is because source requires all textures to have sides that are exactly 2, 4, 8, 16, 32, 64, 128, 256, 512, or 1024.  A bit annoying really because if you are lucky, about half of the textures you need are that size. 
 
[[Image:Ava d.jpg|thumb|A snap shot of ''avalanche'' from DoD 1.3, decompiled and loaded into Hammer]]
 
If you want these textures to show up in hammer, just navigate to that folder containing all the TGAs and open the ones that are not powers of two in Photoshop.  Then its just a simple matter of making both sides a power of two.
 
 
 
After that's all cleared up go ahead and click proceed in the options box and watch as MapFool converts all the textures in the map to materials that will show up in hammer. 
 
If you open your map in Hammer now, (hopefully you exported those textures into a folder that Hammer will look in, i.e. in the source <code>cstrike\materials</code> folder), the map should look a bit more complete. If everything is still grey then you might have to go back into MapFool and use Tools->Texture Folder name->Prepend to prepend the character "/" onto each texture name.  What usually happens is Hammer looks for the textures in <gamedir>/materials and not <gamedir>/materials/ a small difference, but one that will come back to bite you. 
 
When all is said and done, you should have an almost perfect replica of your favorite HL1 map.  Do not try and compile it right off the bat, you might have gotten a warning message from Hammer saying it wasn't able to load a X amount of solids.  That is because BSP2MAP and BSP Viewer have to reconstruct all the brushes and no program can do that big of a task and not get something wrong.  The way these programs work is instead of creating real solids, it creates faces of solids.  A box in a HL1 map turns into 6 1 unit thick walls after being decompiled.  This causes a number of errors in Hammer, the major one being the max vertice count and max brush count.  As both of those are pretty much guaranteed to be exceeded. 
 
After you take a good look at this map, and you are still determined to port it, then you will have to go through and stretch all the 1 unit thick walls into their proper places as real solids.  That means for a box you will have to stretch one side of it and delete the other 5 walls.  Needless to say, this is the reason why there are only a few Half-Life 1 to Source ports.
 
To understand why this happens you need to understand the format of a map file, and how hammer and the HL1 engine reads maps, which is beyond the scope of this article. If you are terribly interested however, post something and there might be an article written later about it.
 
 
'''Link:''' Another way to Port Half-Life 1 maps. A Plan View image and Entities only: [http://www.dodbits.com/dod/index.php?option=com_content&view=article&id=89:porting-old-half-life-1-maps&catid=36:half-life-2-mapping&Itemid=69 dodbits.com Porting a HL1 map to Source]
 
== Levels of security for map makers ==
The method mentioned above is one of the simplest ways to protect your map from being decompiled.  VMEX offers three levels of security.  A slightly more secure method is to make one brush in your map and texture it with '''"tools/locked"''', this is not a real texture so you will have to create it.  The most secure way you can stop the hackers is by downloading VMEX's custom prefab from [http://www.bagthorpe.org/bob/cofrdrbob/files/protector.zip here], add it anywhere to your map, but do not rotate, scale, or retexture it.  All three of these can still be bypassed of course, but only by those who absolutely know what they are doing.  And when someone does eventually decompile your map, who better to have the vmf than a pro?
 
Map designers need to know that people will get a hold of your VMF in some version or another, whether it be by your hand or not.  Adding protection like this only delays the inevitable.  For the most part, those who get a VMF only look, but if you do eventually login to a server that is running your map + sniper tower in the middle, yelling at the person will not help anyone.  Just feel confident that your original design will win over some poor guys attempt to create his own map. What does one server running his bad version of your map affect you anyway?


Also, unlike HL1 maps, the Source format stores practically all the information for the map in the BSP.  Nothing is interpreted by the engine.  Thus, all that someone would need is a $10 copy of DJ Java Decompiler, a bit of free time, and a drive to hack your map and they can take VMEX and edit out all of the protections.  The main point being, if you want to create a map that no one else will be able to change, you better do it for another engine because Source might as well be called ''open'' Source.
Some mappers will offer help and advice, they may even go so far as to provide examples of what you wanted to borrow from the map.


== Before you go ==
==Porting newer maps to older Source games==
Before you take this jump to run off and decompile a map take a second or two to think, "What am I trying to learn/do?".
Decompiled maps from [[Left 4 Dead 2]], [[Portal 2]], [[Counter-Strike: Global Offensive]], and [[Alien Swarm]] will crash when loaded in the Hammer for [[Half-Life 2]], [[Team Fortress 2]], [[Counter-Strike: Source]], [[Day of Defeat: Source]] and other games based on {{src13|1}}, [[Orange_Box_(engine_branch)|Orange Box branches]] ({{src07}} - {{src09}}/{{srcmp}}) or earlier. Opening the map in [[Left 4 Dead]] Hammer and re-saving it will automatically convert it to be compatible.
Read the included TXT files, you may just find the creator's e-mail, and that could be more helpful than you imagine.
{{Note|This VMF incompatibility issue does not affect {{gmod|1}}, as it ran on heavily modified version of {{src13|1}} that support assets from {{l4dbranch|1}} and later. Garry's Mod ships with modified version of Hammer which fixed the issue, allowing you to load VMFs (or decompiled maps) from Left 4 Dead 2 and later branches.}}


Some mappers will offer up help and advice, they may even go so far as to provide perfect examples of what you were trying to "borrow" from the map.
{{todo|Document how to fix this manually. It's caused by L4D2 changing the character used to separate output values, but I'm not sure what character it changed to.}}


[[Category:Level Design]]
==See also==
[[Category:Tutorials]]
*[[Porting GoldSrc content (maps, models, etc.) to Source]]
[[Category:Source]]

Latest revision as of 04:59, 9 July 2025

English (en)한국어 (ko)Русский (ru)中文 (zh)Translate (Translate)

Decompiling is the process of taking a compiled map file and generating a file that Hammer can open. Decompiled maps are not perfect recreations, because of format differences[Clarify], loss of data (such as missing Func viscluster, tools textures such as NPC Clip or Player Clip all replaced with either Nodraw or entirely replaced with NPC Clip/Player Clip which cause lighting issues or NPCs/players to get stuck in some area, and imprecise vertex position on complex brushes). This makes them useful for learning, measurement, and inspection, but not direct recompilation. The original source is ideal for reproductions or modifications.

Releasing reproductions, modifications, or approximations of another's work without permission nor credit may be considered rude to the author, is largely frowned upon, and is in fact copyright infringement, which may have legal consequences depending on the source of the map. There are methods of preventing maps from being fully decompiled (obfuscating textures, keeping entities or VScript server-side, etc.), but these are rarely used because most people who decompile maps are doing so to learn from them rather than to steal.

Decompilers

Source 2 Source 2

  • Source 2 Viewer Source 2 Viewer (used to view the entities and other lumps on a map, decompiling to vmap, and exporting to glTF)

Source Source

  • EntSpy (used to view and edit the entities of a BSP without recompiling)
  • BSPSource (open source map decompiler for Source engine maps, based on VMEX)
  • VMEX (decompiler for Source engine maps)
  • MapFool (helps porting HL1 maps to Source)
  • xwad (command-line tool to convert texture formats)

GoldSrc GoldSrc

  • BSP Viewer BSP Viewer requires .NET Framework v1.1. (not recommended)
Warning.pngWarning:In Crafty Crafty the function of bsp decompilation was cut out.
Warning.pngWarning:BSP2MAP, MBSPC, 005 and WinBSPC are unstable and may not compile correctly in the end; Half-Life Unified SDK Map Decompiler instead. It will still require manual cleanup, but is significantly more accurate.

Files

Most maps do not need to be extracted, as they are located in the file system rather than VPK. The map files are normally located in a maps folder, for example common\Team Fortress 2\tf\maps\ holds all the official Team Fortress 2 maps.

Note.pngNote:A few games may have extra maps sectioned into other mod folders, for example Portal 2\portal2_dlc1\maps\.
Note.pngNote:A few games use lump file edits to edit existing entities, and comment files to add new entities without recompiling maps. for example: Left 4 Dead 2 adds info_remarkable entities and env_player_blocker entities this way.
BSPSource is capable of reading the .lmp files and incorporate its data, but it will not load entities from the commentary files. Luckily you can convert commentary files back to VMF files by following these steps. and merging those entities into the decompiled map.

Before you go

Before you take this jump to run off and decompile a map take a second or two to think, "What am I trying to learn/do?". Read the included TXT files, you may just find the creator's e-mail, and that could be more helpful than you imagine.

Some mappers will offer help and advice, they may even go so far as to provide examples of what you wanted to borrow from the map.

Porting newer maps to older Source games

Decompiled maps from Left 4 Dead 2, Portal 2, Counter-Strike: Global Offensive, and Alien Swarm will crash when loaded in the Hammer for Half-Life 2, Team Fortress 2, Counter-Strike: Source, Day of Defeat: Source and other games based on Source 2013, Orange Box branches (Source 2007 - Source 2009/Source Multiplayer) or earlier. Opening the map in Left 4 Dead Hammer and re-saving it will automatically convert it to be compatible.

Note.pngNote:This VMF incompatibility issue does not affect Garry's Mod, as it ran on heavily modified version of Source 2013 that support assets from Left 4 Dead engine branch and later. Garry's Mod ships with modified version of Hammer which fixed the issue, allowing you to load VMFs (or decompiled maps) from Left 4 Dead 2 and later branches.
Todo: Document how to fix this manually. It's caused by L4D2 changing the character used to separate output values, but I'm not sure what character it changed to.

See also

Retrieved from "https://developer.valvesoftware.com/w/index.php?title=Decompiling_Maps&oldid=484956"