Before you read on you should be aware that decompiling maps can be a very sticky business. It is widely frowned upon by the majority of professional level designers, for good reason. When you decompile a map you are taking a designer's hard work and using it for yourself. So, before you go around decompiling maps as you please and throwing the VMFs up on the internet, please take a moment to think about yourself dedicating a small portion of your life to something and then someone breaking it into a million pieces.
When you decompile a map it should be for one of two reasons:
- You are curious how a certain part of his/her map works
- You are going to greatly change the map and only use the skeleton, and of course give the author of the original map the full rights to your work.
The Hacker's Toolbox
For decompiling maps the following tools have been proven to work:
To start off, you are required to have installed the JRE, which can be downloaded from http://java.sun.com/j2se/1.5.0/download.jsp. Once that is installed you are ready to decompile your first map. Download VMEX and copy it to the Source SDK bin folder. You'll notice VMEX comes with two bat files which can be run to start up VMEX.
To decompile a map run VMEX and browse to the map, then hit decompile.
Playing in the Mud
Through your adventures into the world of open source mapping, you might come across some words that will make you scratch your head, "**** This map is protected from decompiling." This output is produced by VMEX when you try to decompile a map that someone really did not want decompiled. If you are a map maker and you want your map to produce this kind of output for those who try, just add a keyvalue called "no_decomp", with a value of 1, to any entity. But do not jump up and down yet, because as easy as that process is, its even easier to undo it and safely decompile the map again.
For those of you who do find one of these rare maps, this is where EntSpy comes into play. Open the map in EntSpy and search for the light_environment entity, when you find it, go to the window directly to the right and find a box that says "no_decomp 1" and simply remove it. Then save the map in EntSpy and go back to VMEX to decompile the map. If you cannot find the key word in light_environment, you might have to go searching. Look in the common entities first, the ones that usually are only created once and will not break the map if they fail.
For those who are more pro, what you can do is load the bsp into wordpad and search for no_decomp yourself. Then make a mental notes of which entity it's in and go back to EntSpy to finish the job.
Porting Half Life Maps
This is what had the most amount of attention right after Source came out. People wanted to port their old 1.3 maps to source, or, people wanted to "help" someone else by doing it for them. To do this little hack you will need BSP2MAP or BSP Viewer and MapFool, as well as all the wads that the map uses (texture packs). If you use BSP2MAP then go into the command prompt (Start->Run->type cmd) and navigating to the folder you hold BSP2MAP in. Run the program with the argument being the filename of the map you want to decompile. If it succeeds, your in.
The other option is to use BSP Viewer. Run the program and load the map you want. Then goto File->Export->Map File, and the program will generate a map file for you. This program will generate a map file for all maps, so you do not have to worry about a windows error like in BSP2MAP, however, this program does not produce the best quality map files. More solids are lost and even worse, it creates irregular spacing in the file that neither hammer nor MapFool can read. Once you get the map file you will need to go into the file and replace all the spaces with nothing. You can do this by opening notepad and replacing all " "(double space) with nothing. Notepad does this really slow so you will probably want to do it overnight, or you can get a better program that does search and replace faster.
So now that you have a brand new .map you can load it into hammer and look around. Pretty devistating isn't it? All those grey walls... But this is where MapFool steps up to the plate. Close hammer and open MapFool. Load your map into the program and you are confronted by a big text box with all the raw data. There is only one line in all this mess you should be worried about. Look just below mapversion and you will see wad and a list of wad files. Hopefully you know where all these are because now is when you need them. Click File->Options and hit the "Add" button to add your wad files. After you think you got them all, go just below that list and enter the vtex path and the texture's folder path.
Next you are going to want to use the Tools->Upgrade Obsolete Entities program to do just that, update all the func_walls and such that source does not use.
Now the fun part, go to Tools->Export Textures to tga... and export them to a folder with a bit of space and memerable. Next go to the item below that called "Make Materials..." in the textbox labeled tga files, navigate to the folder you just exported all those textures to and you should see a list of files appear in the list box below. You'll notice some textures are red and have some text saying they are not powers of two. This is because source requires all textures to have sides that are exactly 2, 4, 8, 16, 32, 64, 128, 256, 512, or 1024. A bit annoying really because if you are lucky, about half of the textures you need are that size.
If you want these textures to show up in hammer, just navigate to that folder containing all the tga's and open the ones that are not powers of two in photoshop. Then its just a simple matter of making both sides a power of two.
After thats all cleared up go ahead and click proceed in the options box and watch as MapFool converts all the textures in the map into materials that will show up in hammer.
If you open your map in hammer now, (hopefully you exported those textures into a folder that hammer will look in (eg, materials, in the source cstrike folder), the map should look a bit more complete. If everything is still grey then you might have to go back into MapFool and use Tools->Texture Folder name->Prepend to prepend the character "/" onto each texture name. What usually happens is Hammer looks for the textures in <gamedir>/materials and not <gamedir>/materials/ a small difference, but one that will come back to bite you.
When all is said and done, you should have an almost perfect replica of your favorite HL1 map. Do not try and compile it right off the bat, you might have gotten a warning message from Hammer saying it wasnt able to load a X amount of solids. That is because BSP2MAP and BSP Viewer have to reconstruct all the brushes and no program can do that big of a task and not get something wrong. The way these programs work is instead of creating real solids, it creates faces of solids. A box in a HL1 map turns into 6 1 unit thick walls after being decompiled. This causes a number of errors in hammer, the major one being the max vertice count and max brush count. As both of those are pretty much garenteed to be exceeded.
After you take a good look at this map, and you are still determinded to port it, then you will have to go through and strech all the 1 unit thick walls into their proper places as real solids. That means for a box you will have to strech one side of it and delete the other 5 walls. Needless to say, this is the reason why there are only a few Half life 1 to source ports.
To understand why this happens you need to understand the format of a map file, and how hammer and the HL1 engine reads maps. Which is beyond the scope of this article. If you are terribly insterested however, post something and there might be an article written later about it.
Levels of Security for Map makers
The method mentioned above is one of the simplest ways to protect your map from decompilation. VMEX offers three levels of security. A slightly more secure method is to make one brush in your map and texture it with "tools/locked", this is not a real texture so you will have to create it. The most secure way you can stop the hackers is by downloading VMEX's custom prefab from here, add it anywhere to your map, but do not rotate, scale, or retexture it. All three of these can still be bypassed of course, but only by those who absolutly know what they are doing. And when someone does eventually decompile your map, who better to have the vmf then a pro?
Map designers need to know that people will get ahold of your vmf in some version or another, whether it be by your hand or not. Adding protection like this only delays the inevitable. For the most part, those who get a vmf only look, but if you do eventually login to a server that is running your map + sniper tower in the mid, yelling at the person will not help anyone. Just feel confident that your original design will win over some poor guys attempt to create his own map. What does one server running his bad version of your map affect anyway.
Also, unlike HL1 maps, the source format stores practically all the information for the map in the bsp. Nothing is interpreted by the engine. Thus, all that someone would need is a $10 copy of DJ Java Decompiler, a bit of free time, and a drive to hack your map and they can take VMEX and edit out all of the protections. The main point being, if you want to create a map that no one else will be able to change, you better do it for another engine because source might as well be called open source.
Before you go
Before you take this jump to run off and decompile a map take a second or two to think, "What am I trying to learn/do?". Read the included txt files, you may just find the creators e-mail, and that could be more helpful than you imagine.
Some mappers will offer up help and advice, they may even go so far as to provide perfect examples of what you were trying to "borrow" from the map.
What's better, copying someone else's test and getting the answers then, or having someone teach you what to do so you can do it yourself next time.