Moderator elections are being held. See Valve Developer Community:Moderator elections for more details.
Users who would like to run for moderator must be autoconfirmed and have at least 100 edits. Users can check their own edit count at Special:Preferences.
Quake3-16px.png
This article's documentation is for anything that uses the Source engine. Click here for more information.

User:SirYodaJedi/Porting IdTech 3 maps to Source

From Valve Developer Community
Jump to: navigation, search

Icon-essay.png
This is an essay.
It contains the advice or opinions of one or more of the Valve Developer Community's contributors.
This page is not an encyclopedia article, nor is it one of the Valve Developer Community's policies or guidelines, as it has not been thoroughly vetted by the community.
Some essays represent widespread norms; others only represent minority viewpoints.
Under construction icon-blue.png
This is a draft user page. It is a work in progress open to editing by anyone.
Remember to check for any notes left by the tagger at this article's talk page.
Info icon
This page documents information about an engine not developed by Valve. This information is documented here because it has technical and/or historical relevance to Valve's engines.
English (en)
Edit

This page attempts to document the fiddly process of converting a map from a id Tech 3 id Tech 3 game to a Source Source game. This guide will demonstrate it by porting the Wikipedia icon Wolfenstein: Enemy Territory map "Würzburg Radar" to Day of Defeat: Source Day of Defeat: Source, as the two games core game modes are similar, the original map sources are officially publicly available, and most of the necessary auxiliary assets are provided with GTKRadiant (with the missing ones found in this repo).

Getting started

Skills required

This article assumes some familiarity with the following skills:

  • Mapping
  • Basic 3D model editing
  • Image manipulation and conversion
  • Audio conversion and looping a sound
  • The command-line interface

Installing tools

Required tools:

  • The original game (duh)
  • J.A.C.K. J.A.C.K. - Can convert the Id Tech 3 map files to GoldSrc GoldSrc format, as well as export as VMF
  • GtkRadiant GtkRadiant - has a lot of models and material information that aren't present in the games' files, due to being baked into the BSP. It also includes Q3MAP2, which can decompile maps you don't have the source MAP files for.
    • NetRadiant-Custom NetRadiant-Custom - it's fork of Q3MAP2 may be preferred for decompiling maps, as it supports Valve220, allowing for more accurate texture mapping. Additionally, it can convert IdTech 3 maps to OBJ, instead of only MAP or the ill-supported ASE.
  • Blender Blender - Can do necessary fine-tuning to the converted models, as well as do the conversion itself with the proper plugins. The following plugins are used:
Icon-Important.pngImportant:For plugin compatibility reasons, this guide uses Blender 3.1.1. Any version between 2.92 and 3.1.1 should work, but newer versions may have issues.
  • GIMP GIMP - Assorted texture manipulation. Adobe Photoshop Photoshop can also be used, but this guide focuses on the free option.

Additional useful tools

  • TrenchBroom TrenchBroom - Can export the unconvertible patch meshes to OBJ, which can be used as a prop_static.
  • Noesis - Can open most relevant models, except MDC. SMD export has flipped UVs and lacks animation; use FBX instead.
    • Half-Life Quake3 Map Converter - can allegedly be used instead of Jack for map format conversion, but is older and untested, and creates a MAP instead of a VMF (not to mention, is Windows-only).
  • Wikipedia icon 7-Zip (or WinRAR) - Preferable for extracting PK3s. Much faster than using Windows's built-in ZIP extractor, and doesn't require renaming the file extension.

Preparing workspace

To start, it will be necessary to decompress the PK3 archives containing the game assets.

  • Create a new folder which will house the assets. This should be separate from the game installation, for example 🖿D:\Applications\IdTech 3 Modding\Wolfenstein Enemy Territory\.
  • Inside the newly-created folder, create a folder with the same name as the game's PK3s are contained in. For example 🖿D:\Applications\IdTech 3 Modding\Wolfenstein Enemy Territory\etmain\. This will be henceforth referred to as the source asset directory.
  • If the game has any subfolders inside the folder containing the PK3s, copy those folders to the source asset directory.
  • Extract all the PK3s from the original game to the source asset directory.
  • In the GTKRadiant folder, navigate to the 🖿installs subdirectory. This contains the "game packs" which have the necessary auxiliary assets.
Note.pngNote:GTKRadiant 1.6 is missing the Jedi Outcast game pack. This will need to be downloaded separately; it can be found in GTKRadiant 1.4.
Note.pngNote:The Wolfenstein: Enemy Territory game pack is missing several map prop models; this can be found separately.
  • Blank image.pngTodo:

Decompiling maps

It is preferable to work from source map files, when available. Compiled id Tech 3 maps don't (usually) have lighting entities, and lightmapped models are embedded into the map's mesh data. When a map is decompiled, the embedded models are removed, no lighting data is available, .

Note.pngNote:While the lighting entities can't be used in Source 1, and should be recreated anyway, their placement can be handy when determining where to place the Source 1 lighting entities.

Nonetheless, map sources are frequently not available, leaving no other option. Fortunately, Q3MAP2, the same compiler used to compile id Tech 3 BSPs, can also decompile them! The following command will decompile the map into a format that can be properly edited:

q3map2 -game xxxx -convert -format map mapname.bsp
Note.pngNote:xxxx is the game abbreviation, as defined in games.cpp.

As this doesn't include embedded models, it often is also necessary to convert the BSP to a 3D model. NetRadiant-Custom's Q3Map2 supports converting the BSP to OBJ, as such:

q3map2 -game xxxx -convert -format obj mapname.bsp
Note.pngNote:For other versions of Q3Map2, replace obj with ase, and use Noesis to convert the resulting file to OBJ or FBX (Noesis's SMD exporter results in flipped UVs).
Tip.pngTip:This can also be used to bake the vertex coloring and texture blending into megatextures for terrain models, as id Tech 3 id Tech 3 uses a much different method for blended textures compared to the displacements used by Source Source.
Keep static prop lighting limitations in mind if doing this; consider prebaking lightmaps into the $basetexture or a Modulate overlay if not using Source 2013 Multiplayer.
Note.pngNote:Neither setting includes patch meshes; a Quake III BSP importer for Blender should be used to recreate these.
Blank image.pngTodo: Provide such an importer.
Note.pngNote:Q3Map2 does not support BSP files from Call of Duty, which uses a slightly modified version of RtCW's BSP format. CoD BSP Decompiler can decompile CoD 1+2 BSPs. Unlike most IdTech 3 BSP formats, CoD BSPs don't embed static prop geometry in the compiled map.

Converting uncompiled map files

Blank image.pngTodo: Cover stuff:
  • Using Jack to create a VMF with the brushes and entities, and a MAP file with just the patch meshes
  • Using Q3Map3 from NRC to convert the patches-only MAP directly to OBJ (which does work, unlike BSP to ASE/OBJ)

Converting materials

Tip.pngTip:Having the Quake III Shader Manual (and the newer Q3Map2 version) at one's fingertips is essential to ensuring the converted VMTs are as accurate to the original Quake III "shaders" as possible.

Converting textures

Choosing the right image format

Blank image.pngTodo: Discuss a bunch of stuff
  • IdTech 3 supports texture compression, but not all versions used pre-compressed textures, instead using JPG, TGA, and sometimes PNG.
    • Not all GPUs supported texture compression back in 1999, so it was done on-the-fly and is able to be disabled. Most people disable it nowadays, as VRAM is much more plentiful, and the benefits of disabling are apparent on low-res textures.
  • In RtCW, WolfET, and possibly other games, textures with resolution of 128x128 or lower are never compressed.
  • Some games used DDS. If it is necessary to use a DDS texture as a source, use the same image format as the original texture if possible. For example, VRAD has a bug where it can't read the alpha channel from DXT3 textures for generating texture shadows, so it will be necessary to use DXT5 instead.

Selecting material parameters

See User:SirYodaJedi/Porting_IdTech_3_maps_to_Source/Shader2VMT

Replacing tool textures

IdTech 3 Source Notes
common/caulk tools/toolsnodraw
common/nodraw tools/toolsinvisible
common/hint tools/toolshint
common/skip tools/toolsskip
common/trigger tools/toolstrigger
common/areaportal tools/toolsareaportal
common/origin tools/toolsorigin Deprecated; preferably delete and set the origin KV.

Converting models

MD3, MDC, MDS, MDM/MDX

  • In a version of Blender from 2.92 to 3.1.1, install RtCW:ET Model tools v1.0.0_rc3 and Blender Source Tools v3.2.5.
  • Open a new workspace in the selected version of Blender. Delete the default cube.
  • Press N to open the sidebar, then switch to the "RtCW/ET" tab.
  • In the import section, select the source format and file path, then press the import button.
  • For each mesh, select it, right click, and choose shade smooth.
  • Export the collection as SMD.
  • Open a new workspace, then reimport the SMD.
  • Unflag sharp edges as necessary.

Other

Blank image.pngTodo: Using FBX results in models being really tiny when imported into blender; can this be fixed?
  • Use Noesis to convert the models to FBX.
    • Noesis can export directly to SMD, but the UV map will be flipped. (I reported this bug to the creator, but he refused to believe the current behavior was incorrect)
Confirm.pngConfirm: Do animated props need to be converted via a different method?
  • Import to and export from Blender. Editing can be done, if necessary.
    • For static props in Source 2013 Multiplayer Source 2013 Multiplayer, it may be desireable to redo the model's UVs so that everything is on one texture, and no UVs are overlapping or tiled. This allows model lightmapping to be properly used, as model lightmaps use the same UVs as the $basetexture.
  • Write QC files.
  • Compile.
    • Crowbar Crowbar can be used as a handy frontend for compiling.

Converting sounds

For the most part, sounds from id Tech 3 games can be used in Source without much hassle as long as they are converted to a compatible format (if they aren't already). The exception, of course, is looped sounds. Unlike id Tech 2 and Valve's engines, id Tech 3 does not require WAV cue points for looped sounds. This means that cue points must be added, and non-WAV sounds that are looped must be converted to WAV, which greatly increases file size.

Converting map content

Entities

A few entities in IdTech 3 have analogous entities in Source 1.

id Tech 3 id Tech 3 entity Source Source entity Notes
func_group N/A Func_group brushes should be untied from entities and returned to worldspawn, as VBSP lacks func_group support. If the brush had the detail flag, tie to func_detail instead.
misc_model prop_static Unlike prop_static in Source 1, misc_model in IdTech 3 can be lightmapped in the same way as brushes. This is done sparingly, via spawnflag 2 "force meta" or material command q3map_forceMeta, as it basically converts the model into trisoup brushes. Most small or medium-sized misc_models are lit about the same as prop_static (with Q3Map2 doing bounced lighting on them like CSGO).
PlacementTip.pngWorkaround: There are ways of improving lighting, depending upon the engine branch. These are more expensive, however, and should only be done if the default vertex lighting is unsatisfactory.
  • (in any branch) - Subdivide each convex face of the mesh, being careful not to change the overall topology (don't add verts to "curved" edges). Lighting is done per-vertex, so increasing the number of vertices results in more accurate lighting, and a dense enough mesh can be more accurate when vertex lit than when lightmapped.
If $lod is available for the engine branch, consider having two versions of the subdivided mesh: one with heavy subdivision for the default close-up mesh, and one with much less subdivision for further away. For example, a simple rectangular prism (ex: fence post) could be subdivided on the long sides with 12x3 faces for the close up mesh, and 7x2 faces for the $lod 50 mesh (potentially using loop cuts, so that the short sides are subdivided evenly).
  • (in Source 2013 Multiplayer and Garry's Mod) - Make all of the model's UVs fit onto one material with no overlap or tiling, then use a static prop lightmap.
  • (in any branch) - Do the same UV edits as above, but pre-bake the lighting in Blender (possibly onto a detail texture with $detailblendmode 8 or Modulate mesh). Use UnlitGeneric instead of VertexLitGeneric to prevent VRAD lighting from conflicting with the Blender lighting. Note that this will cause the model to be unaffected by most dynamic lights except for flashlights and CSM.
  • (in any branch) - Use OBJ2MAP or similar to convert the model to brushwork (only if the model is simple enough!). This can work around the lack of bounced static prop lighting on pre-Counter-Strike: Global Offensive versions of VRAD.
misc_model_static prop_static In Jedi Academy. Like misc_model, but the model is rendered as a runtime entity, instead of baked into the map geometry. If you want to be accurate to how this affects lighting, you could disable per-vertex lighting.
misc_model_ghoul prop_dynamic In Jedi Outcast and Jedi Academy. Arbitrary animated GLM model; destroyable.
misc_model_* prop_dynamic In Raven Software games. Specific MD3 model to display at runtime, sometimes animated; see DEF or ENT entity description file for model path and usage specifics.
misc_gamemodel prop_dynamic In RtCW and WolfET. Source 2013 Source 2013 doesn't completely support arbitrary vertex animation, so animated MD3s or MDCs will need to be re-animated with a skeleton. If unnamed and not animated, consider using a prop_static with per-vertex lighting disabled, to save on edicts without affecting lighting.
script_model_med generic_actor
monster_generic
In RtCW and WolfET. A model which has its animations and movement defined in an external script file.
dlight light_dynamic Flickering light. Only affects dynamically lit models; set the "Do not light world" flag to be accurate.
lightJunior N/A Only affects dynamically lit models, and does not use an edict. Source 1 and its BSP format technically supports this, but the compilers lack a way to set it, so you'd have to manually edit/replace the WORLDLIGHTS and WORLDLIGHTS_HDR lumps.
info_camp info_null Basically the same thing.
info_notnull info_target Works the same way; Valve decided to rename the entity in GoldSrc.
target_position info_target Same as info_notnull.
target_relay
target_delay
logic_relay Only difference between relay and delay is whether the target is triggered immediately or after a delay. The delay can have a random variance to it, in which case some complicated logic_timer shenanigans may be necessary, probably in conjunction with a logic_case.
func_timer logic_timer Despite the func_ prefix, this is a point entity.
trigger_always logic_auto IdTech version is poorly named. It is only fired on map spawn, but does not fire if a map is entered from a level change without starting a new unit. As such, it might be necessary to Blank image.pngTodo:

target_speaker ambient_generic
info_player_intermission Counter-Strike: SourceDay of Defeat: Sourcepoint_viewcontrol Controls camera in between rounds.
Blank image.pngTodo: In at least WolfET, the is also a flag to set whether this is used after which team wins; is there an equivalent way to handle this in CS:S/DoD:S?
misc_portal_surface func_monitor (if has target)
func_reflective_glass (if no target)
Portal 2Strata Sourcelinked_portal_door (if navigable)
Point entity which affects the nearest portal surface, which will need to be turned in to the appropriate brush entity.
misc_portal_camera point_camera Target for the misc_portal_surface.
team_CTF_redplayer
team_CTF_redspawn
Half-Life 2: Deathmatchinfo_player_combine
Day of Defeat: Sourceinfo_player_axis
Counter-Strike: Sourceinfo_player_terrorist
Black Mesainfo_player_scientist
Team Fortress 2info_player_teamspawn (RED)
In Quake III (and maybe RtCW?), the team_CTF_*player are initial spawn points, whereas team_CTF_*spawn are respawn points. WolfET always uses the latter; toggling them based upon conditionals.
In RtCW and WolfET, red is Axis (German), and blue is Allied (American).
team_CTF_blueplayer
team_CTF_bluespawn
Half-Life 2: Deathmatchinfo_player_combine
Day of Defeat: Sourceinfo_player_allies
Counter-Strike: Sourceinfo_player_counterterrorist
Black Mesainfo_player_marine
Team Fortress 2info_player_teamspawn (BLU)
team_WOLF_checkpoint Day of Defeat: Sourcedod_control_point In WolfET. A checkpoint, flag, command post, or control point; whatever you want to call it.
  • Displays models/multiplayer/flagpole/flagpole.md3 if no model is defined, and plays sound/movers/doors/door6_open.wav during capture animation.
  • Any spawnpoints that are targeted (target KV) by this entity are only active while the point is in the respective team's control.
  • Unlike the DoD:S equivalent, this model is solid, so cover the pole with a small rectangular func_detail using a nodraw material with a metal $surfaceprop.

Notable KVs

Some KVs that are common across several entities

  • target (target_destination) - send an input to the specified entity (whatever that entity's "do" input is)
  • model2 (studio) - The first frame of a dynamically-lit model is displayed at runtime wherever this brush entity's origin is (which will move if the entity moves). Put a prop_dynamic where the origin is, and parent it to this entity.
  • _lightmapscale (float) - Adjusts luxel scale relative to the material's q3map_lightmapSampleSize, or if not present, the -samplesize Q3Map2 parameter). If neither are present, assume a default luxel scale of 16. Do note that brush/disp lightmaps in Source are much lower resolution than lightmaps in Quake 3 (32px vs 128px+), so to avoid excessive subdivision on really low luxel scales, consider using static prop lightmaps if possible.
  • _remap* (string "original_material;new_material") - The first material is replaced with the second material in the compiled map (usually used for misc_model). Use skins.
  • targetShaderName (material) + targetShaderNewName (material) - When triggered, the specified material in the target entity with a different material. Note that targetShaderName will always be the original material name, even if the material being displayed has been changed at runtime. This will require some complicated material proxy stuff env_texturetoggle or material_modify_control on brushes, but on prop_dynamic, you can just send the skin input.

Terrain

Patch meshes

Blank image.pngTodo: Three main methods:
  • Use models
  • Recreate as displacements (if possible; not possible if tied to entity)
  • Convert to brushwork, using OBJ2MAP or similar (if practical)
Note.pngNote:In older compilers, patch meshes were always vertex lit, not lightmapped (Q3Map2 and Call of Duty changed that). Nonetheless, it may still be desirable to use lightmaps for better lighting on the Source Engine ports of the geometry, as Source doesn't support the tessellation that IdTech 3 uses to smooth out curves, which will result in less smooth lighting. Patch meshes generally have UVs that are very suitable for lightmapped props.

External links