Without "visibility", everything in a map is drawn all the time regardless of whether or not the player can see it. Clearly the list of objects to be rendered must be trimmed, but how? It would take far longer to test whether each object or surface could actually be seen than it would to render them in the first place!
The interior (i.e. not occupied by a world brush) space of a BSP map is split into visleaves. Visibility between these 3D volumes is calculated when the map is compiled, and embedded into the map file for use by the engine. Like brushes, leaves are always convex.
The image to the left shows how leaves might be created in u-turn corridors (gaps between them have been added for clarity). There is no line of sight between leaf 1 and 3, and therefore when the camera is in one the contents of the other are not drawn. Calculating this is as simple for the engine as glancing at the map's embedded visibility chart.
But there is a problem with leaf 2. The contents of all three leaves are drawn when the camera is within it, frustum culling aside, even if the left-hand wall completely fills its view. There are tools to overcome this that we'll look at shortly, but keep in mind that in a lot of cases fixing the problem will be more expensive than simply drawing the less-than-optimal scene.
Bear in mind at all times that displacements, point and brush entities (including detail brushes) and brushes with $translucent faces don't affect leaves. Create nodraw world brush 'foundations' if this is a problem: it's quite common for displacements to be created as detailed 'skin' on top of a world brush skeleton, for example.
Warning: Leaves cannot be created if your map has a leak.
Reducing VIS compile time
VVIS is the tool that calculates visibility between leaves (whereas VBSP creates them). It shouldn't take more than a few minutes to finish work on even the most complex maps, and if it does the chances are you need to do some of the following. Even if it doesn't they're good practice!
- Use detail brushes appropriately.
- Keep world brushes on the grid as much as you can. Dimensions in the power of two are good.
- Use simple world brushes. Do not carve unless you're sure the results will be OK.
- Place func_viscluster (New with Source 2007) in large areas with unbroken visibility. Leaves within it will be assumed able to see each other.
- Avoid creating large open areas that the player won't see or interact with in the first place, if it's reasonable to do so. Use a 3D Skybox to reduce sky size, and create a world brush skeleton beneath displacements.
Remember that VIS compile time and in-game performance are two very different things. It may well be worth bearing a long compile if shortening it compromises your result.
The Orange Box version of Hammer has a new function to view the current map's leaves in the 3D view: Map > Load Portal File. This displays leaf edges that touch other edges as thick blue lines. It's a fantastic learning tool, as if you compile your map without VIS or RAD (then reload its portal file to refresh the display) you can see your changes' effects very quickly.
For in-game inspections, there is a console variable called "mat_leafvis".
mat_leafvis 3 will outline all the visleaves in the PVS, while
mat_leafvis 1 will just outline the leaf that the camera is currently in. (This variable is a cheat.)
Unfortunately, leaves don't always work out so well as they did in our first example. Consider the alternative layout on the left (in practice things would be configured as per the first image, but put that aside for a moment): in this example the leaves can all see each other, leading to the whole area being drawn at once for no good reason. This is where Hints come in.
A hint is a brush face textured with the special
tools\toolshint material, which slices any leaves it intersects in two (faces of the brush that should not split leaves must be textured with
tools\toolsskip). In our example, we would place a hint where the purple line indicates, so that it slices leaves 1 and 3 to the same shape that they were in our first example.
It isn't possible to merge leaves together, so after hinting there will be three separate leaves on the right-hand side. Thankfully this isn't a huge problem.
As was mentioned above, leaves are moulded around world brushes. But what if you don't want a brush to do that? If you have a brush that can be seen around at all times (e.g. a statue plinth, a small free-standing wall) there’s no point in creating lots of extra leaves around it.
In these situations you want func_detail. It's an internal entity which makes the compilers ignore the brush when calculating visibility, without otherwise affecting its behaviour. It's not uncommon for large portions of a map's brushes to be detail.
If a player cannot see a brush face without cheating or being a spectator, it's a good idea to apply the special nodraw material to it. Nodraw removes the entire face when the map is compiled without affecting visibility, which reduces both rendering time and, since no lightmap needs to be calculated, map filesize.
You do not need to apply nodraw to faces that are touching the void (i.e. outside the map) or to brush faces touching other brush faces tied to the same entity (the world being one big entity of its own). See Brush#In-game.
In the image to the left all of the leaves can see each other, and this time there is very little that hint brushes can achieve.
In this situation we should create Areaportals at the mouth of selected openings. These make the engine perform the per-object line of sight testing that Carmack created the BSP/visleaf system to avoid – but with that system behind it, and with the calculations restricted to when the camera looks through the areaportal, performance can be gained if one blocks enough objects from view.
The image has areaportal locations marked over each mouth, but placing all four would likely hinder performance more than help it. Your strategy depends on the relative cost of each area - in most situations, creating the large areaportal to the right and leaving the corridors alone would be best.
(It is possible to use hints to reduce visibility in this situation, but only with a ridiculous number of them at very precise angles, which impacts performance in its own way. If you can't see across a room for hint brushes then you should be using an areaportal!)
Warning: Areaportals can cause problems if placed incorrectly, so be sure to read about them carefully.
Sometimes you want to block visibility in a way that visleaves simply don't allow: consider a destructible free-standing wall behind which are standing several expensive character models. You don't want to draw the models if they aren't being seen, but without any connected world brushes to work with except the ground you can't do it by separating leaves.
An Occluder is the tool for the job. It's a brush that hides models (not brushes, alas) behind it when enabled in a similar manner to an areaportal hiding things that aren't behind it. For obvious reasons, it should be entirely contained inside an opaque object.
If you are dealing with a large open area without any cover at all then there isn't a lot you can do but remove detail or employ fog to disguise a low draw distance. Configure draw distance with Map > Properties > Far z_clip plane.
Draw distances can also be applied selectively to props, even prop_static, with the Start Fade Dist/Pixels and End Fade Dist/Pixels keyvalues. As the names suggest, these values can represent either distance in units or pixels on-screen as per the Screen Space Fade KV.
If you enter fade distances on props, it’s a good general rule to have the difference between the two numbers (start and end) be 200. It looks pretty good, and the larger that number is, the longer the model incurs an increased cost to render. Fading models do not go through the fast path and incur additional sorting cost.
func_lod is a special brush entity that can fade out. You can't use any brushes tied to it as any other entity, however!
- Func_lod (World detail that doesn't need to be drawn far away, for performance reasons.)