VMPI

An unreleased tool which is used for distributed compiles, reducing the compile time by distributing the work among multiple networked machines. The Source Engine features page, under the tools header, implies that VMPI is included to licensees as part of the Source Engine package. It is common belief that Valve did not release VMPI due to licensing issues with MySQL. In order for VMPI to be properly licensed, its source code must be public domain or Valve must purchase a proprietary license from the MySQL allowing distribution of parts of the MySQL code in their program.

There are several parts to have a working VMPI "cluster".

The MySQL server

VMPI uses the SQL server to track statistics on the compile. It does not actually need the SQL server to do a distributed compile, but it will produce errors when a SQL server isn't present.

The master machine

When the master initiates a VMPI compile, it will only keep track of the compiled data. It will not do any of the compiling itself (CPU wise). You can make the master machine contribute to the compile if you run a VMPI worker client on the master but, you may run out of system memory from having two copies of the compile tools running at the same time.

The master initiates a compile by first searching for available non-busy workers. When it finds one or more worker machines, it will upload a specific set of files to a network directory that both the master and the worker machines can see and have read / write access to. If one or more worker machines cannot access the uploaded files, the compile will fail.

The VMPI worker machine

A VMPI worker machine is a computer that has both the vmpi_service and the vmpi_service_ui running. When the vmpi_service is started, it will start broadcasting on the LAN that it is available to start working on compiles. VMPI worker machines do not have to all be mirror images of each other (like beowulf clusters), you may have a plethora of different kinds of hardware as long as they are running some form of Windows NT (Win9x, WinME and Vista don't work very well with VMPI.)

The tools

• vmpi_service_ui.exe

This is the UI for the vmpi_service.exe, it will add an octagonal icon to the system tray. the icon has a total of 3 states; one being a brown color with a yellow question mark indicating that it cannot find a vmpi_service process. One being a dark green color indicating that vmpi_service.exe is running and waiting for jobs, and the final variation being a bright green (sometimes accompanied by the system repeatedly beeping out of its PC speaker) indicating that it is working on a compile.

• vmpi_service.exe

The process that runs on the worker machine that broadcasts itself for available jobs and listens for a master.

• vmpi_browser_job_search.exe

Reads the SQL database and displays information on current and past compiles. This tool is somewhat buggy and doesn't work correctly.

• vmpi_browser_job_watch.exe

A command line tool used to monitor a compile in progress, it is somewhat useful if you can catch the JobID from the compile window before it disappears.

• vmpi_browser_services.exe & vmpi_ping.exe

These two tools are used to find worker machines on the network. vmpi_ping.exe is the command line version while the latter is the Windows version.

• vmpi_service_install.exe

Installs the VMPI worker services on a machine.

• WaitAndRestart.exe

Seems to be some sort of reset program to end botched compiles on the network.

Options

VMPI seems to be called by applications such as Vvis and Vrad through two command line switches on their respective tools

• mpi - use VMPI to distribute computations.

• mpi_pw <password> - use a password to choose a specific set of VMPI workers.

• mpi_workers <integer> - specifies the maximum amount of VMPI workers that are allowed to work on a process (defaults to 32).