Head Tracking

From Valve Developer Community
Jump to: navigation, search

This tutorial shows how to interface an external 6 DOF tracking or 6 DOF input device to your Valve Source SDK Mod so you can control the player (head) with the device instead of the mouse and keyboard. Example 6 DOF tracking devices:

Note, for many 3 DOF (orientation only) input devices (e.g., trackball) you can simply use your device's driver to masquerade as a mouse. But with 6 DOF devices (position and orientation) you have to something more. This tutorial supports external tracking devices that control the location and/or orientation of the player's head.

Step 1: Create an Interface to work with Any Tracker

The first step is to create a helper interface that will be used to access your tracker. This class will have interface methods that Valve will call when it needs position and orientation information. This is a software engineering drill, but will be very useful when you want to use more than one type of tracker.


Step 1A: Create the file CPPInterfaces2.h

Put this code inside: 

//
// CppInterfaces2.h
//

#define Interface class

#define implements public

#define DeclareInterface(name) __interface actual_##name {

#define DeclareBasedInterface(name, base) __interface actual_##name \
     : public actual_##base {

#define EndInterface(name) };                \
     Interface name : public actual_##name { \
     public:                                 \
        virtual ~name() {}                   \
     };

This is just a bunch of compiler macros that will enforce interface constraints on your code. C++ doesn't provide native interface objects, so this is a way to enforce it so you can have the idea of an interface object. If you want the full details, check out this article by Jose Rios

Step 1B: Create the file cl_dll\IMovementController.h.

Put this code inside: 


//Include interface enforement directives
#include "CPPInterfaces2.h"

/****************************************************
*  This interface is used to integrate all 
*  Valve movement controllers
*
*
*  Each movement controller provides 6-DOF on
*  a particular object (body part, other tracked object
*
*  This interface assumes that any implementing class
*  will perform any required post processing to transform
*  tracking results into a right handed coordinate system
*  with +Z up -- with units in inches.
*
*****************************************************/
#ifndef IMOVEMENT_CONTROLLER_H
#define IMOVEMENT_CONTROLLER_H
DeclareInterface(IMovementController)
	/**
	*  Returns the orientation from the tracker.  Assumes angles are relative to a right handed coord system with +Z up.
	*  Assumes update() has been called.
	*/
	int		getOrientation(float &pitch, float &yaw, float &roll);

	/**
	*  Returns the position from the tracker.  Assumes coordinates are relative to a right handed coord system with +Z up.
	*  Assumes update() has been called.
	*/
	int		getPosition(float &x, float &y, float &z);
	
        /**
        * Returns true if the tracker is initialized and ready to track
        */
	bool	        isTrackerInitialized();

	/**
	*  Reads the hardware and updates local internal state variables for later read by accessors.
	*/
	void	update();

        /**
        *  Returns true if the tracker has good position info
        */
	bool	hasPositionTracking();

        /**
        *  Returns true if the tracker has good/reliable orientation info
        */
	bool	hasOrientationTracking();
EndInterface(IMovementController)
#endif //IMOVEMENT_CONTROLLER_H

Again, this is just an interface and only ensures that when you implement a tracker, you obey a set of rules that will allow you to swap out trackers at compile time very easily.

Step 2: Create an Instance of the Interface for Your Particular Tracker

The class will implement the interface in Step 1, but have tracker specific API calls and code (the "guts") that deal with the nuances of your particular tracker SDK (e.g., FACE API, Intersense API, etc).



Header File

Create a file cl_dll\MyMovementController.h. Place this inside: 

#include "IMovementController.h" // The movement control interface
#include /path/to/your/tracker/api


class MyMovementController : implements IMovementController {

/************************** Member Functions **************************/
public:
	
	/**
	*  Construct a new ar_movement_controller
	*/
	MyMovementController();

	/**
	*  Destructor  Closes tracker and performs clean up
	*/
	~MyMovementController();

		
  /**	
  * Returns the pitch, yaw, and roll Euler angles of the tracker
  */
	int		getOrientation(float &pitch, float &yaw, float &roll);
	
	/**
	* Returns the position (in inches) of the tracker
	*/
	int		getPosition(float &x, float &y, float &z);
	
	/** 
	* Returns true if the tracker has reliable position information
	*/
	bool	hasPositionTracking(void);
	
	/**
	* Returns true if the tracker has reliable orientation information
	*/
	bool	hasOrientationTracking(void);
	
	/**
	* Returns true if the tracker is alive/ready
	*/
	bool	isTrackerInitialized(void);
	
	/**
	* Tells the tracker that its time/safe to update.
	*/
	void	update(void);

	
};

Class File

Create a file cl_dll\MyMovementController.cpp. Place this inside: 

/****************************************************************************
*  Sample shell for specific tracker instance
*
*  This is very dependent on what tracker and API you are using
* 
*  This example is a conceptual template of how it would look
*
******************************************************************************/
#include "cbase.h"
#include "MyMovementController.h"

/****************************************************************************
*  Constructor 
******************************************************************************/
MyMovementController::MyMovementController() {

	//Put in code here that is needed to connect to your tracker and initialize it
	my_tracking_api.open();

}

/****************************************************************************
*  Destructor
******************************************************************************/
MyMovementController::~MyMovementController(){

	//Put in code here that shuts down the tracker
	my_tracking_api.close();
}


/****************************************************************************
*
*   functionName:   GetOrientation
*   Description:    Get the current orientation from the tracker
*
******************************************************************************/
int MyMovementController::getOrientation(float &pitch, float &yaw, float &roll){

	//Here just set pitch, roll, yaw using your tracker API/SDK
	pitch = my_tracking_api.getPitch();
	yaw = my_tracking_api.getYaw();
	roll = my_tracking_api.getRoll();

	//Note:  If you need to do any rotations, say to adding 90 degrees to turn the tracker to face
	// 			 the player, then this is a good place

	return 0;
}


/****************************************************************************
*
*   functionName:   GetPosition
*   Description:    Get the current position from the tracker
*
******************************************************************************/
int MyMovementController::getPosition(float &x, float &y, float &z){

	//Here just set x, y, and z using your tracker API/SDK
	x = my_tracking_api.getX_Pos();
	y = my_tracking_api.getY_Pos();
	z = my_tracking_api.getZ_Pos();

	//Note:  If you need to add offsets or convert units (meter's the inches, etc) then you can
	//				do that here

	return 0;
}

/****************************************************************************
*  Cues the tracker for an update
*
******************************************************************************/
void MyMovementController::update() {
	
		//Here use your tracker API/SDK to perform any per-frame steps that must
		//be called to update the tracker
		my_tracking_api.update();
}

/****************************************************************************
*
*  Checkes if the tracker is alive
*
******************************************************************************/
bool MyMovementController::isTrackerInitialized() {
	
		//Here use your tracker API/SDK to check the status of the tracker
		return my_tracking_api.trackerReady();
}

/****************************************************************************
*
*  Checkes if the tracker has valid orientation data
*
******************************************************************************/
bool MyMovementController::hasOrientationTracking() {
	
		//Here use your tracker API/SDK to check the orientation of the tracker
		return my_tracking_api.hasOrientationData();
}


/****************************************************************************
*
*  Checkes if the tracker has valid orientation data
*
******************************************************************************/
bool MyMovementController::hasPositionTracking() {
	
		//Here use your tracker API/SDK to check the position tracking of the tracker
		return my_tracking_api.hasPositionData();
}

Step 3 : Modify Valve's in_main.cpp File to work with the Tracker Class

Modify cl_dll\in_main.cpp as follows.


Includes

In the includes section, include your tracker implementation from Step 2 and the interface from Step 1: 


#include "IMovementController.h"
#include "MyMovementController.h"


Variables

In the top of the file, add a reference to your tracker and also a global tracking enabled flag: 

//The movement controller used for head tracking
IMovementController* arctl_head;
bool useTracking = true;

Note 1: This is creating an interface, which will be use by all the code we add. This will allow you to easily swap out trackers without having to tweak a bunch of code.

Note 2: The useTracking flag is a way to toggle tracking. Use a ConVar for runtime control

Creation

Find the method void CInput::Init_All (void)

Add code (anywhere in the method is OK) to instantiate your tracking class: 

arctl_head = new MyMovementController();

Note: This is the only place we specify the particular implementation of the tracker.


Destruction and Cleanup

Find the method void CInput::Shutdown_All(void). Add the following (anywhere): 

delete arctl_head;

This forces a call to the destructor in our tracking class.

Allowing Use of the Mouse to Control Orientation when Tracker not Ready

Find the method void IN_CenterView_f (void). Change it to this: 

void IN_CenterView_f (void)
{
        //Use mouse if tracker is unavailable
	if(!arctl_head->isTrackerInitialized() || (useTracking == false)){ 
		QAngle viewangles;
		
		//arctl_head->report();

		if ( UsingMouselook() == false )
		{
			if ( !::input->CAM_InterceptingMouse() )
			{
				engine->GetViewAngles( viewangles );
				viewangles[PITCH] = 0;
				engine->SetViewAngles( viewangles );
				//prediction->SetLocalViewAngles( viewangles );  //STEVE ALBANY

			}
		}
	}
}

Find the method void CInput::ExtraMouseSample( float frametime, bool active ). Change it to this: 

void CInput::ExtraMouseSample( float frametime, bool active )
{
        //Use mouse if tracker is unavailable or disabled
	if(!arctl_head->isTrackerInitialized() || (useTracking == false)){ 
		CUserCmd dummy;
		CUserCmd *cmd = &dummy;

		cmd->Reset();

		QAngle viewangles;

		if ( active )
		{
			// Determine view angles
			AdjustAngles ( frametime );

			// Determine sideways movement
			ComputeSideMove( cmd );

			// Determine vertical movement
			ComputeUpwardMove( cmd );

			// Determine forward movement
			ComputeForwardMove( cmd );

			// Scale based on holding speed key or having too fast of a velocity based on client maximum
			//  speed.
			ScaleMovements( cmd );

			// Allow mice and other controllers to add their inputs
			ControllerMove( frametime, cmd );
		}

		// Retreive view angles from engine ( could have been set in IN_AdjustAngles above )
		engine->GetViewAngles( viewangles );

		// Set button and flag bits, don't blow away state
		cmd->buttons = GetButtonBits( 0 );

		// Use new view angles if alive, otherwise user last angles we stored off.
		if ( g_iAlive )
		{
			VectorCopy( viewangles, cmd->viewangles );
			VectorCopy( viewangles, m_angPreviousViewAngles );
		}
		else
		{
			VectorCopy( m_angPreviousViewAngles, cmd->viewangles );
		}

		// Let the move manager override anything it wants to.
		g_pClientMode->CreateMove( frametime, cmd );

		// Get current view angles after the client mode tweaks with it
		engine->SetViewAngles( cmd->viewangles );

		prediction->SetLocalViewAngles( cmd->viewangles );
	}
}

Find the method void CInput::ControllerMove( float frametime, CUserCmd *cmd ). Change it to this: 

void CInput::ControllerMove( float frametime, CUserCmd *cmd )
{
        //Use mouse if tracker is unavailable or tracking disabled
	if(!arctl_head->isTrackerInitialized() || (useTracking == false)){ 
		if ( IsPC() )
		{
			if ( !m_fCameraInterceptingMouse && m_fMouseActive )
			{
				MouseMove( cmd);
			}
		}
		JoyStickMove( frametime, cmd);
	}
}


These allow you to use the mouse to control orientation when the tracker isn't active.

Adjusting the Player/Camera Orientation

Find the method void CInput::AdjustAngles ( float frametime ). Change it to this: 

void CInput::AdjustAngles ( float frametime )
{
	float	speed;
	QAngle viewangles;
	
	// Determine control scaling factor ( multiplies time )
	speed = DetermineKeySpeed( frametime );

	// Retrieve latest view direction from engine
	engine->GetViewAngles( viewangles );

        //Get view angle from the tracker
	if(arctl_head->isTrackerInitialized() && (useTracking == true)){
		
		arctl_head->update();			

		viewangles = getCameraAngles();
		//Msg("myang = %f, %f, %f\n", myAng.x, myAng.y, myAng.z);

	}
	else{

		// Adjust YAW
		AdjustYaw( speed, viewangles );

		// Adjust PITCH if keyboard looking
		AdjustPitch( speed, viewangles );

		// Make sure values are legitimate
		ClampAngles( viewangles );
	}


	// Store new view angles into engine view direction
	engine->SetViewAngles( viewangles );

}

You'll notice the code above calls a method getCameraAngles(). Add it to the code (anywhere above AdjustAngles method): 

QAngle getCameraAngles() {

	QAngle viewangles;

	//pitch, yaw, roll
	float pitch, yaw, roll;

	arctl_head->getOrientation(pitch, yaw, roll);

	//Note: the mapping of angles to your tracker might be different
	viewangles.x = pitch;
	viewangles.y = yaw;
	viewangles.z = roll;
		
	return viewangles;
}

Adjusting the Player/Camera Position

Find the method void CInput::CreateMove ( int sequence_number, float input_sample_frametime, bool active ).

Look for this code: 

if ( active || sv_noclipduringpause.GetInt() )
{
		// Determine view angles
		AdjustAngles ( input_sample_frametime );

                // Allow mice and other controllers to add their inputs
		ControllerMove( input_sample_frametime, cmd );
}

Add code to read the tracker: 

if ( active || sv_noclipduringpause.GetInt() )
{
	// Determine view angles
	AdjustAngles( input_sample_frametime );

	//Use tracker for position if position tracking is available and enabled
	if(arctl_head->hasPositionTracking() && (useTracking == true)){
		
		float x,y,z;
		getCameraPosition(x,y,z);
		char command[64];
		//Msg("set_user_position %f %f %f\n", x, y, z);

                //Note -- passing Console commands this is not optimal and considered poor programming
                //        Consider using a UserCMD
		sprintf(command, "set_user_position %f %f %f", x, y, z);
		engine->ClientCmd(command);			
	}
				
	if(!arctl_head->hasPositionTracking() || (useTracking == false)){
		
		// Determine sideways movement
		ComputeSideMove( cmd );

		// Determine vertical movement
		ComputeUpwardMove( cmd );

		// Determine forward movement
		ComputeForwardMove( cmd );

		// Scale based on holding speed key or having too fast of a velocity based on client maximum
		//  speed.
		ScaleMovements( cmd );
	}

	// Allow mice and other controllers to add their inputs
	ControllerMove( input_sample_frametime, cmd );
}


You will notice a call to getCameraPosition(x,y,z). We will actually need to three methods for this call. Add the following three methods anywhere in the file above the CreateMove() method: 

/**
* Given an eye position and orientation, return the feet position required to set the eyes there..
* 
* NOTE:  We are required to leave the body of the player in an upright player.  So we always
*		 Place the feet position straight down (z-64) from the tracker.  This will ensure
*		 the eyes are where they should be.
*/
matrix3x4_t getFeetTransfrom(float tx, float ty, float tz, float tp, float tyaw, float tr) {

	//Get a copy of the player to use as a temporary tracker
	C_BasePlayer* tracker = C_BasePlayer::GetLocalPlayer();

	//Record the Abs Oring and Angles

	Vector oOrigin = tracker->GetAbsOrigin();
	QAngle oAngles = tracker->GetAbsAngles();

	//Set the virtual tracker to the supplied eye postion
	tracker->SetAbsOrigin(Vector(tx,ty,tz-64));
	
	
	//Rotate the tracker to match the supplied angles
	//tracker->SetAbsAngles(QAngle(tp, tyaw, tr));	
	tracker->SetAbsAngles(QAngle(tp, tyaw, 0));	

	//Create to the player's head
	matrix3x4_t xform;
	xform[0][0] =1;
	xform[0][1] =0;
	xform[0][2] =0;
	xform[0][3] =0;

	xform[1][0] =0;
	xform[1][1] =1;
	xform[1][2] =0;
	xform[1][3] =0;

	xform[2][0] =0;
	xform[2][1] =0;
	xform[2][2] =1;
	xform[2][3] =0;

	
	//Execute the matrix operation (translation)
	matrix3x4_t translationTemp;		
	ConcatTransforms( tracker->EntityToWorldTransform(), xform, translationTemp);

	//Set everything back
	tracker->SetAbsOrigin(oOrigin);
	tracker->SetAbsAngles(oAngles);

	return translationTemp;
}

/**
* Returns the current camera transformation
*/
matrix3x4_t getCameraTransformation() {

	float x, y,z;
	//First get the angles of the tracker
	float pitch, yaw, roll;				
	
	arctl_head->getOrientation(pitch, yaw, roll);
	
	QAngle viewangles;

	viewangles.x = pitch;
	viewangles.y = yaw;
	viewangles.z = roll;
	
	// get the position of the tracker
	arctl_head->getPosition(x, y, z);

	//Get the player model feet position corresponding to the tracker
	matrix3x4_t fTransform = getFeetTransfrom(x, y, z, viewangles.x, viewangles.y, viewangles.z);

	//Translate the tracker to the center of the player's head
	matrix3x4_t xform;
	xform[0][0] =1;
	xform[0][1] =0;
	xform[0][2] =0;
	xform[0][3] =0; //if you need to add an x-axis offset between the tracker and head's center put it here

	xform[1][0] =0;
	xform[1][1] =1;
	xform[1][2] =0;
	xform[1][3] =0; //if you need to add an y-axis offset between the tracker and head's center put it here

	xform[2][0] =0;
	xform[2][1] =0;
	xform[2][2] =1;
	xform[2][3] =0; //if you need to add an z-axis offset between the tracker and head's center put it here	

	//Execute the matrix operation (translation)
	matrix3x4_t translationTemp;		
	ConcatTransforms( fTransform, xform, translationTemp);

	return translationTemp;			
}

/**
* Returns the camera position
*/
void getCameraPosition(float &x, float &y, float &z) {
	matrix3x4_t translationTemp = getCameraTransformation();	
	x = translationTemp[0][3];
	y = translationTemp[1][3];
	z = translationTemp[2][3];
}

In order to get the player's head (which is probably what you are tracking) in the correct location, we need to move the player's feet. These methods figure out where to place the player model feet so the head is where the tracker is...

Step 4 : Modify Valve's Server Notion of Player Position

In the code above, you might have noticed this: 

getCameraPosition(x,y,z);
char command[64];
sprintf(command, "set_user_position %f %f %f", x, y, z);
engine->ClientCmd(command);

in_main.cpp is client-side code, and only effects the game state on the client. However, we also need to tell the Valve server where the player is, even if running a single MOD on a single machine (single player or multiplayer). This snippet we added above sends our calculated player position to the server side of the SDK via a console command (hacky hacky I know -- need to look at a UserCMD here).

We need to create a method on the server side to receive this console command.

Create a file dlls/my_pos_server.cpp.

Add this file to your Visual Studio Project so it gets compiled in.

Put this in the file: 

/***************************************************
*
* Purpose: position change player on the server
*
****************************************************/

#include "cbase.h"
#include "convar.h"

void SetUserPosition( void ){
	CBasePlayer *pPlayer = ToBasePlayer( UTIL_GetCommandClient() );
	if(pPlayer != NULL && (engine->Cmd_Argc() == 4)){

		float x = atof(engine->Cmd_Argv(1));
		float y = atof(engine->Cmd_Argv(2));
		float z = atof(engine->Cmd_Argv(3));

		Vector pos;
		
		//Translate player's feet to tracker
		pos = pPlayer->GetAbsOrigin();
		pos.x = x;
		pos.y = y;
		pos.z = z;
		pPlayer->SetAbsOrigin(pos);
		QAngle angles = pPlayer->GetAbsAngles();

	}
}


static ConCommand set_user_position( "set_user_position", SetUserPosition, "Set the user's position from tracker x,y,z with offset oX, oY, oZ, oPitch, oRoll, oYaw", 0 );

See also

Retrieved from "https://developer.valvesoftware.com/w/index.php?title=Head_Tracking&oldid=327184"