MDX info - Tutorials

Picking tutorial, intersecting a mesh with a mouse ray

By Rim van Wersch, April 9 2006

Intersecting a mesh with a mouse ray, often referred to as picking, is a technique to select meshes in your 3D world by clicking your mouse. In this tutorial we'll take a look at a nice clean way to do this in MDX and what can be done with the information obtained from an intersection test. This tutorial comes with a port of the C++ picking sample from the SDK.

How does it work?

The technique of picking essentially comes down to translating your mouse coordinates to a ray defined in world space and performing an intersection test between this ray and suitable meshes you want to be able to pick. Suitable meshes typically means visible meshes, so as a side note, picking may be optimized by using view frustum checks and/or occlusion queries to reduce the number of intersection tests. But let's not get ahead of ourselves and preview the steps we'll perform in this tutorial.

Obtain proper mouse coordinates
Construct a world space ray from these screen spaced coordinates
Perform the intersection test between the ray and a mesh, ie. performing the actual pick
Use intersection information to obtain the intersected triangle, which may be useful in some applications

So let's start with obtaining our mouse coordinates. Since we're using the SampleFramework, we might as well use the OnMsgProc approach rather than the more common .NET mouse events. The OnMsgProc approach allows you to handle window messages directly, which is more efficient than using the event model and it's also potentially more flexible, since you can use one single method to handle all keyboard and mouse input. It's a bit more lowlevel, but as you can see in the code sample below, it's no rocket science. Picking will work with whatever approach you prefer though, so if you are more comfortable with events you can just as well use those.

if (msg == NativeMethods.WindowMessage.LeftButtonDown) 
{ 
 ? ?// set up mouse capture for 'drag picking' 
 ? ?NativeMethods.SetCapture( this.sampleFramework.Window.Handle ); 
 ? ?isPicking = true; 
 ? ? 
 ? ?// perform picking/intersection test 
 ? ?short mouseX = NativeMethods.LoWord((uint)lParam.ToInt32()); 
 ? ?short mouseY = NativeMethods.HiWord((uint)lParam.ToInt32()); 
 ? ?DoPicking(mouseX, mouseY); 
}

Now that we have our mouse coordinates, let's take a look at what's going on in our DoPicking method, for which you can find the code below. The first thing you'll notice is that we clamp our mouse coordinates to our device's viewport. For translating our mouse coordinates to a world space ray, we'll be using our viewport's height and width to do some calculations (as explained in Toymaker's tutorial linked below), so we'll need to make sure the coordinates are contained within this expected range. Normally you wouldn't need to worry about this, but since we're capturing mouse movements (tracking drags outside our control's client area), we'll need this adjustment.

Next we move on to constructing the ray. We define two Vector3's that describe the origin (near) and direction (far) of the ray in screen space. By using the Vector3.Unproject method, we translate this screen space ray into our mesh's model space. What's that you say? We were going to translate the ray into world space? That's correct, but for performing the intersection test, we'll need to have our ray define in the mesh's model space, so we conveniently supply the world matrix used for our mesh to the Unproject method to skip this final transformation. By supplying Matrix.Identity you could obtain the ray defined in world space, but for typical intersection tests you won't need this.

Now that we finally have obtained our mouse ray in the desired model space, we can perform the intersection test. The Mesh.Intersect method provides various overloads that allow you to retrieve all intersections of the ray with the mesh, but since these can be handled exactly like the closest intersection, we'll stick to this closest one only for clarity. We obtain the IntersectionInformation object from the Mesh.Intersect method as an out parameter, while the boolean returned from this function indicates whether or not an intersection occured. For most applications, you'll only want to know if a specific mesh was clicked on, so in these cases the boolean value is sufficient and you're done.

private void DoPicking(short mouseX, short mouseY) 
{ 
 ? ?// Clamp mouse coordinates to viewport 
 ? ?if (mouseX < 0) mouseX = 0; 
 ? ?if (mouseY < 0) mouseY = 0; 
 ? ?if (mouseX > this.sampleFramework.Device.Viewport.Width) mouseX = (short)this.sampleFramework.Device.Viewport.Width; 
 ? ?if (mouseY > this.sampleFramework.Device.Viewport.Height) mouseY = (short)this.sampleFramework.Device.Viewport.Height; 
 
 ? ?// Put mouse coordinates in screenspace Vector3's. These are the points 
 ? ?// defining our ray for picking, which we'll transform back to world space 
 ? ?Vector3 near = new Vector3(mouseX, mouseY, 0); 
 ? ?Vector3 far = new Vector3(mouseX, mouseY, 1); 
 
 ? ?// Transform points to world space 
 ? ?near.Unproject(this.sampleFramework.Device.Viewport, camera.ProjectionMatrix, camera.ViewMatrix, scannerWorldMatrix * Matrix.RotationY(scannerRotationTimer)); 
 ? ?far.Unproject(this.sampleFramework.Device.Viewport, camera.ProjectionMatrix, camera.ViewMatrix, scannerWorldMatrix * Matrix.RotationY(scannerRotationTimer)); 
 
 ? ?// Retrieve intersection information 
 ? ?IntersectInformation closestIntersection; 
 ? ?bool intersects = scannerMesh.Intersect(near, far, out closestIntersection); 
 
 ? ?if (intersects) 
 ? ?{ 
 ? ? ? ?// If you only want to confirm intersection of the mesh (ie. whether it 
 ? ? ? ?// was clicked on), you can just use this boolean and you're done. 
 ? ? ? ?status = string.Format("Face={0}, tu={1}, tv={2}", closestIntersection.FaceIndex, closestIntersection.U, closestIntersection.V); 
 
 ? ? ? ?// We'll continue here with showing how to obtain the intersected face 
 ? ? ? ?HighlightIntersectedFace(closestIntersection); 
 ? ?} 
 ? ?else 
 ? ?{ 
 ? ? ? ?status = "Use mouse to pick a polygon"; 
 ? ?} 
}

If you need further information about the intersection, you can use the IntersectInformation object to calculate what you need. This may be useful for a modelling package for example, where users need to be able to select a specific face. So, let's investigate how we can obtain this face from our mesh by using the data in our IntersectionInformation object.

The IntersectionInformation object contains the index of the face that was intersected during the test, so we'll need to find a way to extract the vertex information for this face from our mesh. Fortunately, that's not too hard, as you can see in the code for this below. Since all faces in a typical D3DX mesh are triangles, we can easily obtain the indices for the intersected triangle from the mesh's index buffer by reading three indices starting from 3 * FaceIndex. Once we have these indices, it's simply a matter of extracting the vertex information from the mesh's vertex buffer for these indices.

private void HighlightIntersectedFace(IntersectInformation ii) 
{ 
 ? ?// create an array to hold the indices for the intersected face 
 ? ?short[] intersectedIndices = new short[3]; 
 
 ? ?// fetch indices for the intersected face from the mesh 
 ? ?short[] indices = (short[])scannerMesh.LockIndexBuffer( typeof(short), LockFlags.ReadOnly, scannerMesh.NumberFaces * 3); 
 ? ?Array.Copy( indices, ii.FaceIndex * 3, intersectedIndices, 0, 3 ); 
 ? ?scannerMesh.UnlockIndexBuffer(); ? ? ? ? 
 
 ? ?// NOTE: we could also use these indices to render the intersected 
 ? ?// face, by simply reusing the vertexbuffer of the mesh for our 
 ? ?// vertex stream source. We'll continue to extract this data below. 
 
 ? ?// create an array to hold the vertices for the intersected face 
 ? ?CustomVertex.PositionNormalTextured[] tempIntersectedVertices = new CustomVertex.PositionNormalTextured[3]; 
 
 ? ?// extract vertex data from mesh, using our indices we obtained earlier 
 ? ?CustomVertex.PositionNormalTextured[] meshVertices = (CustomVertex.PositionNormalTextured[])scannerMesh.LockVertexBuffer(typeof(CustomVertex.PositionNormalTextured), LockFlags.ReadOnly, scannerMesh.NumberVertices); 
 ? ?tempIntersectedVertices[0] = meshVertices[ intersectedIndices[0] ]; 
 ? ?tempIntersectedVertices[1] = meshVertices[ intersectedIndices[1] ]; 
 ? ?tempIntersectedVertices[2] = meshVertices[ intersectedIndices[2] ]; 
 ? ?scannerMesh.UnlockVertexBuffer(); 
 
 ? ?// we'll wait till here before updating the vertices used  
 ? ?// for rendering, so we don't render incomplete data 
 ? ?this.intersectedVertices = tempIntersectedVertices; 
}

This concludes our tutorial on performing picking. One step remains undiscussed here, that of determining the exact point of intersection by using the barycentric coordinates supplied in our IntersectionInformation object. This is however rarely used for picking, so if you require more information on this, please refer to the article on FlipCode linked to below or to the SDK documentation, where you'll find this explained in more detail.

About the sample project

The sample project you can download below is a port of the C++ picking sample provided in the DirectX SDK, with the ommission of checking for multiple intersections and calculating the exact point of intersection using the barycentric coordinates. For more information about the latter, please refer to the link below to the article on FlipCode. Handling multiple intersections isn't much different from handling a single one, so this is left as an exercise for the reader

The transformation of the mouse coordinates from screen space to world space is implemented a bit easier by using the Vector3.Unproject function from the Managed DirectX API. Under the hood this works exactly like the C++ sample and as such uses the same steps as explained in Toymaker's turorial on picking, which you can also find in the Further Reading section below.

Files for this tutorial

Filename	Size
? Picking.zip	708.5 KB

Further reading