MDX info - Articles

Disecting ShipHo! - Resource management

By Rim van Wersch, January 12 2006

We'll begin this series with a look at ShipHo!'s resource management. Every game needs to manage some resources and it pays off later to have a proper resource manager implemented so you don't have to worry about it when you start using more resources in your game. Our resource manager for ShipHo! is pretty simple, since we can easily have all of our limited number of resources loaded in memory at the same time. This greatly simplifies our design, because we won't have to worry about unloading any resources from memory to free space for others.

So what's really left to implement for our resource manager then? Don't worry, there's still plenty to do. Obviously you'll need to load your resources in the first place, but if you've ever coded something in DirectX, you know you'll need to recreate those resources whenever your device gets reset. So we'll need to include that into the design of our resource manager too. And finally it would be nice if we could get a list of the resources loaded by the resource manager at any given time our game is running. This is useful for debugging and we can use the list to preload our resources when the game is started, which will remove any lag due to 'on demand' resource loading.

About the implementation

Now that we've figured out the basic requirements for our resource manager, how do we implement it? For this we'll use a basic caching setup, based on the common hashtable container (this data structure basically allows you to associate keys with objects). We'll ignore the preloading for now as we can easily fit that in later. So we'll define a ResourceManager class with a bunch of hashtables, for textures, meshes, effects and whatever resource we need. By writing resource specific get functions, like GetTexture(), we can lazily load the resources whenever we need them.

Now what does that mean? Easy, whenever we need a resource, we'll call the appropriate GetSomeResourceType method on the resource manager. The manager will then check if it has already loaded the requested resource, by looking up the key for the specific resource in the hashtable for this resource type (we'll get to these keys in a bit). If it exists, it simply returns the resource object from the hashtable (often called the cache, or pool as in our resource manager). If it doesn't, it will load the resource on the fly, using the appropriate method for the specific resource type. This may sound very fancy, but it's really quite simple. For textures for example, this method looks like this:

public Texture GetTexture( string textureName ) 
{ 
 ? ?return this.GetTexture( textureName, true ); 
} 
 
public Texture GetTexture( string textureName, bool generateMipSublevels ) 
{ 
 ? ?if (textureName == null) 
 ? ?{ 
 ? ? ? ?return null; 
 ? ?} 
 ? ?else if ( !texturePool.ContainsKey( textureName )) 
 ? ?{ 
 ? ? ? ?Texture texture = LoadTexture( textureName ); 
 
 ? ? ? ?if (generateMipSublevels) 
 ? ? ? ?{ ? ? ? ? ? ? ? ? ? ? ? ? 
 ? ? ? ? ? ?texture.GenerateMipSubLevels(); 
 ? ? ? ?} 
 
 ? ? ? ?texturePool.Add( textureName , texture ); 
 ? ? ? ?return texture; 
 ? ?} 
 ? ?else 
 ? ?{ 
 ? ? ? ?return (Texture)texturePool[textureName]; 
 ? ?} 
}

As you can see, this method does what we explained above. First we catch the case when the texture name is null, then we simply return a null texture. We check if the texturePool hashtable object contains a texture with the requested name. If it doesn't, we'll load the texture, optionally generate Mips, store it in the hashtable and return it. Otherwise, we simply return it from the hashtable. The LoadTexture method calls our FileResourceLoader helper object, which is just a wrapper around the normal MDX functions for loading resources from local files. By doing the actual loading in our seperate ResourceLoader object, we can easily plug in other ways to load the resources later, like from a network or an archive file for example.

Loading and rendering meshes

As we've seen in the previous code snippet, we use the texture (file)name as the key in our texture hashtable pool and we do the same for any Effect and Audio objects we need, since we only need to get these objects themselves to use them. For meshes though, we need some more information to render them, like the number of subsets, which textures and materials to use and some more useful properties. To accommodate this, we'll create a little struct to hold this data for us to simplify our rendering process. The definition of this MeshDescription struct looks like this:

public struct MeshDescription 
{ 
 ? ?public string meshName; 
 ? ?public string[] textureNames; 
 ? ?public Material[] meshMaterials; 
 ? ?public int subsets; 
 ? ?public bool hasAlphaTextures; 
 ? ?public Vector3 meshCenter; 
 ? ?public float meshRadius; 
 ? ?public bool hasHitRegion; 
 ? ?public Util.Polygon hitRegion; ? ? ? ? 
 ? ? 
 ? ?public MeshDescription( string meshName, string[] textureNames, Material[] meshMaterials, bool hasAlphaTextures, Vector3 meshCenter, float meshRadius, bool hasHitRegion, Util.Polygon hitRegion ) 
 ? ?{ 
 ? ? ? ?this.meshName = meshName; 
 ? ? ? ?this.textureNames = textureNames; 
 ? ? ? ?this.meshMaterials = meshMaterials; 
 ? ? ? ?this.subsets = meshMaterials.Length; 
 ? ? ? ?this.hasAlphaTextures = hasAlphaTextures; 
 ? ? ? ?this.meshCenter = meshCenter; 
 ? ? ? ?this.meshRadius = meshRadius; 
 ? ? ? ?this.hasHitRegion = hasHitRegion; 
 ? ? ? ?this.hitRegion = hitRegion; 
 ? ?} 
}

So, this struct holds the material and texture information obtained from a typical .X mesh file and some more useful information, like information on the mesh's bounding sphere, a 2D hit region for collision detection and a boolean indicating whether or not the mesh uses textures with an alpha component. We'll see how this information can be used in some of the next sections, but for now we'll just check out a quick example on how our ResourceManager can be used to render a simple mesh. Let's take a look at the code needed for this:

public class SomeFictionalSceneObject 
{ 
 ? ?private bool initialized = false; 
 ? ?private MeshDescription meshDescription; 
 
 ? ?private void Initialize() 
 ? ?{ 
 ? ? ? ?// fetch a mesh description for our mesh name 
 ? ? ? ?this.meshDescription = ResourceManager.DefaultManager.GetMeshDescription( "someMeshFile.x" ); 
 ? ? ? ?this.initialized = true; 
 ? ?} 
 
 ? ?public void Render(Device device) 
 ? ?{ 
 ? ? ? ?// fetch our mesh description, once in the object's lifetime 
 ? ? ? ?if (!initialized) 
 ? ? ? ?{ 
 ? ? ? ? ? ?Initialize(); 
 ? ? ? ?} 
 ? ? ? ? 
 ? ? ? ?// render the mesh as usual 
 ? ? ? ?Mesh mesh = ResourceManager.DefaultManager.GetMesh( meshDescription.meshName ); 
 
 ? ? ? ?for(int i = 0; i < meshDescription.subsets; i++) 
 ? ? ? ?{ ? ? ? ? ? ? ? ? 
 ? ? ? ? ? ?device.Material = meshDescription.meshMaterials[i]; ? ? ? ? ? ? ? ? ? ? 
 ? ? ? ? ? ?device.SetTexture( 0, ResourceManager.DefaultManager.GetTexture( meshDescription.textureNames[i] ) ); 
 
 ? ? ? ? ? ?mesh.DrawSubset(i); 
 ? ? ? ?} 
 ? ?} 
}

Handling device resets

Hopefully you'll agree that this is a nice, clean way of rendering a mesh. We only need to store a few local instance variables to get the information we need from our ResourceManager. And, as you may have noticed, the SomeFictionalSceneObject class does not store any device-bound data, so no Texture or Mesh objects. This is great, since now we don't need to worry anymore about recreating the resources for each scene object on a device reset.

Obviously we'll still need to handle the device resets, but we can do that in our ResourceManager class now since that's where all resources are kept. Simply put, the scene objects only 'borrow' the resources from the ResourceManager for their rendering process. So to handle device resets, we implement a Reset(Device device) and a Dispose() method for the ResourceManager class, which will take care of our resources for us. After that, you can just call these methods from your application's main Reset and Dispose event handlers/callbacks. Below is the code for these two functions, which isn't all that complicated.

public void Reset(Device device) 
{ 
 ? ?// update local reference 
 ? ?this.currentDevice = device; 
 
 ? ?// create a new effect pool 
 ? ?effectPool = new EffectPool(); 
 ? ? 
 ? ?if (reloadResourcesAfterReset) 
 ? ?{ 
 ? ? ? ?// Reload all resources described in our MeshDescriptions. 
 ? ? ? ?// This will leave out any 'loose' textures, but we won't use
 ? ? ? ?// these much, so we'll rely on our lazy approach to load these. 
 ? ? ? ?foreach( MeshDescription desc in descriptionPool.Values ) 
 ? ? ? ?{ 
 ? ? ? ? ? ?ExtendedMaterial[] unused; 
 ? ? ? ? ? ?meshPool.Add( desc.meshName, LoadMesh( desc.meshName, out unused ) ); 
 
 ? ? ? ? ? ?foreach (string textureName in desc.textureNames) 
 ? ? ? ? ? ?{ 
 ? ? ? ? ? ? ? ?if (textureName != null && (!texturePool.ContainsKey( textureName ))) 
 ? ? ? ? ? ? ? ?{ 
 ? ? ? ? ? ? ? ? ? ?texturePool.Add( textureName, LoadTexture( textureName )); 
 ? ? ? ? ? ? ? ?} 
 ? ? ? ? ? ?} 
 ? ? ? ?} 
 ? ?} 
} 
 
public void Dispose() 
{ 
 ? ?// dispose all meshes 
 ? ?foreach( Mesh mesh in meshPool.Values ) 
 ? ?{ 
 ? ? ? ?mesh.Dispose(); 
 ? ?} 
 
 ? ?// clear our hashtable 
 ? ?meshPool.Clear(); 
 
 ? ?// dispose our textures 
 ? ?foreach( Texture texture in texturePool.Values ) 
 ? ?{ 
 ? ? ? ?texture.Dispose(); 
 ? ?} 
 
 ? ?// and clear a hashtable again 
 ? ?texturePool.Clear(); 
 
 ? ?// dispose of the effectpool object 
 ? ?effectPool.Dispose(); 
}

Resource usage and preloading

So with all that out of the way, let's see what else we need to implement for our ResourceManager. We've got the resource loading taken care off and we've implemented an easy way to handle device resets. All that's left is our resource usage output and resource preloading. First let's take a look at outputting the resources our ResourceManager has currently loaded into memory. Because we used hashtables to stored our resource objects, ouputting the resources is simply a matter of printing the hashtable keys we used for them:

public void PrintResourceList() 
{ 
 ? ?System.Diagnostics.Debug.WriteLine("---------------- Resource List Start --------------------"); 
 
 ? ?foreach(string meshName in meshPool.Keys) 
 ? ?{ 
 ? ? ? ?System.Diagnostics.Debug.WriteLine( "mesh:" + meshName ); ? ? ? ? ? ? ? ? 
 ? ?} 
 
 ? ?foreach(string textureName in texturePool.Keys) 
 ? ?{ 
 ? ? ? ?System.Diagnostics.Debug.WriteLine( "texture:" + textureName ); ? ? ? ? ? ? ? ? 
 ? ?} 
 
 ? ?foreach(string audioName in audioPool.Keys) 
 ? ?{ 
 ? ? ? ?System.Diagnostics.Debug.WriteLine( "audio:" + audioName ); ? ? ? ? ? ? ? ? 
 ? ?} 
 
 ? ?System.Diagnostics.Debug.WriteLine("---------------- Resource List End ---------------------"); 
}

Armed with this resource list, we can also quite easily implement our resource preloading to finish the implementation of our ResourceManager. We'll store the resource list printed by the code above in a simple text file and use that as our list of resources which we want to preload. As luck would have it, our ResourceManager already supports a method for retrieving the lines of such a simple text file in an easy-to-use ArrayList, so we'll use that to find out which resources we'll be preloading. The code below shows how this is done in our ResourceManager:

// variables 
private ArrayList preloadList = new ArrayList(); 
private int currentPreloadIndex = 0; 
 
public bool ResourcesPreloaded 
{ 
 ? ?get { return (currentPreloadIndex >= preloadList.Count); } 
} 
 
public int CurrentPreloadIndex 
{ 
 ? ?get { return currentPreloadIndex; } 
} 
 
public int ResourcesToPreload 
{ 
 ? ?get { return preloadList.Count; } 
} 
 
// constructor 
private ResourceManager() 
{ ? ? ? ? ? ? ? ? 
 ? ?preloadList = this.GetTextResourceLines( "preload.stuff" ); 
} ? ? 
 
// our workhorse 
public void PreloadNextResource() 
{ 
 ? ?if( ResourcesPreloaded ) 
 ? ?{ 
 ? ? ? ?// bail out 
 ? ? ? ?return; 
 ? ?} 
 
 ? ?string line = (string)preloadList[ currentPreloadIndex ]; 
 
 ? ?int colonIndex = line.IndexOf(":"); 
 ? ?string type = line.Substring(0, colonIndex); 
 ? ?string resourceName = line.Substring( colonIndex + 1 ); 
 
 ? ?if (type == "mesh") 
 ? ?{ 
 ? ? ? ?this.GetMeshDescription( resourceName ); 
 ? ?} 
 ? ?else if (type == "audio") 
 ? ?{ 
 ? ? ? ?this.GetAudio( resourceName ); 
 ? ?} 
 ? ?else if (type == "texture" ) 
 ? ?{ 
 ? ? ? ?this.GetTexture( resourceName ); 
 ? ?} 
 ? ?else 
 ? ?{ 
 ? ? ? ?System.Diagnostics.Debug.WriteLine("Ignoring unknown resource preload, type: " + type); 
 ? ?} 
 
 ? ?currentPreloadIndex++; 
}

So what's going on here? First our constructor gets called, which will fetch the arraylist filled with lines of text that contain the resource information as printed by our PrintResourceList(). This way, we know the type of resource we need to preload as well as its filename. The main point of interest here is of course our workhorse method PreloadNextResource(), which will preload the next resource in the list by simply fetching it from the ResourceManager itself. Here our lazy loading approach will kick in again and make sure the resource is loaded correctly. The CurrentResourceIndex and ResourcesPreloaded properties can be used to show a nice loading screen with a progress bar.

For people coming from a Java background or those who have been coding Windows Forms applications in C#, this method may work a bit counter-intuitive. After all, you could just preload all resources at once on a seperate thread, right? While this may be entirely possible, in MDX you are typically better off using a single thread of execution.

When using a seperate thread for this, you'll need to update the currentResourceIndex from that thread. That's no big problem in itself, but because the currentResourceIndex could be accessed more than a thousand times per second by the main update/rendering thread, this becomes very sensitive to locking. So we'll save ourselves these troubles and just call PreloadNextResource() from the update/rendering thread each frame before doing any rendering. This will indeed decrease the framerate, but since we'll only be showing a progress bar in this preload phase, it doesn't matter much.

Wrapping it up

This conludes our section on the ResourceManager used by ShipHo! Hopefully this article clarified how it works and maybe gave you some ideas on how to go about resource management yourself. For more information on resource management, check out the links below. In the next part of this series, we'll be taking a closer look at how ShipHo! manages its scene objects for rendering and interaction.

Back to Introduction | Continue reading about Scene Management

Files for this article

Filename	Size
? ShipHo source project.zip	4.2 MB

Further reading

MSDN page about best practices in resource management
?