Game Development Community

dev|Pro Game Development Curriculum

Resource Manager: Allow new files to be loaded while still running

by Matt Fairfax · 06/08/2005 (6:28 pm) · 15 comments

When Torque loads, it scans the modPath directories and builds a file list that is used for the duration of that run to access resources and files. If you add a new file to the directories you will be unable to exec/load/access it until you restart Torque. This is a small code change that will allow you load new files without restarting.

It is important to note that this version of the code does allow for files external to the modPaths to be accessed which could be a potential security risk so you may want to comment out a portion of ResManager::find () when you actually release your game.

It is also important to note that this code fix does not handle the case when an existing file changes (reloading the resource). It merely allows new files to be accessed.

In core/resManager.cc replace ResManager::find () with:

ResourceObject *ResManager::find (const char *fileName)
{
   if (!fileName)
      return NULL;
   StringTableEntry path, file;
   getPaths (fileName, path, file);
   ResourceObject *ret = dictionary.find (path, file);
   if (!ret)
   {
         if (Platform::isFile(fileName))
         {
            ret = createResource (path, file);
            dictionary.pushBehind (ret, ResourceObject::File);

            ret->flags = ResourceObject::File;
            ret->fileOffset = 0;
            
            S32 fileSize = Platform::getFileSize(fileName);
            ret->fileSize = fileSize;
            ret->compressedFileSize = fileSize;

            return ret;
         }

      fileIsMissing(fileName);
   }
   return ret;
}

in platform/platform.h somewhere around line 262 (in struct Platform) add:

static S32  getFileSize(const char *pFilePath);

in platformWin32/winFileio.cc around line 610 add:

//--------------------------------------
S32 Platform::getFileSize(const char *pFilePath)
{
   if (!pFilePath || !*pFilePath) 
      return -1;

   // Get file info 
   WIN32_FIND_DATA findData;
   HANDLE handle = FindFirstFile(pFilePath, &findData);
   FindClose(handle);

   if(handle == INVALID_HANDLE_VALUE)
      return -1;

   // if the file is a Directory, Offline, System or Temporary then FALSE
   if (findData.dwFileAttributes & 
       (FILE_ATTRIBUTE_DIRECTORY|                                      
        FILE_ATTRIBUTE_OFFLINE|
        FILE_ATTRIBUTE_SYSTEM|
        FILE_ATTRIBUTE_TEMPORARY) )                             
      return -1;

   // must be a real file then
   return findData.nFileSizeLow;;
}

in platformMacCarb/macCarbFileio.cc around line 1001 add:

//-----------------------------------------------------------------------------
S32 Platform::getFileSize(const char *pFilePath)
{
   if (!pFilePath || !*pFilePath) 
      return -1;
   
   OSErr err = noErr;
   CInfoPBRec catinfo;
   U8 macpath[MAX_MAC_PATH_LONG];
   U32 size;
   
   size = makeMacPath(pFilePath, (char*)(macpath+1));
   if (size>MAX_MAC_PATH)
      return -1;
   macpath[0] = size; // set PString length.

   // clear the paramblock.
   dMemset((void*)&catinfo, 0L, sizeof(catinfo));

   // fill in paramblock.
   catinfo.dirInfo.ioVRefNum = macpath[1]==':'?platState.volRefNum:0;
   catinfo.dirInfo.ioDrDirID = macpath[1]==':'?platState.dirID:0;
   catinfo.dirInfo.ioNamePtr = macpath;
   err = PBGetCatInfoSync(&catinfo);
   if (err!=noErr)
      return -1; // !!!!TBD should log?

   if (CAT_IS_FOLDER(catinfo))
      return -1;

   return catinfo.hFileInfo.ioFlLgLen;
}

in platformX86UNIX/x86UNIXFileio.cc around line 900 add:

//-----------------------------------------------------------------------------
S32 Platform::getFileSize(const char *pFilePath)
{
   if (!pFilePath || !*pFilePath)
      return -1;
   // Get file info
   struct stat fStat;
   if (stat(pFilePath, &fStat) < 0)
      return -1;

   // if the file is a "regular file" then true
   if ( (fStat.st_mode & S_IFMT) == S_IFREG)
      return fStat.st_size;

   // must be some other file (directory, device, etc.)
   return -1;
}

About the author

I am a Game Designer at PopCap who has worked on PvZ Adventures, PvZ2, Peggle Blast, and Bejeweled Skies. I am an ex-GarageGames employee who helped ship TGE, TGEA, Torque 3D, and Constructor.


#1
06/08/2005 (6:35 pm)
nice! This will be very usefull, thank you fellow Matt.
#2
06/09/2005 (6:21 am)
Couldn't you just open the console and exec the script ?
#3
06/09/2005 (6:40 am)
Nice! Thanks for this.
#4
06/09/2005 (1:46 pm)
Chris - When you start TGE it scans the modPath and indexes all of the files it finds. If you create a new file while the engine is running and try to exec it the engine will not be able to find it. you would need to force the engine to rescan the current modPath settings to be able to exec it.
#5
06/09/2005 (4:17 pm)
I can't think of a reason for the life of me to use it, but I'm sure it will come in handy someday...
Anyways thank you SO much for making this crossplatform!
#6
06/10/2005 (8:35 am)
Simply brilliant. I love the little changes that make things so much more flexible. Nice job Matt.
#7
06/10/2005 (9:23 am)
What this is insanely usefull for (someone posted a forum thread about doing the rest of it) is receiving files over the net (or LAN) while the game is running! I haven't done that yet but I have no doubt I'll eventually want to integrate a seamless download scheme while the game is running and this saves a huge headache... lol well saves me the huge headache, Matt probably endured one for all of us :)
#8
06/10/2005 (10:13 am)
Most Excellent.
#9
08/30/2005 (9:28 am)
It seems I found a use for this. :)
#10
11/26/2005 (3:32 pm)
This is *exactly* what I need right in this moment. setModPaths(getModPaths()); is nice, but it seriously drops framerate remarkable for a moment when beeing called in the game. This resource allows me to add only the new file during game runtime. Awesome! Thanks a lot! :-)
#11
11/26/2005 (4:06 pm)
I used this code snippet in main.cc around line 232 to call the function from within script code:

ConsoleFunction( addNewResourceFile, bool, 2, 2, "(string path)"
                "Add a new file to the resource manager.")
{
	char buf[512];
	dStrncpy(buf, argv[1], sizeof(buf) - 1);
	buf[511] = '[[60c1f4f1ac70c]]';

	return bool(ResourceManager->find(buf));
}
#12
08/26/2006 (2:29 am)
Aha ....


Thanks alot , :)
#13
05/21/2007 (8:34 am)
This appears to have already been added to HEAD
#14
10/23/2009 (3:39 am)
Can I use this to access a file anywhere on the c drive? Example:

%fo.openForRead("file://C:/documents/doc.txt");
#15
10/23/2009 (8:37 pm)
Yes I can, thanks to Matt, the slashes have to face the other way though.

%fo.openForRead("C:\\documents\\doc.txt");