Asset Security and Torque File Stream
by Travis Evans · in Technical Issues · 07/31/2010 (8:55 pm) · 7 replies
I've written a nice little application that will encrypt a file with a given offset parameter. I'm wanting to incorporate this into T2D and T3D to secure my models, images, sounds, etc but looking though the code, I see a lot of stream read calls to what looks like different file streams. I'm not sure where to even begin, and was hoping some one a little more familiar with the engine could give me a hand.
If I could get this working, this would be an excellent resource I think.
Thanks
Travis
If I could get this working, this would be an excellent resource I think.
Thanks
Travis
#2
07/31/2010 (10:46 pm)
Sweet! Can't wait then. A reason to look forward to Monday. 
#3
In T3D, there's two separate file system interfaces. One inherited from previous engine versions which is found in core/fileio. This is a raw abstraction over the direct platform file API. Second, there's Torque::FS with its main header in core/volume.h which is a virtualized file system. For the platform file systems, the various backends are in the platform folders (for Windows, it's platformWin32/winVolume.cpp).
Fortunately, no one really uses core/fileio.h and most file I/O (especially for resources) goes through the Stream system with its FileStream class. This is based on Torque::FS.
So, in your case, I'd go and plug my code into Torque::FS and ignore fileio.h.
If you want to go for a real sophisticated solution and also avoid having to change the Torque::FS backend for each platform, then write a separate Torque::FS::FileSystem class that represents a file system with encoded files and then mount that system as "game:/" (look in mainLoop.cpp). Within that file system, access the backend platform file system for raw file reads and writes.
08/01/2010 (12:58 pm)
In T3D, there's two separate file system interfaces. One inherited from previous engine versions which is found in core/fileio. This is a raw abstraction over the direct platform file API. Second, there's Torque::FS with its main header in core/volume.h which is a virtualized file system. For the platform file systems, the various backends are in the platform folders (for Windows, it's platformWin32/winVolume.cpp).
Fortunately, no one really uses core/fileio.h and most file I/O (especially for resources) goes through the Stream system with its FileStream class. This is based on Torque::FS.
So, in your case, I'd go and plug my code into Torque::FS and ignore fileio.h.
If you want to go for a real sophisticated solution and also avoid having to change the Torque::FS backend for each platform, then write a separate Torque::FS::FileSystem class that represents a file system with encoded files and then mount that system as "game:/" (look in mainLoop.cpp). Within that file system, access the backend platform file system for raw file reads and writes.
#4
08/01/2010 (6:58 pm)
Thanks Rene. I'm hoping I don't have to make huge changes. My encoding scheme is it's own class, and really only needs a buffer. I'll poke around and see what I'll be working with.
#5
Actually, while I think that implementing a quick FileSystem is the best solution (look at FileSystemRedirect for an example; you don't need much; basically just a FileSystem class and a File class with the code you already have in File::read and File::write; you can even derive directly from FileSystemRedirect and work on top of that), another option you have is to directly add your code to FileStream. The only problem you have then is how to differentiate between non-encoded and encoded streams and respectively toggle your code in there on and off.
08/01/2010 (7:08 pm)
Actually, while I think that implementing a quick FileSystem is the best solution (look at FileSystemRedirect for an example; you don't need much; basically just a FileSystem class and a File class with the code you already have in File::read and File::write; you can even derive directly from FileSystemRedirect and work on top of that), another option you have is to directly add your code to FileStream. The only problem you have then is how to differentiate between non-encoded and encoded streams and respectively toggle your code in there on and off.
#6
I used a loop, when testing my code, to copy and de/encode. I'm trying to find where this is in Torque, if a loop even exists (Probably missing it as I've been working all weekend and am tired..). But if it does, the implementation is pretty simple.
08/01/2010 (7:48 pm)
If I do add directly into the FileStream, I don't think it'd be too hard to tell the difference between streams, once the file is opened I just have to check a header, say 4 bytes.I used a loop, when testing my code, to copy and de/encode. I'm trying to find where this is in Torque, if a loop even exists (Probably missing it as I've been working all weekend and am tired..). But if it does, the implementation is pretty simple.
// This, of course, is an over simplification
while(!EOF)
{
FS->read(mBuffer, mBufferLen);
if(rF->encodedStream())
rF->decode(mBuffer, mBufferLen);
}
#7
FileStream won't necessarily slurp the entire file into a buffer in one go but rather let the user read out data incrementally and buffer it internally. Unfortunately, that's not all in one place. The two functions you probably want to look for are FileStream::_read and FileStream::fillBuffer. In both cases, simply add your decode() call directly after the mFile->read calls.
08/01/2010 (7:57 pm)
FileStream won't necessarily slurp the entire file into a buffer in one go but rather let the user read out data incrementally and buffer it internally. Unfortunately, that's not all in one place. The two functions you probably want to look for are FileStream::_read and FileStream::fillBuffer. In both cases, simply add your decode() call directly after the mFile->read calls.
Employee Michael Perry
GarageGames