Game Development Community

Zip files - What am I doing wrong?

by Jaimi McEntire · in Torque Game Engine Advanced · 03/11/2009 (9:23 pm) · 22 replies

I zipped up the ScriptsAndAssets directory (by going into the ScriptsAndAssets directory) - I used "copy" mode, there was no compression, and no encryption. The path inside the ScriptsAndAssets.zip file does not include "ScriptsAndAssets" - just the directories underneath. They look like this:

client/config.cs
data/textures/detail001.png

etc.

the resource manager loads the zip file, and creates a resource object for every file in the zip.

The engine goes to execute the .CS files, and they work. It loads the profiles, etc.

But textures and DTS files do not load - every single one of them errors out, the screen turns pink and the engine just sits there.

Has anyone successfully used zip files in TGEA 1.8? Have I missed something?

I've tried 7zip, WinRar, and WinZip.

I'm absolutely sure that I must be misunderstanding something, and that TGEA is golden. However, I can't figure it out.
Page «Previous 1 2
#1
03/11/2009 (10:47 pm)
The console.log is littered with notices like these:

[Win32FileSystemChangeNotifier::internalAddNotification] : failed on [C:\Torque\test\scriptsAndAssets\data\shapes\character01] [3]
Resource<TSShape>::create - Could not open 'scriptsAndAssets/data/shapes/character01/player.dts'

Failed to load profile bitmap (scriptsAndAssets/client/ui/demoWindow)

CubemapData::createMap - Failed to load texture 'scriptsAndAssets/data/water/sky_x_pos3'

But it had to execute the .CS files in the zip before it even knew about these, so I am convince the CS files are working. However, nothing else appears to be.
#2
03/12/2009 (5:01 am)
If memory serves me correctly, Torque has always balked at reading texture files from zips.
#3
03/12/2009 (5:59 am)
I confirm what Jaimi found.

I just made the test on Mac on TEGA 1.8.1. I zipped the content of the scriptsAndAssets folder to a ZIP file, remove the folder and launch my app. It launches, but no graphic file is shown (even in GUI°.

It used to work correctly with TGE 1.5.2.

Nicolas Buquet
www.buquet-net.com/cv/
#4
03/12/2009 (8:36 am)
I can confirm this too. However, it wasn't the case back in 1.7.1.
#5
03/12/2009 (10:46 am)
Man, I had hoped so much it was me. :(
#6
03/12/2009 (11:55 am)
This is probably a bug originating from the fact that there are two resource managers in TGEA 1.8.x. I've forwarded it to the Torque 3D team and hopefully we'll be able to get a fix to current TGEA users as well.
#7
03/12/2009 (8:38 pm)
Yes, this is exactly what I have seen.

Sound effects, Scripts, and DIF files use the old resource manager - the one that supports Zip files. Shapes, Terrains, Textures, and fonts use the New resource manager, which does not support zip files.
#8
03/13/2009 (4:13 am)
Yeah, it's quite unfortunate. :(

Holding my thumbs, Alex.
#9
03/13/2009 (4:49 am)
Ahhh this would explain a lot, it's on my list of things to have looked into as it worked ok in 1.7.1 but not in 1.8.1 - certainly something I'd like to see resolved.
#10
03/18/2009 (8:31 pm)
Just a note that I have programmed around this. It required way too many changes for me to list here, and I did most of the work under contract so I can't just post all of the files, but if anyone needs to do this, I would be willing to post in a little more detail.
#11
05/30/2009 (1:13 pm)
Any progress? Has been saying "We're working on it" for a while now.
Alex, if there's a solution for this, could it be posted seperatly at the forums? Since it's not certain that we'll see a 1.8.2.
#12
05/30/2009 (2:04 pm)
There is no benefit for GG to release a 1.8.2 version. IMO, they don't want TGEA improved because that means less T3D potential customers.
- Otherwise, we would have all of Alex's fixes for 1.8.1 released for our use. Call me stupid if I am wrong about that...but.. I severely doubt that I am wrong.

Note: *If you would just like to call me stupid anyways..go ahead. I am from NY...I hear worse most days! :P

[edit: oops!, sorry for the hijack!]
#13
05/31/2009 (6:53 am)
Ye, that's not exactly what I wanted to discuss..

I don't think it would have an impact on the number of people making the upgrade, but rather the opposite. If TGEA is gonna be left with bugs like this then that would make me think twice about upgrading to something which also might be left with bugs when the time comes for a new upgrade.

I'm still hoping some of these fixes will be made available.
#14
07/05/2009 (5:26 am)
Just to let you conspiracy theorists know: T3D also has the very same bug.
#15
07/05/2009 (5:52 pm)
This is not really a bug per se. The new resource manager does not do any file I/O of its own and is completely separate of the filesystem handling for which there is a new system. Unfortunately, zip volumes are not implemented yet.

Also, the new filesystem unfortunately follows the Windows model where volumes are mounted to separate global namespaces rather than the Unix model where volumes are all mounted into a single namespace. This makes it less straightforward to just mount individual zip files within the game folder to the paths they sit in.
#16
09/06/2009 (4:27 pm)
Jaimi, when you worked around this did you switch shapes, textures etc back to the old resource manager or did you get zip loading working in the new resource manager. Any pointers would be greatly appreciated as content protection is pretty important to a project I am working on.
#17
09/06/2009 (8:44 pm)
I got it into the new resource manager. It wasn't that much of a hassle, just look at ZipArchive.cpp. Instead of passing filenames to Win32File or whatever it is named, pass it to ZipArchive and open a stream there.

Then it's just a matter of making sure the zip is closed, and file handles into it are cleared when it's closed. At first I went around and did all sorts of complex hacks before I realized it wasn't much harder than that.

I got it working with the old resource manager before, but that was toooons more work than the above solution.

Edit: Obviously that doesn't mimic how the old zip stuff is working but I wanted the whole data folder into a zip anyway. If you need the old Torque zip functionallity, I think it's easier to use ResourceOld as mentioned.
#18
09/06/2009 (8:59 pm)
Cheers Stefan, I was going down the ResourceOld route but that sounds much simpler. I don't really need it to mimic the old functionality as long as it allows me to protect the art assets.

#19
09/06/2009 (10:05 pm)
@Adrian - I hacked it into the new Resource Manager also. I created a ZipFileStream that leveraged the existing zipArchive code, and used that when the resource was in a Zip File:

#include "zipFileStream.h"
#include "core/zip/zipArchive.h"

ZipFileStream::ZipFileStream(ResourceObject *ro)
{
   baseStream = NULL;
   zipResource = ro;
}

ZipFileStream::~ZipFileStream()
{
   if (baseStream)
      delete baseStream;
}

void 
ZipFileStream::Init()
{
   mStreamCaps = 0;
   Stream::setStatus(Closed);
}

void ZipFileStream::SetStatus()
{
   Stream::setStatus(baseStream->getStatus());
}

U32  ZipFileStream::getPosition() const
{
   if (baseStream)
   {
      U32 ret = baseStream->getPosition();
      // Wow, this cast is retarded.
      ((ZipFileStream *)this)->SetStatus();
      return ret;
   }
   return 0;
}

bool ZipFileStream::setPosition(const U32 i_newPosition)
{
   if (baseStream)
   {
      bool ret = baseStream->setPosition(i_newPosition);
      Stream::setStatus(baseStream->getStatus());
      return ret;
   }
   return false;
}

U32  ZipFileStream::getStreamSize()
{
   if (baseStream)
   {
      U32 ret = baseStream->getStreamSize();
      Stream::setStatus(baseStream->getStatus());
      return ret;
   }
   return 0;
}

   // additional methods needed for a file stream...
bool ZipFileStream::open(const String &inFileName, Torque::FS::File::AccessMode inMode)
{
   if (!zipResource) return false;
   if (!zipResource->mZipArchive) return false;

   baseStream = zipResource->mZipArchive->openFileForRead(zipResource->mCentralDir);
   Stream::setStatus(baseStream->getStatus());
   if (baseStream->getStatus() == Stream::Ok)
      return true;
   else
      return false;
}

void ZipFileStream::close()
{
   // Don't actually close the zip.
   // that will be done at cleanup time.
   return;
}

bool ZipFileStream::flush()
{
   // read only!
   return true;
}


bool ZipFileStream::_read(const U32 i_numBytes, void *o_pBuffer)
{
   return ((FileStream *)baseStream)->_read(i_numBytes, o_pBuffer);
}

bool ZipFileStream::_write(const U32 i_numBytes, const void* i_pBuffer)
{
   return true;
}
#20
09/07/2009 (5:59 pm)
Cool, Thanks for that Jaimi!
Page «Previous 1 2