Bug in ZipSubRStream?
by Jari · in Torque Game Engine · 04/17/2006 (6:03 am) · 8 replies
Hello I have ran into a bug using the zip decompression functionality TGE 1.4 has.
First everything works fine in my code, it unzips few files but after that I get this assertion failed message: "Fill from a closed stream?"
After tracking down this problem I came to one line that seems to cause at least one bug, in core\zipSubStream.cc line 137 there is call to inflate() which happens only if "if(m_pZipStream->avail_in == 0)". This is wierd because the ziplib faq says that inflate should never be called when avail_in is zero. See: http://www.zlib.net/zlib_faq.html#faq05
And quess what avail_in was before the assertion failure, zero and the inflate of course then returned Z_BUF_ERROR.
I tried simply changing the "== 0" check to "!= 0" but this did not help. I am sorry but I don't know what else to say about this bug because I don't understand the code behind zlib.
It would be great if some one could take a look on this.
PS. I'm thinking that there could be buffer overflow some where in my code but since the code is illogical in ZipSubRStream::_read I decided to report this.
Jari.
First everything works fine in my code, it unzips few files but after that I get this assertion failed message: "Fill from a closed stream?"
After tracking down this problem I came to one line that seems to cause at least one bug, in core\zipSubStream.cc line 137 there is call to inflate() which happens only if "if(m_pZipStream->avail_in == 0)". This is wierd because the ziplib faq says that inflate should never be called when avail_in is zero. See: http://www.zlib.net/zlib_faq.html#faq05
And quess what avail_in was before the assertion failure, zero and the inflate of course then returned Z_BUF_ERROR.
I tried simply changing the "== 0" check to "!= 0" but this did not help. I am sorry but I don't know what else to say about this bug because I don't understand the code behind zlib.
It would be great if some one could take a look on this.
PS. I'm thinking that there could be buffer overflow some where in my code but since the code is illogical in ZipSubRStream::_read I decided to report this.
Jari.
#2
yes I can reproduce it, it happens when fifth zip file is being read. I mean my code unpacks four zip files normally but fifth causes the assertion but it doesnt seem to matter what is the fifth file. Doesnt that imply that some buffer runs out of space?
04/17/2006 (8:25 am)
Hi Stefan, yes I can reproduce it, it happens when fifth zip file is being read. I mean my code unpacks four zip files normally but fifth causes the assertion but it doesnt seem to matter what is the fifth file. Doesnt that imply that some buffer runs out of space?
#3
It's a defaultProfile.cs.dso packed in a zip (it really is dso even the file name is different)
Could some one check this out? I am glueless what to try next.
PS. I used windows XP's packagin tool to create the zip but I have also tried winace, powerarchiver and even python's zib lib. I also ran CRC test in python which reported no errors.
Jari.
04/21/2006 (4:45 am)
It turns out that the crash happens on some files only, like with this one: zipIt's a defaultProfile.cs.dso packed in a zip (it really is dso even the file name is different)
Could some one check this out? I am glueless what to try next.
PS. I used windows XP's packagin tool to create the zip but I have also tried winace, powerarchiver and even python's zib lib. I also ran CRC test in python which reported no errors.
Jari.
#4
Please let me know if you see anything odd with that code.
Jari.
04/21/2006 (5:03 am)
I just realized that it's possible that the bug is in my code since I have added new function in ZipAggregate class and also a consolefunction, here they are:FilterStream* ZipAggregate::openStreamForEntry(const ZipAggregate::FileEntry &fe)
{
AssertFatal(m_pZipFileName,"ZipAggregate is not open")
FileStream diskStream;
diskStream.open(m_pZipFileName,FileStream::Read);
diskStream.setPosition(fe.fileOffset);
ZipLocalFileHeader zlfHeader;
if (zlfHeader.readFromStream(diskStream) == false)
{
Con::errorf("'%s' is not in the zip",fe.pFileName);
return NULL;
}
if (zlfHeader.m_header.compressionMethod == ZipLocalFileHeader::Stored
|| fe.fileSize == 0)
{
// Just read straight from the stream...
ResizeFilterStream *strm = new ResizeFilterStream;
strm->attachStream(&diskStream);
strm->setStreamOffset(diskStream.getPosition (), fe.fileSize);
return strm;
}
else
{
if (zlfHeader.m_header.compressionMethod == ZipLocalFileHeader::Deflated)
{
ZipSubRStream *zipStream = new ZipSubRStream;
zipStream->attachStream(&diskStream);
zipStream->setUncompressedSize(fe.fileSize);
return zipStream;
}
else
{
Con::errorf("'%s' Compressed inappropriately in the zip!",fe.pFileName);
return NULL;
}
}
}
#include "console/console.h"
#include "console/consoleTypes.h"
ConsoleFunction(readZip,bool,2,2,"(filename)")
{
ZipAggregate zipAggregate;
if(!zipAggregate.openAggregate(argv[1]))
return false;
ZipAggregate::iterator itr = zipAggregate.begin();
if(itr != zipAggregate.end())
{
FilterStream* zipfstream = zipAggregate.openStreamForEntry(*itr);
if(!zipfstream)
return false;
U8 *buffer = new U8[zipfstream->getStreamSize()];
zipfstream->read(zipfstream->getStreamSize(),buffer);
delete [] buffer;
delete zipfstream;
}
return true;
}Please let me know if you see anything odd with that code.
Jari.
#5
I also do not understand your new function so I won't comment on that. The ConsoleFunction looks like it should be okay though.
04/21/2006 (6:31 am)
I'm afraid I still can't get this to reproduce in HEAD 1.4. :/I also do not understand your new function so I won't comment on that. The ConsoleFunction looks like it should be okay though.
#6
The new function actually has it's code from ResManager::openStream() .
and the console function calls the new function (openStreamForEntry).
04/21/2006 (7:49 am)
Ok thanks for trying though.The new function actually has it's code from ResManager::openStream() .
and the console function calls the new function (openStreamForEntry).
#7
Thank you!
04/25/2006 (9:16 am)
Would any one have time to test those two functions out with the zip I attached? I really don't know what is wrong.Thank you!
#8
04/28/2006 (9:59 am)
Hey I just got this bug fixed, it was my own functions that didn't work! :)
Torque Owner Stefan Lundmark
I've been into those functions alot and I've never seen that error message before.
Edit: What type of file are you trying to load when you get this error?