Game Development Community

storing binary data in a basic_string

by orion · in Technical Issues · 03/24/2003 (4:30 pm) · 8 replies

Hi all, I wonder if someone can help me.

How do you load a binary file into an stl string without having all the problems with null bytes i.e. binary zeros. When one of thos encountered the string is terminated.

I think that can be avoided by using ifstream.get() and replacing each null byte with something else or escaping it or something. But if the binary file is of a picture or something, then the picture will have artefacts when being displayed. This method also seems like it might be slow.

I was wondering if there is a better way, or just a way that works.

cheers

About the author

Recent Threads


#1
03/24/2003 (4:47 pm)
Don't use a string for non-string data. If you want non-string behavior just use vector. After all string is a specialized version of a vector.

You're better off not using any STL containers for holding image data though.
#2
03/25/2003 (4:57 am)
How should I store an image or other binary data in memory? should I use a char array?

wouldn't I have to keep resizing it when loading the data from the ifstream until the entire file is loaded? or is there a way to load the whole file in one go?

cheers
#3
03/25/2003 (12:32 pm)
Heres something.

I don't think it is possible to get the file size using the fstream. so I use the stdio FILE to get the file size. But then I feel more comfortable with STL so, I use fstream to get the binary data from the file.

The code gives you the option to load the data using get() or read(). (not at runtime!)

#include <iostream>
#include <fstream>
#include <string>
#include <vector>

#include <stdio.h>
#include <io.h>

using namespace std;

int main(int argc, char* argv[])
{
	int byteCount = 0;
	int data = 0;
	char *charStar;


// First get the filesize using C file handling.
//-------
	// Open somefile.bin for reading in binary mode
	FILE* fileToSize = fopen("i.bin", "rb");

	if( fileToSize == NULL )
	{
		cerr << "Error opening file\n";
		return 1;
	}

	// Get the file's size
	long fileSize = filelength(fileno(fileToSize));

	fclose(fileToSize);
//--------

// Now read the actual file.
	ifstream inFile("i.bin", ios::binary | ios::in);
	if (!inFile)
	{
		cerr << "\nCould not open input File.\n";
		return 1;
	}

	charStar = new char[fileSize];

// use this if you want to read the file a byte at a time using get()
#ifdef ONE_BYTE_AT_A_TIME

	char myChar;
	while (inFile)
	{
		inFile.get(myChar);
		if (inFile)
		{
			charStar[byteCount] = myChar;
			byteCount++;
		}
	}

// or use this single line if you want it all!
#else

	inFile.read(charStar, fileSize);

#endif

	inFile.close();

// -- done reading the file


// lets make a copy of the file

	ofstream outFile("o.bin", ios::binary | ios::out);
	if (!outFile)
	{
		cerr << "\nCould not open input File.\n";
		return 1;
	}

	outFile.write(charStar, fileSize);
	outFile.close();

	delete[] charStar;

	return 0;
}

If some one has better ideas, please let me know. I'm kind interested in this too at this point of time.
#4
04/05/2003 (1:55 pm)
Thanks. That lets me load the file correctly.

With your code, sizeof doesn't always return the right size. Can I make a class like string that will store everything, even binary zeros like above and still give me the right size and let me pass the image as parameters?

Or is something like that already available.
#5
04/07/2003 (2:20 pm)
class DataBlock {
   BYTE* buf;
   int   len;

public:
   DataBlock() : buf(0), len(0) { }
   ~DataBlock() { delete [] buf; }

   BYTE* getData() { return buf; }
   int   getSize() { return len; }

   void load(const char* filename) {
      delete [] buf;
      buf = 0;
      len = 0;

      FILE* f = fopen(filename, "rb");
      if (f) {
         fseek(f, 0, SEEK_END);
         len = ftell(f);
         buf = new BYTE[len];

         fseek(f, 0, SEEK_SET);
         fread(buf, len, 1, f);
         fclose(f);
      }
   }
};
#6
11/14/2007 (8:14 pm)
Hi,

how should I store data in torque x, should I use SQL or Xml?And what is the method of storing data?
#7
11/15/2007 (7:35 am)
I would recommend XML since you will have to use a secondary .Net framework call to access a SQL database (which breaks XBox 360 compatibility with XNA). For the PC, it could definitely work with the large number of .NET SQL interfaces out there.

EDIT:
I also thought I would note that the last post was from 2003, well before TorqueX and XNA was even a glimmer in the eyes of the development staff.
#8
11/15/2007 (8:18 am)
SQL or XML? They're nothing alike.

I can't help but comment on the dead part of this thread, uuencode binary data into a string. Basically, take six bits of binary data, add 0x32 and store it as an eight-bit character. This method guarantees that the string will be valid ASCII: no nulls, no non-printable characters, no accented characters. To uudencode the string, take the eight bits, subtract 0x32 and copy the six bits to the stream. It works very well. I think this is how email handles binary attachments.