Previous Blog Next Blog
Prev/Next Blog
by date

Download sizes...

Download sizes...
Name:Jon Frisby
Date Posted:Apr 06, 2006
Rating:Not Rated
Public:YES
Comments:YES
RSS Feed:GarageGames Blog feedor Subscribe with .
Profile Page:View profile page for Jon Frisby

Blog post
I had been laboring under the assumption that a 20-40MB download would be fine, but after my last blog entry someone noted that 10MB is the "magic number" after which downloads fall off.

All right then, what do I need to do to get my download under 10MB? Let's look at the components of my game from biggest to smallest. Remember, I'm working on MacOS X here, so the numbers are likely to be somewhat... different... on Windows:

Torque Game Builder.app/ -- 20MB
T2D/data/music/ -- 5.4MB
T2D/data/images/ -- 3MB
T2D/data/movies/ -- 1.2MB
common/ -- 596KB
T2D/data/particles/ -- 440KB
T2D/{gameScripts,gui}/ -- 316KB

Pretty scary, even for uncompressed sizes! That's a non-debug, Universal Binary build of my game with graphics for 7 skins, (very bad) placeholder music for 2 skin, and a video background for 1 skin. So how do things break down when they're actually compressed?

Torque Game Builder.app.tgz -- 5.2MB
T2D/data/music.tgz -- 5.1MB
T2D/data/images.tgz -- 2.6MB
T2D/data/movies.tgz -- 1.1MB
common.tgz -- 124KB
code.tgz (gameScripts and gui) -- 71KB
T2D/data/particles.tgz -- 16KB

Ok, so this clearly isn't workable. 10MB with just the app and a fraction of the music I'm going to need? Not good. The script code and particles are clearly not worth bothering with, but everything else needs to go on a diet.

So where are my easy wins? First, let's take a look at that overweight binary! A MacOS X ".app" "file" is actually a directory structure that contains a binary, any embedded frameworks (the analogue of DLLs), and any other resources that are needed.

Digging inside we find that the actual binary itself is 9MB, the embedded frameworks are 11MB, and everything else is negligible (40KB). Recently, someone in the IRC channel refreshed my memory about the strip utility. I've never had a REAL need to use this tool until now, but I had previously played with it in Linux. For those who don't know, strip is a tool that removes symbol information from a binary. After stripping, the core binary is now down to 6MB. Not *too* bad given that it has 2 copies of the code (PowerPC and Intel).

On to the frameworks. Frameworks for the record, are structured just like apps -- they are directories with a binary and any other frameworks/resources needed. Actually, it's a little more complex than that: A framework can contain multiple VERSIONS of the binary in order to preserve application compatibility in the face of upgrades. So which frameworks do we have? Ogg.framework (216KB), OpenAL.framework (1.3M), Theora.framework (1.5MB), and Vorbis.framework (7.6MB)! Ack! So let's see what we can do with these...

The way a Framework is laid out is usually like this:

X.framework/
X.framework/X (symlink to Versions/Current/X)
X.framework/Headers/
X.framework/Headers/*.h
X.framework/Resources/
X.framework/Resources/* (localization-related resources)
X.framework/Versions/
X.framework/Versions/Current (symlink to the most current version, for example, A)
X.framework/Versions/A/ (version A)
X.framework/Versions/A/X (actual binary)
...

So digging in I find that Vorbis.framework/Vorbis is an actual *copy* of the binary, and X.framework/Versions/Current is an actual *copy* of X.framework/Versions/A! THREE COPIES of the binary, which happens to be 2.5MB in size! One of these appears to be a side-effect of me not properly using rsync in my build process but the other one is there in the direct build product from the Xcode build. Yech!

So going through and fixing symlinks and stripping binaries from the various frameworks shaves it down from 11MB to 3.4MB. This leaves us with a .app that takes only 9.9MB -- quite slim compared to the original but still pretty big. The real question though is how big is it *compressed*? Answer: 3.1MB, a 40% reduction. Of course that still only leaves me with 6.9MB for code and content. But does the game still WORK after all this stripping and symlinking?

NO!

Seems I was too aggressive, and I broke things. Turns out that I shouldn't have stripped the binaries in the frameworks -- but fixing the copies vs. symlinks problems works fine. So my compressed binary is now 3.2MB instead of 3.1MB leaving 6.8MB for content.

So now the question becomes: How do I make the most of that 6.8MB and pack as much content in as possible?

The first candidate is the music. Music for a skin consists of: 1) Primary audio track, 2) "Danger" audio track, 3) 8 separate "sound effects" (which are actually musical components -- chords, guitar riffs, vocals, whatever). My placeholder music is somewhat less complete: No danger audio tracks, and one of the skins is missing 5 of the effects. That spells bad news for when I have real music! The obvious choice here is to reduce the audio quality somewhat. A quick experiment shows I can shave 30% off the size of the files and not hear the difference. I imagine I will spend a fair amount of time optimizing audio files before release.

Next up: Images. There are a couple easy wins here: A few of my backgrounds are good candidates for being converted to JPG (currently all art assets are PNG) which will let me shave about a megabyte off the download size with a negligible decrease in image quality.

So now my download is down to ~9.7MB but I still have a lot more skins to add, and a TON more music... So I have a long way to go. I'm going to be playing with video compression, and music compression to find the right balanace, and I could probably still shave the .app a couple more MB by stripping out code that I don't need -- shape3d, level editor, and so forth. Either way, I'm thinking my demo build won't be including every skin...

-JF

Recent Blog Posts
List:04/13/08 - Hordes of Orcs for Windows, Harmonic Convergence, and Revenge!
02/20/08 - I CAN HAZ GAME BALANCE?
02/11/08 - 240 days, and counting!
12/15/07 - My milkshake brings all the orcs to the yard...
09/20/07 - "Gentlemen, I'm goin' home in my new car."
09/17/07 - OH GOD!!!! THE ORCS ARE INVADING!!!!!!
09/11/07 - 5 days, and counting... (Or: We Likes Numbers!)
09/06/07 - When Orcs Attack!

Submit ResourceSubmit your own resources!

Phil Carlisle   (Apr 06, 2006 at 08:03 GMT)
Ok, here are some things that have worked in the past from memory:

1) Use a different music format. Basically .ogg files will still be big. If youre serious about this limit, then a tracker based format is a good way to go. But it requires totally different skills from your composer.

2) Use pngcrush (i think its called that) on your PNG files might be better than the jpg option

3) Use less..

4) Is the limit of 10mb for the demo or the full game?

Lee-Orr Orbach   (Apr 06, 2006 at 08:19 GMT)
why dont you have one/two skins "built-in" to the game, and add more for download seperatly? That will descrease the download size, and allow people who "don't like that skin" to waste less diskspace on it.

Mike Fosker   (Apr 06, 2006 at 08:38 GMT)
How about having the absolute bare minimum in your main game, and include a "Check for Updates" button that allows users to check for additional skins, music etc. They end up downloading the same amount of data, but maybe they won't realise :)

Jon Frisby   (Apr 06, 2006 at 08:46 GMT)
@Phil: A tracker format is interesting but adds technological overhead (T2D doesn't support it out of the box, so I have to add it -- I don't have the expertise to do that myself so that will involve contract programming work which adds cost...) and might prove limiting (no vocals and such). I'll definitely consider it though if I can't squeeze the Ogg files into shape, and/or for skins where the relevant composer doesn't go beyond what's capable in such a format. I'll also grab pngcrush and see how it does. As for using less: Lumines (the game I'm basically cloning) is so addictive precisely because it's so incredibly lush in terms of the audio-visual experience. Most of the skins have movie clips for backgrounds, many songs have vocals, and so forth. In short, the experience is the point of the exercise, so I don't really have the luxury of "less". Fortunately the 10MB limit is for the demo, so I probably won't be including all the skins in the demo.

@Lee: I'm following the Lumines model where you unlock skins through successful gameplay. Not the Winamp model of pick a skin you like and download it. :)

I'll probably aim for a target of 5 skins for the demo (I'm aiming for 15-20 for the full game), possibly at somewhat reduced quality and use the idea of higher quality audio and more skins to unlock as a way to encourage people to buy the full game.

-JF

Jon Frisby   (Apr 06, 2006 at 08:47 GMT)
@Mike: Funny you should mention that... The thought HAD occurred to me, but I eventually decided I wasn't Evil enough to do that... :p

-JF

Stephan (viKKing) Bondier   (Apr 06, 2006 at 08:57 GMT)
John,

Where is the link, so I can pick up your file and give it a "positive" look?

Jon Frisby   (Apr 06, 2006 at 09:07 GMT)
@Phil: Just tried "PNGCrusher" (A Mac app -- I'll try pngcrush later when I have the time and energy to try compiling it on OSX) and it shaved 330KB off the .PNGs, so I'm definitely adding this tool to the pipeline!

@Stephan: You mean a demo release? Not until I have one or two more-or-less-complete skins and a Windows build... :)

-JF
Edited on Apr 06, 2006 09:18 GMT

Stephan (viKKing) Bondier   (Apr 06, 2006 at 09:13 GMT)
JF,

No, that could even be a beta ;-) if you need it.
Though I'm still on 10.3.9.

STef

BigPapa   (Apr 06, 2006 at 12:43 GMT)
@Jon - I think 4 skins would still be good for the demo. Three would be too little.

Rubes   (Apr 06, 2006 at 15:40 GMT)
Jon, this is great information for us Mac developers. It would be really nice to see this all as a resource or something when you're finished.

Andy Schatz   (Apr 06, 2006 at 16:39 GMT)
This all partly depends on your distribution goals. If this is a "portal game" that you intend to get out on the big portals, under 10MB is a good goal. If it's a shooter or a strategy game or something not portal-y, under 20MB is the magic number.

Jon Frisby   (Apr 06, 2006 at 16:41 GMT)
@Rubes: I might throw together a "cleanup app" that goes through and scrubs .app files using the same methodology that I describe above.

@Andy: It's basically a puzzle game, so portal distribution would be good...

-JF

Luke D   (Apr 06, 2006 at 18:07 GMT)
You might want to look at using 8-bit (w/ alpha) PNGs on some graphic elements This can reduce the file size around 50%. This only works well for graphics that can tolerate being limited to 256 colors, though you'd be surprised how good things can look, provided they don't have a full rainbow palette in use or a lot of gradient alpha.

Photoshop's PNG support is weak all the way up through and including CS2, so I use pngquant, a command-line program to make the conversion. I basically converted all PNGs to 8-bit, then reverted any I thought looked visibly different from the original. More than 80% of my art was almost indistinguishable and I went from 23MB to 16MB from that change alone.

Regarding portal game sizes, smaller will always be better, and 10MB was definitely a widely-quoted barrier for years, but with the huge adoption of broadband and the increasing production standard of titles the trends have started to inch upward. Most modern portal releases are 12-18MB in size, and a few are even breaching the 20MB mark. That doesn't mean a 10MB game won't do better than a 15MB game, but the emphasis on the size doesn't have to be THAT strong. Don't over-sacrifice quality to hit it.

Joe Rossi   (Apr 06, 2006 at 19:23 GMT)
Most vocal sound fx and maybe some others will do fine as Mono and they'll be much smaller. You should use LZMA compression (7-zip) it's usually better than the other algorithms. Use as many uncompressed files as you can, because recompression is bad. A lot of people still don't know this. But when you zip the final distributable with your installer, that's what happens. As a simple experiment try zipping some of those pngcrushed png's, and then try the same png's uncompressed. Compare them and you should see the uncompressed data is way smaller in your archive. Hope this helps :)

Tom Bampton   (Apr 06, 2006 at 19:32 GMT)
One small note on tracker formats ...

They are just a collection of samples and info on when/how to play them back. The samples can be anything. Now granted, they're not that good if what you want is something like a normal song that's mostly vocals. However, it doesnt exclude vocals entirely.

The other thing is they've been used in games since pretty much when they were invented many moons ago. This means there are a plethora of good freely available libs to play them, and it would not be too horrendous to integrate with Torque.

Of course, perhaps the best solution would be to use a mix of ogg and trackers. Since your audio is all AudioProfile driven, it wouldn't matter to the actual game code what format it was in ... it would all play through the same script code from the game's point of view.

If you have access to composers that can handle making something in a tracker format, I think it's worth seriously considering that option.

T.

Jon Frisby   (Apr 06, 2006 at 20:34 GMT)
@Luke: Actually, that's a good idea! Many of my sprites are mono/bi-chromatic so there's probably a lot of opportunity to reduce file sizes there...

@Joe: On the Mac side the compression format is dictated by the distribution format: The expectation is a .dmg file, possibly compressed with something that StuffIt Expander can handle (although .dmg files DO support compression of their contents themselves but they don't tend to do as well as just compressing the .dmg file itself). On the Windows side I have some leeway of course as self-extracting binary installers are commonplace. As for compressing the .png files before/after crushing them:
Raw (uncrushed images): 3.0MB
Crushed images: 2.8MB
Gzipped .tar of raw images: 2.6MB
Gzipped .tar of crushed images: 2.3MB

Oddly crushing the images IMPROVES the compression gzip is able to achieve. Bizarre, and highly unusual I know. One possibility for why this is so might be that there's a lot of "garbage" in the RGB channels of entirely transparent (not translucent) pixels and the crush utility is "cleaning" those up so that the RGB values are all stripped to 0. This would have the effect of making the existing data more compressable without changing the file size directly. This is all speculation of course. All I care about is that my 3.0MB of images eventually results in a 2.3MB zip file with no reduction in image quality. Judicious application of JPGs will likely push that down even further with only negligible differences in quality...

@Tom: Yeah, I'm familiar with tracker formats -- used to collect .mod files back in the BBS days. :) My biggest concern is that I don't want to tie the hands of the composers working on this project. There's just a ton of stuff you can't do, or can't do well with tracker formats -- generative sounds (from synthesizers), post-processing/mastering effects (echo, reverb, distortion, vocoding, any sort of feedback loop, etc.). Tracker formats are really only good if you're working with discrete notes, like from pre-electronic instruments. I'm still considering it if I can't find a way to get acceptable quality with acceptable file sizes from the .ogg files, but it's a really major imposition upon the creative process and workflow of the composers so I kind of consider it a route of last resort.

I remain cautiously optimistic that simply using a lower bitrate for the .ogg files will be adequate without impairing quality enough to be a problem. Besides, there are other possibilities if I'm willing to invest in technological improvements. Most notably, Wikipedia notes that mp3PRO seems to produce the best results for low bitrates (64kbps) so using other codecs might open the door to lower bitrates.

-JF

Frogger   (Apr 07, 2006 at 23:45 GMT)
A program that usually cuts apps by a 1/3 is UPX. I've used it with Torque and there is no performance difference either. I'm no compression expert but it says on the website that it uses NRV.


Good Luck!

Jon Frisby   (Apr 08, 2006 at 00:18 GMT)
That's a great resource for my Windows build, when I actually have one. I'd completely forgotten about these sorts of programs, but what they basically do is turn your app into a compressed, self-extracting executable that extracts your program into memory instead of to disk, and then executes it. It's seamless and works great on any program that doesn't depend on looking at it's own executable file.

On the Mac side, I did a little digging and found that it comes with a utility called "gzexe" that uses a very low-tech method of doing something similar, resulting in my current binary being reduced from 9.2MB to 2.9MB. Unfortunately this is the same algorithm I will likely be using to compress my package for distribution, so doing it "in advance" would offer no benefits, and the low-tech methodology employed by gzexe could cause headaches for users so on the Mac front. A quick search of VersionTracker shows nothing like UPX for Mac so nothing to help me on that front.

-JF

Tom Bampton   (Apr 08, 2006 at 00:42 GMT)
My experiences with UPX are pretty much the same as what you found with gzexe. Executable compression is only worth using if you're worried about the final installation size; which end users tend not to care too much about. In terms of optimising download size it pretty much gains you a big fat nothing.

T.

You must be a member and be logged in to either append comments or rate this resource.