Source Control

#1 We are on a multi-member team, all remote, so concurrent access was one of our big requirements, as well as limiting access based on license requirements (only licensed SDK owners get access to the source code, and they have to have all the licenses, etc.), so we had some decently tight restrictions in our repo design, but overall we put just about everything into a repo of some sort:

engine -- source code, with all purchased kits (RTS, Lighting Pack)
client scripts -- we're dedicated server, so client scripts are grouped separately
server scripts -- same as above
art -- we put all of our artwork as it's own repository, and the server and client environments check them out as part of their own checkout using subversion's "external module" capability.
audio -- see above
docs -- design documentation
standalone env -- since we're dedicated server, mission editing isn't enabled by default, so we have a combined repo specifically designed to allow for use of the editors, which brings the client and server script repos back into one environment.

It took us a long time (couple of months, factoring in part-time members, learning SVN, learning TGE, defining our dedicated server -- dedicated client requirements, etc.) to build this set of repos, but we are extremely happy we took the time to do so.

We now have all sorts of neat functionality, from being able to tag milestones across multiple repos and deliver just about any rev across the entire environment to our testers, or other targetted recipients. My lead dev even put together some very nice scripts that allows you to define a few variables (milestone tag, staging directory, etc.) that will checkout all the appropriate repos and build a server env, client env, compile both, package up the server env + dedicated exe into an "install server" package (linux), as well as cross-compile a windows client, and use the bitrock installer to package the client env, art, and exe into a single file for distribution.

We're quite proud of our setup!

#2 Everything created, code, art, audio, documents, is placed in to SCM. I have two projects running concurrently, one uses SVN, the other Perforce. SVN is used simply for historical reasons. Both projects place __everything__ under version control. We use a dedicated server for the repositories. Even a team on 1 running SCM on their local machine can benefit from version control (no more "Oh, I was working on a feature last night and broke the build so I can't show you my demo right now but it's really cool, um, can you wait 30 minutes for me to fix it?")

It can be difficult to get artists to commit (ho! ho! pun!) to using an SCM as many are not used to working with them. They often need quite a bit of hand holding and nudging. It's a bit like bug tracking, if you only accept bug reports through a single bug tracking channel, pretty soon people start making use of it; similarly if you only accept artwork through a single channel, people will start to make use of it. Personal "preferences" in a team environment to not use SCM -- where the protection of the assets are usually more valuable than the protection of the individual team member's ego -- has no place. I am quite zealous in ensuring the team makes use of SCM on a regular basis. SCM is really useful when you are trying to ship multi-platform or multi-version software, i.e. Linux, Mac, Windows, XBOX or "free demo", "GarageGames build", "not for commercial distribution build", "bug test build", etc. (Not that we have a "GarageGames build" yet, we're not sure if GG would even be interested in our game :) )

I'm currently in the process of setting up our automated build system for the second project that will convolve the production assets in to the final assets.

It would have been nice to use AlienBrain but difficult to justify the expense at this stage of the company's existence.

#3 Everything into Subversion

We use something similar to Stephen, just somewhat more simple and divided up into
* engine - TGE, Lightpack
* game - the compiled binary + all scripts. Basically what you would send to a player
* art - all raw art. 3d models in 3ds format, audio files in raw format and similar
* docs - various documentation

#4 Whatever you do, don't use sourcesafe, even though quite a few artists are used to working with it in our industry : the people who say they've never had problems with it, are not the admins of the repository :)

If you have to use a free solution (Perforce doesn't count with the limitation of the demo, unless your project is Open Source, in that case you can apply for a full free Perforce license), choose SVN over CVS for its out of the box support for binary files (eg assets)

And while a SCM repository gives you an inherent backup on top of the versioning, it's no excuse not to back up the repository itself, DAILY...
Even better if you have a daily offsite backup, on top of imaging your repository : that can be done by making a backup onto removable media (and putting it somewhere where a fire, flood that would destroy the live rep wouldn't destroy it, obviously), or have another image of the repository on another team member's computer if you're completely doing remote teamwork.

As others have hinted at, installing the software and starting a repository is a fragment of what SCM entails : you need a firm workflow policy, as if one of your artists or coders doesn't commit regularly (or god forbids, ever ;)), and to boot, don't backup their workbench, SCM is useless. In other words, it's only as good as the weakest link in your team workflow.

As to when you should start to go nuts about SCM and code/assets backup : as soon as you're serious enough to want to sell your game, eg the moment you start putting a dollar figure on your work ;)
Too many people stop at only using SCM for the versioning capabilities, when it should be an inherent part of your project's integrity and workflow.

Hope that helps,
Cheers

#5 @Gary--lots of very strong reasons here (obviously) to use SCM, but I just went back and re-read your original post, and I wanted to comment specifically about using "hard drive space to store binarys" as a reason to not use repos for artwork/binary files:

Hard drive space at this point is the least of your concerns. Hard drives are cheap--time (yours, your team members, your project's) is not. You can always throw in a bit more money, but once you burn time, it's gone forever.

#6 Ahhh - yes. As Nic says - _do not_ use CVS, but use Subversion instead. It stores binaries in an efficient manner using a binary diff, where CVS stores a total copy of the file. But then again, who cares about disc space these days where 160 GB drives are the norm.

Now that we are on that subject - I have only used the free tools for now (CVS, SVN). How does SVN stack up against Perforce in practical use? Any other SCM systems worth using?

And yes - I second the NEVER EVER use sourcesafe. I've had business colleagues that didnt take the warning, and who lost several mandays worth of code due to it (they had to go back into their backups to recover). It is totally unusable even for a 1 man team imho.

I can recommend reading more about the entire development process doing software in books like "Rapid Development" and "Code Complete" by Steve McConnell. Analysis, design, automated nightly builds, proper use of the right development methods, SCM, proper bug tracking etc. will save your time in the long run every time, but most developers only see the coding part of it all.

#7 I have to concur with those people who state "Don't ever use SourceSafe." Personally I've never had any of the major problems that most people report when they tell their horror stories to me. I've shipped PlayStation and PS2 games using Microsoft Visual SourceSafe with all code & assets stored in the database. I've also shipped several large scale games making using of CVS (and other SCM packages) but when it comes right down to it, VSS is just plain atrocious. There is the big three for game development these days, in order of cost: SVN, Perforce and AlienBrain. Any one of those will make you happy you used them. All three store their binary files as a diff with compression so gobbling up HD space at an alarming rate (something that always occurred with VSS) is no longer an issue. I've only just broken 20GB of HD usage in Perforce after importing a 7-yr old VSS database (which consumed over 50GB) and then a further six months of continued usage by the team so HD usage on the RAID 5 array is really not an issue.

As others have also pointed out, installing an SCM is only one part of the workflow procedure, and creating daily backups, also preferably offsite if you have the wherewithal, even if only a DVD-R dropped off at a friend's house, is inherently advantageous.

Just as an aside my backup procedure consists of duplicating the file servers main boot drive in the event of a disastrous crash and a nightly duplicate of the Perforce repository in to the "backup" directory on the same RAID 5 array, which is then backed up to an internal 250GB HD (just in case I forgot plug-in the external drive) which is then duped to an external FireWire HD, with a shadow copy of that drive stored at a remote office. I also don't overwrite the previous backups until space runs out, so that's about once every two weeks. The only thing I don't have it do, yet, is e-mail me that the backup completed, I have to manually check that each day myself. The total hardware cost for this backup procedure, removing the RAID 5 array from the equation, was less than $600 total. I haven't lost a single document, art file or source code file since 1995 in any of the SCM repositories that have been under my control since when I first started using VSS as my SCM solution even through multiple HD crashes. SCM and a decent backup procedure is a no-brainer and people who "lose" source code or demos due to HD crashes are just... well, just dumb!

#8 I just have to chime in here.

Please, please, please, don't use SourceSafe. It's horrible!

I was a full-time SCME (Software Configuration Management Engineer) for several years. I was the person in charge of the SCM repository for 60+ developers with 20+ live threads of code (they had a hard time closing off branches due to customers).

SourceSafe was, and still is, the worst SCM tool around. I'd much rather use SCCS or RCS (precursors to CVS).

While I can't say anything for SVN (haven't used it yet), anything in the CVS class is excellent.

#9 Thanks for the replies, I guess the general consensus is to put everything under scm. I'm probably still going to keep exes/libs or anything compiler generated outside of scm, but I'll start including models and graphics which I havn't done before.

My current backup plans are not quite as riggerous as some of those described here. I tend to just make a complete copy of the source repository + important folders on my workstation to an external backup drive nightly. Then once a week I make a complete backup of the data drive of my machine. Alternating the backup drives weekly. I do sometimes write code off to dvd-r but this is generally only when I don't think a project will be worked on for some time, and just need to archive it off.

I've recently become a fan of external usb2 hdds. Been able to put in a disk the same size as your data drive makes backups a lot simpler :)

I do agree with those of you who say using SCM for even a 1 person team is a good idea. I used to only use SCM for group projects, but over the last few months I've been keeping many of my own personal projects in SCM. To anyone who doesn't, the benefits are certainly well worth the small amount of time it takes to setup a SCM and get things running. Been able to make any changes you want, knowing you can revert if things don't work, or jump back to fix bugs in an older version whilst you still work on your new version, is invaluable.

#10 I just recently dumped doing physical backups for my code, and switched to a Internet based one (www.ibackup.com). In their pure storage version its $99/year for 5 GB and $199/year for 10 GB. Now I'm absolutely sure that backup is run every day and transported offsite, and it makes me sleep safe.

DAT, CDR and DVD-R are all fine and nifty (and cheaper too), but its a hassle + you tend to forget to do it + you need to transport them to somewhere out of your house.

#11 I've used SourceSafe, Perforce, and Subversion.

SourceSafe will eat alot of disk space and it can be a little flakey at times, but we had no horror stories for a 50GB+ database. Still we always had nightly backups. The major benefit of SourceSafe is ease of use, great IDE integration, and file linking within the database.

Perforce is my favorite of the bunch. It's fast, keeps a small database size, and pretty easy to use. You even have great IDE integration. The only real down side of Perforce is the cost. While cheaper than some solutions at $700 a seat it's not viable for most indies. There is however a free 2 seat license, but we outgrew that quickly.

Subversion is what we currently use. We use it because it combines the basic feature set of Perforce with a free price tag. It's not as easy to use as Perforce is even with the Tortoise shell extension and even after several months of use I still run into trouble getting some things done. Still it's solid and pretty fast.

I recommend Subversion, but if you have the cash to burn Perforce is better IMO.

#12 If Perforce do not add a wealth of features in the near future then I think SVN/Tortoise/Ankh will eclipse them within two years. The only thing Perforce will then offer is paid for support. Which for $700/seat I'm sure you could also purchase for SVN. I'm certainly keeping an eye on the SVN/Tortoise/Ankh trio and hope that it becomes ubiquitous and also as useful and feature rich as NXN AlienBrain.

I like SCM for all of my work because it allows me, like someone else stated, to try something out and if I break it, no big deal, just revert to an earlier version. I even use it for any articles I write for magazines. Editors are amazed I'm so willing to make the exact changes they request without worrying about losing earlier versions. I also teach various subjects dealing with video game programming at college and I always introduce my students to an SCM, usually Perforce or VSS and make sure we use proper version control over the course of the semester as this is something that they are ultimately going to use in their careers.

I've considered solutions like ibackup.com but have never had the oppurtunity to really put them through the wringer. Perhaps when I have time in the new year I should think of doing a round-up review of the various service providers.

#13 @Thomas

You really trust them that much with your source code?

I'm not talking about trust as in they might steal it but that all your work is down the drain if they slip up. What's your repercussion if they do? All the clauses in a contract are meaningless if they actually loose your work.

I've done a lot with Disaster Recovery for major corporations and they usually don't even trust DR firms 100% and always maintain their own backups as a "just in case".

#14 @Jeff

I'm not saying I wont make physical backups anymore. Just that the day to day backup is fully automated and online. Before that it was like "oh - lets wait another week until I made some real changes" before taking a backup. Once in a while I'll do DVD backups and have them handy. Just in case ;-)

#15 Great thread - thanks everyone. Have been struggling with CVS for ages thinking there must be a better way and there is ! Have downloaded perforce and think its great - exactly what I need. What a great start to the year !

#16 I'd be happy enough with CVS if I could figure out how to make it switch from pointing to the GG repository for HEAD releases, and to my own local repository for code changes.

Meantime, I'm evaluating Evolution (free single-user license), TortoiseCVS, CVSNT, CS-RCS and I also had a look at SourceJammer. Evolution seems powerful, once you set it up. Having read this, I'm going to have a look at Subversion SCM as well. I'm still in the evaluation phase, but when I know more, I'll update this thread.

#17 So Stephen, Thomas, and Tom you all use Subversion ?

Subversion = open source ?

#18 Subversion is open source and i use it, but not for that reason in particular. See the website for more details.

I use the TortoiseSVN windows shell extension client to access repository.

#19 Yes, and for anecdotal reference, GG switched from CVS to SVN for internal source management as well. I can't speak for how successful it's been for them, but I know that I've used both, and now that I've used SVN I'd not want to go back.

It's a very nice, and open source system.

#20 I'm using Subversion and TortoiseSVN for my minigolf project. It has worked well and I will be using it on all future games.