Balancing Iterative Prototyping and Formal Design
by Greg Baker · 03/20/2007 (10:48 am) · 13 comments
As a software engineer, I approach software design from a UML first perspective. For those not familiar with UML, it's simply a structured method of designing software on paper before writing any code. The only exception is in cases where a particular procedure may require an initial proof of concept because it's either a novel approach to a problem or an as yet unproven method. I have found that time is actually gained, not lost, in using several weeks or even months to properly design a software solution on paper before writing code. There are a host of issues that can be addressed well before any code is written. You also gain a very clear and detailed view of what needs to be coded which can make implementation much faster in the long run.
From all the advice I've been reading there is a consistent recommendation to use an iterative prototyping method to developing a game. The primary reason behind iterative prototyping is to determine if a game idea is going to be fun or not before months of development time are wasted on what turns out to be a boring or dull game. From my experience, though it is limited in respect to game development, I would expect that once a game prototype has been verified as "fun" it would be good practice to scrap the prototype, salvage any units of code that my prove useful later, and start a fresh project driven by a formal design document. This design document should be a living document, however, and over the life-time of the project will change and evolve. In practice the design and implementation of the game, or any software project imo, should always follow an iterative procedure. The key difference though is that the initial prototype does NOT become the basis for our final game implementation. The prototype should be used to test and verify our initial game concept and once that's been accomplished it should be scrapped. I've never seen good things result from prototyping till release. The code becomes unwieldy and difficult to maintain. Even as a write this, I can see the problem with my current perspective. Games are not traditional software applications. They generally do not have a long shelf life, don't require long-term maintenance or enhancements, and therefore lend themselves to prototyping until release.
However, I wish to ask those who've been down this road before and actually released a game, "What has your experience been, and at what point does your prototyping end and your design document begin, if ever?"
From all the advice I've been reading there is a consistent recommendation to use an iterative prototyping method to developing a game. The primary reason behind iterative prototyping is to determine if a game idea is going to be fun or not before months of development time are wasted on what turns out to be a boring or dull game. From my experience, though it is limited in respect to game development, I would expect that once a game prototype has been verified as "fun" it would be good practice to scrap the prototype, salvage any units of code that my prove useful later, and start a fresh project driven by a formal design document. This design document should be a living document, however, and over the life-time of the project will change and evolve. In practice the design and implementation of the game, or any software project imo, should always follow an iterative procedure. The key difference though is that the initial prototype does NOT become the basis for our final game implementation. The prototype should be used to test and verify our initial game concept and once that's been accomplished it should be scrapped. I've never seen good things result from prototyping till release. The code becomes unwieldy and difficult to maintain. Even as a write this, I can see the problem with my current perspective. Games are not traditional software applications. They generally do not have a long shelf life, don't require long-term maintenance or enhancements, and therefore lend themselves to prototyping until release.
However, I wish to ask those who've been down this road before and actually released a game, "What has your experience been, and at what point does your prototyping end and your design document begin, if ever?"
#2
Make sure you have design intervals that includes A LOT of playtesting from people who has never seen the code or the game. It's incredible how warped your perspective becomes when you have seen something 800 times. If you can't get it on the screen because you haven't written it, try to see if you can convey the concept in other ways to get the data you need to make a decision. Maybe you could use storyboards, gui mock-ups, or even pictures from other games.
Rarely make assumptions about what someone will percieve, unless it's something you feel you have proven before in the past.
03/20/2007 (1:13 pm)
From my experience, too much of a stress is put on paper design too early. If you write down too many things too early, with too much detail, chances are you will end up scraping a lot of your work. I think paper is mostly to aid in communication, not design. I think you need make sure your early iterations are small as possible. If you try to make "design" only one interation, you are going to waste a lot of time.Make sure you have design intervals that includes A LOT of playtesting from people who has never seen the code or the game. It's incredible how warped your perspective becomes when you have seen something 800 times. If you can't get it on the screen because you haven't written it, try to see if you can convey the concept in other ways to get the data you need to make a decision. Maybe you could use storyboards, gui mock-ups, or even pictures from other games.
Rarely make assumptions about what someone will percieve, unless it's something you feel you have proven before in the past.
#3
03/20/2007 (1:24 pm)
The other thing about programming a game in Torque is that you're building on a solid foundation which is mostly very well designed. Even if your own designs are not very well engineered, 90% of your game will still be (and probably much higher than that in the critical lower-level sections).
#4
My team at Last Straw spent a good eight months ironing out gameplay and design on paper before touching a line of code. We shelved the project for a few other ones, but when it gets unshelved, we'll have most of the game design already done on paper.
Having said that, I built a game in 4 days where 3 days into it we changed the entire premise and story of the game and a good chunk of the gameplay, all with spur of the moment decisions (which ended up saving the game in the end).
03/20/2007 (1:40 pm)
I don't think I'd agree with the 'the engine is good, so your code can suck and it still wont be that bad' mentality. The engine could be the best out there, but your game logic could bring the entire game to it's knees, engine and all. Good design is a must if you want to make it in this business.My team at Last Straw spent a good eight months ironing out gameplay and design on paper before touching a line of code. We shelved the project for a few other ones, but when it gets unshelved, we'll have most of the game design already done on paper.
Having said that, I built a game in 4 days where 3 days into it we changed the entire premise and story of the game and a good chunk of the gameplay, all with spur of the moment decisions (which ended up saving the game in the end).
#5
I think that its just too easy to say "this is how it should work" when in reality, I dont think there is ONE way of making a project work, basically, its a risky business and you'll never take the risk out of it. Much like movie production can go on brilliantly, but still you get a bad movie at the end. Creativity isnt engineered!
03/20/2007 (2:26 pm)
My experience has been that its usually best to prototype any key features where you have unknowns. That can be either technology or gameplay related features. Even on games you know very well (we were writing sequels to our hit game) it can help to write some throwaway prototype code. We had a good 6-9 months of simple prototyping code on one of our projects. It gave us fully destructable 3D terrain, AI based camera control, player view management and some other techy stuff.I think that its just too easy to say "this is how it should work" when in reality, I dont think there is ONE way of making a project work, basically, its a risky business and you'll never take the risk out of it. Much like movie production can go on brilliantly, but still you get a bad movie at the end. Creativity isnt engineered!
#6
03/20/2007 (5:18 pm)
@ Jonathon: that's not what I said. You're taking my statement and inferring something totally out of proportion.
#7
The part above about the game not necessarily requiring a perfect code equation (not meant for years and years of shelf life) sounds right to me. It is a balance of resources, time, and quality for a stable and fun outcome. A licensed source is different.
In my world of work over-planning can paralyze a project for weeks that would otherwise be done to 80% in a few hours. Some of my best work is prototypes that have worked well for two years. Crack them open again for maintenance? Heh, that's painful. Easier to start over.
So what is success? Zen of the perfect code? There is something seriously cool about a hotrod elegant piece of well designed code. Even if I don't (or never will) have the skills to do anything but hack at it until it is broken:)
I'm an art snob, so I get how easy it is to look down the long crooked nose at those less worthy in the competency area:) But I also understand there's a time for perfection and a time for getting the job done right enough. Planning = necessary, so is doing.
In game design, don't the concept and vision lead? Doesn't the code follow the research (someone else has done it?) requirements definition, and prototyping to see if the concept and vision will (or can) work, that the team can actually pull it off, and that it's actually worth it?
03/20/2007 (7:28 pm)
I like Phil's idea. Prototype the biggest unknowns. I think this is one of the few effective ways of hammering out details so you CAN actually draw a map to the promised land. I'm no programmer, don't pretend to be really - and it's tough to relate to some of the young whippersnappers (I'm 34... did I just say whippersnappers?) that design is good, but you can't get there simply by willing textbook processes all of the time:) Making sure you have a handle on the early parts of the trailhead helps to narrow the focus early on, and prove to the management folk that a risky endeavor is that much less risky.The part above about the game not necessarily requiring a perfect code equation (not meant for years and years of shelf life) sounds right to me. It is a balance of resources, time, and quality for a stable and fun outcome. A licensed source is different.
In my world of work over-planning can paralyze a project for weeks that would otherwise be done to 80% in a few hours. Some of my best work is prototypes that have worked well for two years. Crack them open again for maintenance? Heh, that's painful. Easier to start over.
So what is success? Zen of the perfect code? There is something seriously cool about a hotrod elegant piece of well designed code. Even if I don't (or never will) have the skills to do anything but hack at it until it is broken:)
I'm an art snob, so I get how easy it is to look down the long crooked nose at those less worthy in the competency area:) But I also understand there's a time for perfection and a time for getting the job done right enough. Planning = necessary, so is doing.
In game design, don't the concept and vision lead? Doesn't the code follow the research (someone else has done it?) requirements definition, and prototyping to see if the concept and vision will (or can) work, that the team can actually pull it off, and that it's actually worth it?
#8
Due to the infinitely small time frame we had to complete our game demo, we would code a concept out to see how it worked. If it worked, we gold plated it and left it for the next task. What did this mean for us? We had working functionality, but the code got pretty bloated at times and had to be refactored more times than I can count. The demo was a success, at least, but I haven't had the heart to go back and fix the problems.
Cycle forward to a recent event. My company needed to show a proof of concept to the designers/producers of the game. I prototyped the game play and two levels, showed the game, got approval, and scrapped the whole project. I kept the necessary code I knew I could use, and now a week later I'm four times further than I was before with cleaner code and a better working knowledge of the engine.
03/21/2007 (5:56 am)
During my Bach. Final Project at Full Sail, our team did not use a lot of design patterns terminology until the very end, when we realized some of the stuff we had been doing unknowingly. For instance, we didn't realize we were using a SCRUM system, we thought we were just doing what we wanted opposed to what the instructors wanted us to do. Worked well for us. I just realized we used the iterative prototyping process as well. Due to the infinitely small time frame we had to complete our game demo, we would code a concept out to see how it worked. If it worked, we gold plated it and left it for the next task. What did this mean for us? We had working functionality, but the code got pretty bloated at times and had to be refactored more times than I can count. The demo was a success, at least, but I haven't had the heart to go back and fix the problems.
Cycle forward to a recent event. My company needed to show a proof of concept to the designers/producers of the game. I prototyped the game play and two levels, showed the game, got approval, and scrapped the whole project. I kept the necessary code I knew I could use, and now a week later I'm four times further than I was before with cleaner code and a better working knowledge of the engine.
#9
1) Initial Game concept that answers Who, What, Where, and Why?
2) List of game features
3) Prototype game based on concept and initial feature list
4) Draft formal design document
5) Storyboard levels and any mini/main boss encounters
6) Implement game
7) Repeat 1-6 throughout the development life-cycle
8) Ship it!
Storyboarding is something new for me that I haven't had to do before in other industries. It's a lot of fun though. Once the prototype has proven a particular game feature works the way we want it to, storyboarding lets us really be creative laying out levels. In the process it helps us refine game-play elements without having to write a single line of code unless the refined game-play feature has changed significantly and needs to be prototyped again.
03/21/2007 (6:40 am)
My design process is evolving into a multi-step process.1) Initial Game concept that answers Who, What, Where, and Why?
2) List of game features
3) Prototype game based on concept and initial feature list
4) Draft formal design document
5) Storyboard levels and any mini/main boss encounters
6) Implement game
7) Repeat 1-6 throughout the development life-cycle
8) Ship it!
Storyboarding is something new for me that I haven't had to do before in other industries. It's a lot of fun though. Once the prototype has proven a particular game feature works the way we want it to, storyboarding lets us really be creative laying out levels. In the process it helps us refine game-play elements without having to write a single line of code unless the refined game-play feature has changed significantly and needs to be prototyped again.
#10
Try this exercise:
If you are working with an existing team, they should be involved at every stage of this exercise.
1: Pick two game ideas of roughly equal estimated amount of development time (do no design work to estimate the time, just make an educated guess). The amount of time the ideas should take to implement is roughly a week to a month, depending on how much time you can spare. If you can't spare a week, then a weekend will do.
It is worth noting that shorter time spans will provide "unfair" results in favor of rapid prototyping. Longer time spans will provide "unfair" results in favor of traditional design. The sweet spot is probably something like a week or two.
2: Make one of the games using your traditional approach
3: Make the other game using the rapid prototyping approach. I would suggest attempting to throw away all your preconceived notions and keep an open mind when you try this. When you are used to one development methology, switching to another without the old one getting in the way can be kinda tricky.
(The order of 2 and 3 probably doesn't matter)
4: Compare the results. The questions you should be asking yourself for each approach are something along the lines of: How complete is the game ? If I were to take only what I have now and spend X amount of time polishing what I have, but writing no new gameplay related code, would the result be of shippable quality ? You can probably think of more questions.
The aim of the exercise is to guage the suitability of each methology as fairly as is reasonable given appropriate constraints. You will learn through first hand experience the aspects of each methology that suits you best. You can then pick and chose which to keep and which to dump.
You can repeat the experiment to try out additional methologies. Yes, this is rapid prototyping for development methologies ;)
T.
03/21/2007 (12:45 pm)
The thing about development methologies is what works for one thing/person does not neccessarily work for another thing/person.Try this exercise:
If you are working with an existing team, they should be involved at every stage of this exercise.
1: Pick two game ideas of roughly equal estimated amount of development time (do no design work to estimate the time, just make an educated guess). The amount of time the ideas should take to implement is roughly a week to a month, depending on how much time you can spare. If you can't spare a week, then a weekend will do.
It is worth noting that shorter time spans will provide "unfair" results in favor of rapid prototyping. Longer time spans will provide "unfair" results in favor of traditional design. The sweet spot is probably something like a week or two.
2: Make one of the games using your traditional approach
3: Make the other game using the rapid prototyping approach. I would suggest attempting to throw away all your preconceived notions and keep an open mind when you try this. When you are used to one development methology, switching to another without the old one getting in the way can be kinda tricky.
(The order of 2 and 3 probably doesn't matter)
4: Compare the results. The questions you should be asking yourself for each approach are something along the lines of: How complete is the game ? If I were to take only what I have now and spend X amount of time polishing what I have, but writing no new gameplay related code, would the result be of shippable quality ? You can probably think of more questions.
The aim of the exercise is to guage the suitability of each methology as fairly as is reasonable given appropriate constraints. You will learn through first hand experience the aspects of each methology that suits you best. You can then pick and chose which to keep and which to dump.
You can repeat the experiment to try out additional methologies. Yes, this is rapid prototyping for development methologies ;)
T.
#11
A fair suggestion. However, by your own words, the results are likely predictable.
Game development by its very nature tends to be a marathon rather than a race, so I would expect that a traditional approach would be a best fit in most cases. However, given an extremely short deadline, rapid prototyping would likely yield better results. There really isn't a "one size fits all" answer.
Obviously the whole premise of this discussion isn't fair when taken out of context from requirements, deadlines, and investors. Nonetheless, I find it immensely rewarding to discover what has worked for other developers, what hasn't, and why.
@Joel
I rather like the idea of keeping a prototype around throughout the development cycle. Thanks!
03/21/2007 (3:39 pm)
@TomA fair suggestion. However, by your own words, the results are likely predictable.
Quote:
It is worth noting that shorter time spans will provide "unfair" results in favor of rapid prototyping. Longer time spans will provide "unfair" results in favor of traditional design.
Game development by its very nature tends to be a marathon rather than a race, so I would expect that a traditional approach would be a best fit in most cases. However, given an extremely short deadline, rapid prototyping would likely yield better results. There really isn't a "one size fits all" answer.
Obviously the whole premise of this discussion isn't fair when taken out of context from requirements, deadlines, and investors. Nonetheless, I find it immensely rewarding to discover what has worked for other developers, what hasn't, and why.
@Joel
I rather like the idea of keeping a prototype around throughout the development cycle. Thanks!
#12
Some forethought and thinking is always required, and some exploration is always required. How much is really dependent on the project, the team size, and the team experience.
I am not one to lean to the 'less is better' or 'more is better' camp. I also don't draw a line between where the prototype ends and where the design doc begins, or, where the design doc ends and the application becomes the living 'design'.
for me, iteration is key. How fast one can iterate on ideas is highly project dependent. If it is quicker to iterate on paper, do it on paper. If it is quicker to iterate in the actual product, do it there. additionally, throughout a project, it is sometimes quicker to iterate in the product to inform the design and then do design iterations on paper (design doc).
it should also be noted that games are not all code. There is art and 'interaction' that must be tested. the interaction and look need to be tested directly, as in, you cannot test interaction properly without interacting with it. In many cases, the need to test the interaction quickly becomes of utmost importance.
I do see value in documenting in terms of understanding scope and deciding on the big unknowns (identifying priorities).
I tend to look at it at a higher level.
good practices are there to reduce risk and cost while still providing a good experience for the end user. Ultimately, no one gives a rats ass what the code looks like if the game is fun.
the metric is always the end product. If the product is good, and sells, then the process to get to that point worked. If being too 'cowboy' increases costs at the end because of instability of a rats nest of crap code, there can be problems. If costs are high during the 'design' phase, where a lot of time and money is being burned thinking about a project, without having something to test.. there can be problems if the design did not, in fact, turn out to be fun.
I personally lean toward a development style that seeks to control unknown risks and delays at the end of the project, while giving maximum flexibility for design change at ANY point of the project. To achieve this, one must walk a razors edge of thinking ahead, looking at what you have, and trying to anticipate what you might need.
So, there is always a need for balance, but what works for one project may not necessarily work for another project (or team).
hopefully my comments have added to this dicsussion.
03/22/2007 (7:46 am)
My own thoughts on this are that no 'one' methodology or approach will work in all cases. Some forethought and thinking is always required, and some exploration is always required. How much is really dependent on the project, the team size, and the team experience.
I am not one to lean to the 'less is better' or 'more is better' camp. I also don't draw a line between where the prototype ends and where the design doc begins, or, where the design doc ends and the application becomes the living 'design'.
for me, iteration is key. How fast one can iterate on ideas is highly project dependent. If it is quicker to iterate on paper, do it on paper. If it is quicker to iterate in the actual product, do it there. additionally, throughout a project, it is sometimes quicker to iterate in the product to inform the design and then do design iterations on paper (design doc).
it should also be noted that games are not all code. There is art and 'interaction' that must be tested. the interaction and look need to be tested directly, as in, you cannot test interaction properly without interacting with it. In many cases, the need to test the interaction quickly becomes of utmost importance.
I do see value in documenting in terms of understanding scope and deciding on the big unknowns (identifying priorities).
I tend to look at it at a higher level.
good practices are there to reduce risk and cost while still providing a good experience for the end user. Ultimately, no one gives a rats ass what the code looks like if the game is fun.
the metric is always the end product. If the product is good, and sells, then the process to get to that point worked. If being too 'cowboy' increases costs at the end because of instability of a rats nest of crap code, there can be problems. If costs are high during the 'design' phase, where a lot of time and money is being burned thinking about a project, without having something to test.. there can be problems if the design did not, in fact, turn out to be fun.
I personally lean toward a development style that seeks to control unknown risks and delays at the end of the project, while giving maximum flexibility for design change at ANY point of the project. To achieve this, one must walk a razors edge of thinking ahead, looking at what you have, and trying to anticipate what you might need.
So, there is always a need for balance, but what works for one project may not necessarily work for another project (or team).
hopefully my comments have added to this dicsussion.
#13
Yes, the results are likely predictable. But that is not the entire point of the exercise. The most important point was to find the mix of methologies that works best for you and your team through first hand experience without wasting lots of time or taking stabs in the dark.
Joe's post says the rest of the points I was trying to make and a few more better then I did ;)
T.
03/22/2007 (4:47 pm)
Woops, I seem to have a habit of posting in blogs and then forgetting I'd posted in them, then coming back to them later and thinking "oh, that's interesting" then realizing I've already posted a comment ;-)Quote:A fair suggestion. However, by your own words, the results are likely predictable.
Yes, the results are likely predictable. But that is not the entire point of the exercise. The most important point was to find the mix of methologies that works best for you and your team through first hand experience without wasting lots of time or taking stabs in the dark.
Joe's post says the rest of the points I was trying to make and a few more better then I did ;)
T.
Torque Owner Joel Davis
I find that the prototype is useful to keep around for the whole process -- if you have an idea for a change to gameplay you can try it out in the prototype in a hour or two, rather than spending a week on it in the engine.
I don't see it at as a point where you have to stop working on the prototype and start working on the game.. it's a continous process, each feature begins life in the prototype and then "graduates" to the real game.
I agree with the usefulness of UML.
Joel