Spanking new Particle Engine
by Melv May · in Torque Game Engine · 11/18/2003 (12:48 am) · 155 replies
Anyway, please come in, pull up a chair and sit down .. fancy a brew?
For those of you who don't know, I've been involved in the development of a game called Strategem which is a turn-based strategy game with a difference ... major fun with major FX!
During the course of the development it has become pretty obvious that to pull-off the kind of special-fx we need for some of our board creatures, their battling and the use of magical powers within the realm, we need not only an intuitive but flexible method of creating special-fx, specifically particles.
The stock particle engine provides adequate features for simple effects such as basic fire and smoke, even precipitation effects but these can be difficult and in some cases, damn near impossible to achieve easily. At best, the particle effects are embedded within another C++ object to control emissions for the particular task. This is understandable as it wasn't written to be a do-it-all particle engine in the first place.
This has driven the need for me to take the bull by the horns and come up with a new way of getting the huge quantity of effects needed for our game.
During the last year, in parallel with Strategem development, I've spent a considerable amount of time researching all relevant areas of particle systems from offline rendering systems to realtime ones e.g. WonderTouchs 'Particle Illusion' or Max Paynes' 'Particle-FX'. I've focused not only on their features but also more importantly, the user interfaces, in particular features such as being able to set many attributes with a catmull-rom spline graph over unit-time as per 'Particle-FX'.
An area of particle generation that I've focused on is the ability to have a particle engine that can simulate more complex and dynamic effects where emitters move ballistic paths and don't simply generate from fixed points, planes or various volumes. Another critical feature was the ability to reuse particle definitions and form a library system where particles, emissions and overall effects can be reused. This leads to not only a neat feature but performance benefits as well.
To this end, I've come up with a system that I think rivals most, if not all, realtime particle systems and some offline ones as well. A pretty bold statement but this is more than talk and I will bring this to demo sometime early next year, you can count on that.
>>> Continued on next post ...
For those of you who don't know, I've been involved in the development of a game called Strategem which is a turn-based strategy game with a difference ... major fun with major FX!
During the course of the development it has become pretty obvious that to pull-off the kind of special-fx we need for some of our board creatures, their battling and the use of magical powers within the realm, we need not only an intuitive but flexible method of creating special-fx, specifically particles.
The stock particle engine provides adequate features for simple effects such as basic fire and smoke, even precipitation effects but these can be difficult and in some cases, damn near impossible to achieve easily. At best, the particle effects are embedded within another C++ object to control emissions for the particular task. This is understandable as it wasn't written to be a do-it-all particle engine in the first place.
This has driven the need for me to take the bull by the horns and come up with a new way of getting the huge quantity of effects needed for our game.
During the last year, in parallel with Strategem development, I've spent a considerable amount of time researching all relevant areas of particle systems from offline rendering systems to realtime ones e.g. WonderTouchs 'Particle Illusion' or Max Paynes' 'Particle-FX'. I've focused not only on their features but also more importantly, the user interfaces, in particular features such as being able to set many attributes with a catmull-rom spline graph over unit-time as per 'Particle-FX'.
An area of particle generation that I've focused on is the ability to have a particle engine that can simulate more complex and dynamic effects where emitters move ballistic paths and don't simply generate from fixed points, planes or various volumes. Another critical feature was the ability to reuse particle definitions and form a library system where particles, emissions and overall effects can be reused. This leads to not only a neat feature but performance benefits as well.
To this end, I've come up with a system that I think rivals most, if not all, realtime particle systems and some offline ones as well. A pretty bold statement but this is more than talk and I will bring this to demo sometime early next year, you can count on that.
>>> Continued on next post ...
About the author
#62
Melv, can't wait to see what you have in store. :-) Like I said, I love Particle Illusions, and have literally just played with it for hours. lol
11/19/2003 (4:17 pm)
Alex, with the current particle editor you can change things easily, on-the-fly.Melv, can't wait to see what you have in store. :-) Like I said, I love Particle Illusions, and have literally just played with it for hours. lol
#63
Showing it in a particular context e.g. attached to a projectile or different chimney smoke effects is much more difficult as these are obviously live instances and objects in their own right. I was thinking of having the ability to import DTS shapes and the effects can be mounted to them and you move the shape as mentioned above. Also, there is the ability to offset all the elements of the effects so you can position the effects relatively. This offset can also change over unit-time.
The layout of the editor is; editing panels on the left, graph editor bottom-right, particle effects view upper-right. All these use a split-bar so that you can get the most out of the resolution you're in. The minimum I'd like to run my prototype GUI is 800x600 but I don't think that should be a problem for game developers.
When I've got much further down the line, I'll release a proper specification including all the available attributes but for now, with so much in the air, I'll stick to describing it.
... and like Davis said, I should shut-up and get on with it...
- Melv.
11/19/2003 (10:53 pm)
Alex/Davis: I intended to have a live particle preview within the editor itself with the ability to drag the effects around or make it move in set ways e.g. sprial, edges of a box, back and forth along a line, circle etc. You can do this whilst tweaking any of its attributes. There's also different methods to trigger the effect.Showing it in a particular context e.g. attached to a projectile or different chimney smoke effects is much more difficult as these are obviously live instances and objects in their own right. I was thinking of having the ability to import DTS shapes and the effects can be mounted to them and you move the shape as mentioned above. Also, there is the ability to offset all the elements of the effects so you can position the effects relatively. This offset can also change over unit-time.
The layout of the editor is; editing panels on the left, graph editor bottom-right, particle effects view upper-right. All these use a split-bar so that you can get the most out of the resolution you're in. The minimum I'd like to run my prototype GUI is 800x600 but I don't think that should be a problem for game developers.
When I've got much further down the line, I'll release a proper specification including all the available attributes but for now, with so much in the air, I'll stick to describing it.
... and like Davis said, I should shut-up and get on with it...
- Melv.
#64
11/20/2003 (1:28 pm)
Good point Eric, thanks for pointing out that resource to me. That'll keep my tided over till Melv's neat-o tools come out! ;) 
#65
Ya know... I may just code that tonight as a quick break from my current "Bash My Head Against The Wall Why Does The Engine Do THAT!" problem :-)
Melv: I wasn't takin' it as bashin' our work - I was just pointin' out what was possible. I look forward to something with even more ooph to it :-) I'll set down and compile my list - I'm not sure what of it is useful to you, what isn't, but what the heck - if you implemented it all, I'd definitely buy it!
When it comes to protecting your asset... well, it's your baby and I can understand from your point of view. But if I need to add another attribute to particles, for instance, that leaves me at your whims. You a heck of a nice guy, but, you've also got other paying projects to do I'm sure - bigger business goes before customers that only spend $20 on a resource :-)
And as for this:
11/20/2003 (1:55 pm)
Alex: One o' these days I'm going to have extra time, and a burst of energy, and execute one of my plans. An in-game script editor. Really wouldn't take much to do, and just have a nice 'exec' button on top. Edit in game, exec the script, close the editor. I would have saved friggin WEEKS of time I think over the past 16 months of hack, execute, blahblahblah. Yeah, I know I pointed out how to do it a little faster - doesn't mean I do it all the time though ;-)Ya know... I may just code that tonight as a quick break from my current "Bash My Head Against The Wall Why Does The Engine Do THAT!" problem :-)
Melv: I wasn't takin' it as bashin' our work - I was just pointin' out what was possible. I look forward to something with even more ooph to it :-) I'll set down and compile my list - I'm not sure what of it is useful to you, what isn't, but what the heck - if you implemented it all, I'd definitely buy it!
When it comes to protecting your asset... well, it's your baby and I can understand from your point of view. But if I need to add another attribute to particles, for instance, that leaves me at your whims. You a heck of a nice guy, but, you've also got other paying projects to do I'm sure - bigger business goes before customers that only spend $20 on a resource :-)
And as for this:
Quote:... I'll be quiet now and get working ... ;)No way... you'll be back. Oh yes, you'll be back chatting again soon ;-)
#66
-Jeff Tunnell GG
11/20/2003 (2:17 pm)
I doubt we would be willing to publish any Torque type of extention that did not have the source code included. I understand the issues with protecting assets, but we took the plunge, and have taken our lumps along the way. Pretty much just a philosophical thing.-Jeff Tunnell GG
#67
11/20/2003 (2:24 pm)
Very cool, Jeff. Kinda nice to see that sort of stance! 
#68
Thanks....
11/20/2003 (2:54 pm)
Hey Melv... Looks like you might have something good cooking here. I would certainly be willing to drop $25 to $50 in the pot for a good particle plug in. Keep up the good work and please keep everyone posted.Thanks....
#69
I've got no problems with releasing source code whatsoever, I'm not trying to protect how anything is done, simply protecting the product. The good thing about releasing the runtime particle code is that the community could suggest improvements in certain areas and they could be made in a controlled fashion in the same way as the TGE itself.
Am I correct in assuming that we're talking about code for EVERYTHING here and not just the particle runtime but the particle editor as well? How could the asset be protected if this was the case? Why is source code for the games not released in the same context? If it is to trace potential bugs and make improvements then this surely applies to games as well. Hey, I'm not rocking the boat here as I've already stated that I'll release the source-code without question but it's an interesting subject. This should probably happen on Joshuas new thread on the subject I guess.
Anyway, enough of that, it's Friday, last day at work for a while as I'm taking next week off work therefore I've got the next 9 FULL days to work on Strategem and the beginnings of the particle engine ... sweet!
- Melv.
11/21/2003 (12:14 am)
Just to prove Davis correct, I thought I'd post something. ;)I've got no problems with releasing source code whatsoever, I'm not trying to protect how anything is done, simply protecting the product. The good thing about releasing the runtime particle code is that the community could suggest improvements in certain areas and they could be made in a controlled fashion in the same way as the TGE itself.
Am I correct in assuming that we're talking about code for EVERYTHING here and not just the particle runtime but the particle editor as well? How could the asset be protected if this was the case? Why is source code for the games not released in the same context? If it is to trace potential bugs and make improvements then this surely applies to games as well. Hey, I'm not rocking the boat here as I've already stated that I'll release the source-code without question but it's an interesting subject. This should probably happen on Joshuas new thread on the subject I guess.
Anyway, enough of that, it's Friday, last day at work for a while as I'm taking next week off work therefore I've got the next 9 FULL days to work on Strategem and the beginnings of the particle engine ... sweet!
- Melv.
#70
The asset can be protected in the same manner the Torque is. You release the source code, but you continue to control the password protected CVS. While GG will not be willing to set up a CVS for every code related project, we are willing to host those that we feel are strategic or have a proven developer behind.
-Jeff Tunnell GG
11/21/2003 (7:46 am)
Haven't thought about the editor source code. I doubt that would fall under the "we require source" ideal. We don't have source to 3DS Max or Milkshape, and yet they are still useful tools. However, I would like to see the source to a good editor be available since I think it could spur incredible creativity within the community.The asset can be protected in the same manner the Torque is. You release the source code, but you continue to control the password protected CVS. While GG will not be willing to set up a CVS for every code related project, we are willing to host those that we feel are strategic or have a proven developer behind.
-Jeff Tunnell GG
#71
I meant to ask you this before, but it somehow slipped my mind. While playing around with the current particle system, I was trying to come up with a way to have "particle influence" volumes. Nothing fancy, 'cept when an area (probably a simple box or sphere) were placed over any particles, they would be "influenced" by some algorithm or simple formula.
The reason I metion this is because there are some really cool effects that can be "faked" with particle influence volumes. Such as a rocket moving through a particle smoke cloud could have a trailing influence volume that would cause any particles to have an induced momentum in the direction of the volume's motion. Thus, the rocket could "pop" through the smoke cloud and you'd see a bit of the cloud "pulled" along with the rocket for as long as the rocket were in the particle cloud.
Keeping the volume calculations simple would be key, but the overall effect would be very cool indeed. There isn't any reason why in a game there should be "actual physical calculations per particle" happening. This is just silly, because it's a game -- not an exact physical simulation. But I'm digressing.
Let me just say that I hope that you would consider adding something like this. I think it would add that little bit of extra realism. And if it were integrated directly into the particle system, that (to me) would be the best option.
EDIT: I have an image depicting what I wanted to do, but I can't upload from work... Have to wait until I get home.
- Brett
11/21/2003 (8:32 am)
Melv,I meant to ask you this before, but it somehow slipped my mind. While playing around with the current particle system, I was trying to come up with a way to have "particle influence" volumes. Nothing fancy, 'cept when an area (probably a simple box or sphere) were placed over any particles, they would be "influenced" by some algorithm or simple formula.
The reason I metion this is because there are some really cool effects that can be "faked" with particle influence volumes. Such as a rocket moving through a particle smoke cloud could have a trailing influence volume that would cause any particles to have an induced momentum in the direction of the volume's motion. Thus, the rocket could "pop" through the smoke cloud and you'd see a bit of the cloud "pulled" along with the rocket for as long as the rocket were in the particle cloud.
Keeping the volume calculations simple would be key, but the overall effect would be very cool indeed. There isn't any reason why in a game there should be "actual physical calculations per particle" happening. This is just silly, because it's a game -- not an exact physical simulation. But I'm digressing.
Let me just say that I hope that you would consider adding something like this. I think it would add that little bit of extra realism. And if it were integrated directly into the particle system, that (to me) would be the best option.
EDIT: I have an image depicting what I wanted to do, but I can't upload from work... Have to wait until I get home.
- Brett
#72
I thought about something similar myself e.g. Deflectors/Attractors although I decided that it wasn't essential although that can change. :)
There are different ways to approach this, the more flexible ones would probably have too much of a performance hit so a better one from a performance standpoint would be to have a new object, say "fxDeflector". This would be a data-driven object as all the others and would define the attraction/deflection required. You would be able to create these, register them with the Particle Engine and move them as you see fit or attach them to an object.
When you define the "fxEmission", as well as selecting a "fxEmitter" and a "fxParticle" you could select an optional "fxDeflector" that can affect the emission. Any of these "fxDeflectors" created and registered within the particle engine would affect the emission.
This would mean that you would define deflector types that can affect any emission (no matter what effect it's used in) that is created.
I'd have to think about this more so that it provided most flexibility without hindering performance.
- Melv.
11/21/2003 (9:06 am)
Brett,I thought about something similar myself e.g. Deflectors/Attractors although I decided that it wasn't essential although that can change. :)
There are different ways to approach this, the more flexible ones would probably have too much of a performance hit so a better one from a performance standpoint would be to have a new object, say "fxDeflector". This would be a data-driven object as all the others and would define the attraction/deflection required. You would be able to create these, register them with the Particle Engine and move them as you see fit or attach them to an object.
When you define the "fxEmission", as well as selecting a "fxEmitter" and a "fxParticle" you could select an optional "fxDeflector" that can affect the emission. Any of these "fxDeflectors" created and registered within the particle engine would affect the emission.
This would mean that you would define deflector types that can affect any emission (no matter what effect it's used in) that is created.
I'd have to think about this more so that it provided most flexibility without hindering performance.
- Melv.
#73
Thanks for clearing that up; what you suggest sound pretty good. It was inevitable that this subject was going to come out sooner or later I guess.
Thanks.
- Melv.
11/21/2003 (9:09 am)
Jeff,Thanks for clearing that up; what you suggest sound pretty good. It was inevitable that this subject was going to come out sooner or later I guess.
Thanks.
- Melv.
#74
I guess what I was thinking was that the fxDeflector (or fxVolume -- whatever :-) Would be registered as part of the particle engine. As it was moved through the scene, it would be responsible for detecting if any particles were within its bounds. If they were, it would modify each fxParticle directly. It could add a little X, Y, or Z to the particle (maybe an X, Y or Z momentum) then move onto the next particle. Thus, it wouldn't be the responsibility of the particle system to determine which particles fall within a volume. I think that would give the most speed gains.
EDIT: I guess that's why a sphere would make the most sense as the volume. You would only need to check if a particle is within radius units of the volume's center. Much faster than checking if it falls within a box's boundaries. Picture that maybe a particle has 4 params that are reserved for the influence. X, Y, Z momemtum and a percentage of the influence. Thus when a particle must be moved you would only need to add what was in the X, Y, Z influence vars, with a little bleed-off based on the percentage.
I know, I know... I'm not the developer -- so can it... :-)
- Brett
11/21/2003 (2:15 pm)
Melv,I guess what I was thinking was that the fxDeflector (or fxVolume -- whatever :-) Would be registered as part of the particle engine. As it was moved through the scene, it would be responsible for detecting if any particles were within its bounds. If they were, it would modify each fxParticle directly. It could add a little X, Y, or Z to the particle (maybe an X, Y or Z momentum) then move onto the next particle. Thus, it wouldn't be the responsibility of the particle system to determine which particles fall within a volume. I think that would give the most speed gains.
EDIT: I guess that's why a sphere would make the most sense as the volume. You would only need to check if a particle is within radius units of the volume's center. Much faster than checking if it falls within a box's boundaries. Picture that maybe a particle has 4 params that are reserved for the influence. X, Y, Z momemtum and a percentage of the influence. Thus when a particle must be moved you would only need to add what was in the X, Y, Z influence vars, with a little bleed-off based on the percentage.
I know, I know... I'm not the developer -- so can it... :-)
- Brett
#75
Doors! Yes i know there is a resource, which dosnt work anymore, and i dont have quite enough knowledge of torque to play around with it yet.
Pathed Interiors! Must be done and fixed!!!
A better Vehicle system!
As in, better damage system(which im working on using node hiding) to produce visable damage, Slightly Better physics, surface effects(dirt=less traction, road=more) RPM's, variable engine sound, Gears.
Sparks if side of *car* hits *something* and slides..
Work done on the model/render/load code, its confusing as hell in some places, and a bitch to write an exporter for.
-Needs smooth animation transitions.
-Seperate parts of the animation of skelleton, so arm could do something else while running, or just setup similar to Q3 (lower,upper,head style)
-Documentation.
Shadows. Dts cast shadow onto dts
Shaders! Some form of shader, Q3 style perhaps.
And Glass! Must have glass, breakable glass. Trans on difs.
No offense, those are the things I would really prefer to see first, the particles in the engine are decent enough and more than usable. And the engine as it is isnt good for a hellofalot without first making heavy mode to it. MMP Outdoors, with simple buildings.
But even then, i would still happily pay $10 (euro?) which is 20-30 NZ) For what your offering Melv.
Its professional and seems well done, comes with tools to use with it, possibly prebuilt library of particles.
Yeah, but i still rate my list over particles at the moment, functionality is the name of my game until its done, then prettiness can strike.
11/21/2003 (7:19 pm)
There are a few things i would pay for first.Doors! Yes i know there is a resource, which dosnt work anymore, and i dont have quite enough knowledge of torque to play around with it yet.
Pathed Interiors! Must be done and fixed!!!
A better Vehicle system!
As in, better damage system(which im working on using node hiding) to produce visable damage, Slightly Better physics, surface effects(dirt=less traction, road=more) RPM's, variable engine sound, Gears.
Sparks if side of *car* hits *something* and slides..
Work done on the model/render/load code, its confusing as hell in some places, and a bitch to write an exporter for.
-Needs smooth animation transitions.
-Seperate parts of the animation of skelleton, so arm could do something else while running, or just setup similar to Q3 (lower,upper,head style)
-Documentation.
Shadows. Dts cast shadow onto dts
Shaders! Some form of shader, Q3 style perhaps.
And Glass! Must have glass, breakable glass. Trans on difs.
No offense, those are the things I would really prefer to see first, the particles in the engine are decent enough and more than usable. And the engine as it is isnt good for a hellofalot without first making heavy mode to it. MMP Outdoors, with simple buildings.
But even then, i would still happily pay $10 (euro?) which is 20-30 NZ) For what your offering Melv.
Its professional and seems well done, comes with tools to use with it, possibly prebuilt library of particles.
Yeah, but i still rate my list over particles at the moment, functionality is the name of my game until its done, then prettiness can strike.
#76
Luke: Don't disagree with anything you've said here but I will say that developing this feature doesn't have to be at the exclusion of the features you mentioned and so priorities don't come into it. Obviously I'm not the only developer and there are lots of others who should perhaps take these things up. All this development can happen in parallel, that's the beauty of community development. :)
Thanks again for everyones input, it's much appreciated.
Today is the first day of a 9-day intensive development session for Strategem and the beginnings of the particle engine so here goes ............
- Melv.
11/21/2003 (11:59 pm)
Brett: Let me think about this over the next few days as I don't want to jump into promising anything that I've not had a long think about. It sounds like it could add some interesting dynamics though.Luke: Don't disagree with anything you've said here but I will say that developing this feature doesn't have to be at the exclusion of the features you mentioned and so priorities don't come into it. Obviously I'm not the only developer and there are lots of others who should perhaps take these things up. All this development can happen in parallel, that's the beauty of community development. :)
Thanks again for everyones input, it's much appreciated.
Today is the first day of a 9-day intensive development session for Strategem and the beginnings of the particle engine so here goes ............
- Melv.
#77
11/22/2003 (12:19 am)
Go Melv!
#78
I forget that this is work you did for your own game and are now maybe releasing to the community, I sound like i wanted to force you to do the other things first or something, Not release what you had already done..... LOL... geez...
Go nuts Melv, have fun, drink heaps of coffee, and give that game a real knuckle sandwich. (dont forget to not stress)
11/22/2003 (3:01 am)
Heh, just re-read my post above.I forget that this is work you did for your own game and are now maybe releasing to the community, I sound like i wanted to force you to do the other things first or something, Not release what you had already done..... LOL... geez...
Go nuts Melv, have fun, drink heaps of coffee, and give that game a real knuckle sandwich. (dont forget to not stress)
#79
... I'll use this thread to post updates over the weeks ahead.
- Melv.
11/22/2003 (6:05 am)
Thanks. :)... I'll use this thread to post updates over the weeks ahead.
- Melv.
#80
Bearing in mind that I can script well, but am very weak when programming C++, I would like to ask if 3d particles would be feasable. I don't mean any real complex shapes tied into the collision code, just simple low-poly 3d objects for the particles. There probably wouldn't be much of an advantage to it, but I can imagine a few interesting visual effects.
Somthing else I am curious about is if it ends up as a seperate application, what would you see in the "viewing" window, as far as backgrounds go. I think it would be great to have a simple terrain to see the particles against.( I don't know if I missed this or not)
My request for the editor is that you would keep in mind that it may very well be used by non-programmers for putting together a particle effect, so barring moron friendly names for variable adjustments I would like an explination as to what effect each adjustment/option has on the particles for whatever you end up with.
I would certainly be willing to pay $20+ for this type of application. Actually there are a lot of things I'd be willing to buy as an enhancement, just to save on the dev time. Considering everything you have contributed to the community here, I would not consider it the least bit "mercenary" if you wanted to sell an enhancement like this particle system.
11/25/2003 (5:57 pm)
It might be a tad late, but I thought I'd add a bit to this....Bearing in mind that I can script well, but am very weak when programming C++, I would like to ask if 3d particles would be feasable. I don't mean any real complex shapes tied into the collision code, just simple low-poly 3d objects for the particles. There probably wouldn't be much of an advantage to it, but I can imagine a few interesting visual effects.
Somthing else I am curious about is if it ends up as a seperate application, what would you see in the "viewing" window, as far as backgrounds go. I think it would be great to have a simple terrain to see the particles against.( I don't know if I missed this or not)
My request for the editor is that you would keep in mind that it may very well be used by non-programmers for putting together a particle effect, so barring moron friendly names for variable adjustments I would like an explination as to what effect each adjustment/option has on the particles for whatever you end up with.
I would certainly be willing to pay $20+ for this type of application. Actually there are a lot of things I'd be willing to buy as an enhancement, just to save on the dev time. Considering everything you have contributed to the community here, I would not consider it the least bit "mercenary" if you wanted to sell an enhancement like this particle system.
Torque Owner Alex Swanson