an engine design
by William Finlayson · in Technical Issues · 09/12/2001 (3:36 pm) · 12 replies
I am trying to design an engine for a 3D game, and was wondering if anyone has had experience in writing their own. This is the first full engine i've tried to write, so I was hoping that someone who's done this kind of thing b4 could comment. I'm doing this under the assumption that it will only ever be used in windows.
As far as I can see, I'll need these main components:
input
sound
graphics
networking
I thought that each of these should have its own thread, so that input is accepted whatever is happening, network communication is constant, and while rendering would the game logic could continue. There would be another thread where all the game code itself, and AI would go. Maybe the AI needs to have it's own thread? I don't know as I've never done this b4. Anyway, thats my first assumption on the basic layout.
To comunicate between the components, I thought that instead of haveing the modules call functions in other modules directly, a message system could be used, with mutexs and stuff to make it thread safe, where modules post a message to other modules asking them to carry out commands, this means that each module can prioritise things, and carry out actions only when it is ready. I'm really not sure about this though, as it might be very slow. Some actions need to be completed b4 a module can carry on what it was doing, like the game asking the graphics to initialize before doing anything else, I suppose that the game thread would just wait until the graphics module puts a message on the game message que saying it's done. Not sure.
Thats the basic idea anyway, and I would be very glad if anyone could tell em if this is just plain stupid, if its feasible or not, or if there is a better way of structuring the engine, any comments would help me out a lot. Thanks.
As far as I can see, I'll need these main components:
input
sound
graphics
networking
I thought that each of these should have its own thread, so that input is accepted whatever is happening, network communication is constant, and while rendering would the game logic could continue. There would be another thread where all the game code itself, and AI would go. Maybe the AI needs to have it's own thread? I don't know as I've never done this b4. Anyway, thats my first assumption on the basic layout.
To comunicate between the components, I thought that instead of haveing the modules call functions in other modules directly, a message system could be used, with mutexs and stuff to make it thread safe, where modules post a message to other modules asking them to carry out commands, this means that each module can prioritise things, and carry out actions only when it is ready. I'm really not sure about this though, as it might be very slow. Some actions need to be completed b4 a module can carry on what it was doing, like the game asking the graphics to initialize before doing anything else, I suppose that the game thread would just wait until the graphics module puts a message on the game message que saying it's done. Not sure.
Thats the basic idea anyway, and I would be very glad if anyone could tell em if this is just plain stupid, if its feasible or not, or if there is a better way of structuring the engine, any comments would help me out a lot. Thanks.
About the author
#2
09/12/2001 (5:00 pm)
I concur. :)
#3
09/12/2001 (5:25 pm)
Ditto.
#4
if you are heading for 3D game, sound/input/network modules are rather small compare to the graphic bit.
depends on how you modulized things, you might also want a unified memory management module somewhere.
also, AI/physis/collision detection module still no where to be seen in your plan?? what about tools? exporter, file converter etc etc??
09/13/2001 (12:13 am)
usually, these modules not communicated with each other, instead, use a higher level (game level) module to access them if needed.if you are heading for 3D game, sound/input/network modules are rather small compare to the graphic bit.
depends on how you modulized things, you might also want a unified memory management module somewhere.
also, AI/physis/collision detection module still no where to be seen in your plan?? what about tools? exporter, file converter etc etc??
#5
AS to the other replies, I'm wanting to write a 3D engine just for the sake of it, not a game, otherwise, yes it would be far easier just to buy the V12 (or whatever its going to be called).
09/13/2001 (6:56 am)
thanks fred. Is it a good idea to use seperate threads for each subsection? I agree that a single controlling module would be the best way to control it all, but how would that communicate with the other modules? Would it be possible to use a messaging system like I've described, or is there a better way of doing it?AS to the other replies, I'm wanting to write a 3D engine just for the sake of it, not a game, otherwise, yes it would be far easier just to buy the V12 (or whatever its going to be called).
#6
As for thread usage, its ok to use threads for sound and networking, but I'd tend not to use it for anything where there would be a lot of thread context switches, so rendering and gamelogic/ai should be in the same thread.
Finally, I'd suggest using some fairly high level interface between the gamelogic and the renderer so that you have a fairly good level of abstraction between the game and the engine so that its easy to port the game OR replace the renderer WITHOUT affecting the other components.
Phil.
09/13/2001 (10:27 am)
Will, if your set on making an engine, the first thing I suggest is to concentrate on tools and the content management BEFORE doing rendering or anything fancy.As for thread usage, its ok to use threads for sound and networking, but I'd tend not to use it for anything where there would be a lot of thread context switches, so rendering and gamelogic/ai should be in the same thread.
Finally, I'd suggest using some fairly high level interface between the gamelogic and the renderer so that you have a fairly good level of abstraction between the game and the engine so that its easy to port the game OR replace the renderer WITHOUT affecting the other components.
Phil.
#7
I figured that by using a messaging system between components instead of direct calls, it means that the component can interpret and carry out the messages in any way it chooses, giving a high level of flexibility and abstraction, I was just worried that it would be slow.
09/13/2001 (10:33 am)
cheers, this is all very useful. I was going to have a seperate component for the renderer and the scene management, with a high level of abstraction between them, so that the same graphics component can be used to draw any scene, outdoors, indoors, space etc, just by changing the scene component.I figured that by using a messaging system between components instead of direct calls, it means that the component can interpret and carry out the messages in any way it chooses, giving a high level of flexibility and abstraction, I was just worried that it would be slow.
#8
but then again, this all depends on how you gonna do the scene management and render pipe and whats in your mind etc.
message system, I am using the general window's message queue to handle the network messages and some lower priority events. the rest are called directly.
multiple threads. what ever you do, if you are using D3D, don't even try to use multiple thread on the **render pipeline**. Even if you just create the device with multithread aware (without actually calling it in multiple thread), the cost is so much it ain't funny.
09/13/2001 (8:51 pm)
I would suggest polygon/renderer pipe should be seperated from the scene management. that will give you the flexibility on the scene management arrange the stuff you want without making too much hit in the render loop. assume you gonna use D3D, your render pipe will need to group vertices by vertexbuffer, then group them by texture then renderstate etc etc, so your render pipe could reduce the cost of switching vertexbuffer, texture etc etc. you can also twist the render pipe in the way you want to generate whatever effect with out worry too much on the higher level scene management. but then again, this all depends on how you gonna do the scene management and render pipe and whats in your mind etc.
message system, I am using the general window's message queue to handle the network messages and some lower priority events. the rest are called directly.
multiple threads. what ever you do, if you are using D3D, don't even try to use multiple thread on the **render pipeline**. Even if you just create the device with multithread aware (without actually calling it in multiple thread), the cost is so much it ain't funny.
#9
09/14/2001 (7:45 am)
thanks again, I think I have an idea on how to go about getting started now. cheers all. 
#10
As far as writing your own engine, I strongly recomend it. I've done it, from scratch, for the hell of it. Many late nights, and mass quantities of soda, and 2 rewrites were involved, but it was a supurb learning experience, and my favorite 5am victory was smooth shading. There was dancing in the streets and much rejoycing. (Well the hall at least, and nobody was up but me) So I say go for it, but save yourself a headache, and use OpenGL.
09/15/2001 (8:42 am)
Multiple threads, from my experiences and from questions, are *not* helpful at all. I would concentrate on building an efficent single threaded game, because (as I'm sure you're aware) only one thread can run at a time, and most people have only one processor. Besides, then you have to worry about synchronization and such, and that all take...da da...cpu cycles and memory. As far as writing your own engine, I strongly recomend it. I've done it, from scratch, for the hell of it. Many late nights, and mass quantities of soda, and 2 rewrites were involved, but it was a supurb learning experience, and my favorite 5am victory was smooth shading. There was dancing in the streets and much rejoycing. (Well the hall at least, and nobody was up but me) So I say go for it, but save yourself a headache, and use OpenGL.
#11
09/15/2001 (12:23 pm)
How do you acheive smooth input without using a seperate thread? With only one thread I don't see how you could accept input while doing anything else. I know that threads don't actually run concurrently, but it makes sure that even while you are in the middle of rendering or whatever, there will always be time allocated to the input handler. 
Torque Owner Bryan Walters
Forget making an engine for now and go buy the v12 engine after you learn that and finish a game then try to make an engine.
Just my two cents
Bryan