iTorque2D vs. Cocos2D performance.
by Chris W Hill · in iTorque 2D · 04/07/2011 (1:07 pm) · 30 replies
Hi,
I have completed some performance tests on iTorque2D, comparing it to the Cocos2D engine. I do not mean any malice by doing this comparison, but it is unfortunately quite obvious in my tests that iTorque2D has some serious graphics performance issues. I would be happy to be proven wrong:
blog.kihongames.com/2011/04/06/itorque2d-vs-cocos2d-performance/
I cannot mention this article without bringing up the overall quality of development on the engine. I feel that at some point there was a serious development effort by quality coders. But then, due to business problems, or other issues the engine quality has suffered drastically. What's wild is that as a new user I could sense this, without seeing the source code. I could feel a coder struggling through a new feature, with a manager standing behind him saying 'looks good, ship it'.
But I could also feel Torque's foundation shining through from those quality coders. These people know how to make games. They understand a developer's role in providing support to designers, and how solid tools provide designers the ability to create fantastic games. So regardless of the issues that I've seen, I'm still here.
But there are a lot of issues. And a lot of promises of new features. I hate to be soapboxy, but it must be said.
I can sense the bad management decisions that have plagued iTorque2D through the source code. The bugs are just symptoms of this problem. The issues are fundamentally systematic, and process-oriented. For example, shipping torque for the iphone without batch calls. The level datablocks feature is half-baked at best. These are features that perhaps matched some feature spec, but do not meet the expectations of the end user (me). Demand a higher quality of polish and functionality.
Don't worry about new features. Fix the bugs. Refactor. Polish existing features to perfection. Improve the documentation. Forget about Box2D, pinch/zoom or any features I see requested on the forums. Fix what you got.
I make some assumptions in this post, and I apologize if I am off, or offend. As I said, I am a Torque user, and I just think you'd want to hear my thoughts on this.
I have completed some performance tests on iTorque2D, comparing it to the Cocos2D engine. I do not mean any malice by doing this comparison, but it is unfortunately quite obvious in my tests that iTorque2D has some serious graphics performance issues. I would be happy to be proven wrong:
blog.kihongames.com/2011/04/06/itorque2d-vs-cocos2d-performance/
I cannot mention this article without bringing up the overall quality of development on the engine. I feel that at some point there was a serious development effort by quality coders. But then, due to business problems, or other issues the engine quality has suffered drastically. What's wild is that as a new user I could sense this, without seeing the source code. I could feel a coder struggling through a new feature, with a manager standing behind him saying 'looks good, ship it'.
But I could also feel Torque's foundation shining through from those quality coders. These people know how to make games. They understand a developer's role in providing support to designers, and how solid tools provide designers the ability to create fantastic games. So regardless of the issues that I've seen, I'm still here.
But there are a lot of issues. And a lot of promises of new features. I hate to be soapboxy, but it must be said.
I can sense the bad management decisions that have plagued iTorque2D through the source code. The bugs are just symptoms of this problem. The issues are fundamentally systematic, and process-oriented. For example, shipping torque for the iphone without batch calls. The level datablocks feature is half-baked at best. These are features that perhaps matched some feature spec, but do not meet the expectations of the end user (me). Demand a higher quality of polish and functionality.
Don't worry about new features. Fix the bugs. Refactor. Polish existing features to perfection. Improve the documentation. Forget about Box2D, pinch/zoom or any features I see requested on the forums. Fix what you got.
I make some assumptions in this post, and I apologize if I am off, or offend. As I said, I am a Torque user, and I just think you'd want to hear my thoughts on this.
#2
I did also test this example using one full sprite sheet generated via unChaos (a 4 minute optimisation) and noted that there are no real performance gains even though it would have been possible to batch render all objects from a single texture so definitely some improvements to be made here. I think this bit of research was great as it's showed that we can significantly boost the performance of TGB through a relatively simple enhancement and it makes clear that physics isn't necessarily the major bottleneck when using a large number of sprites, which is very useful and as suggested I would prioritise this over Box2D for this reason.
04/07/2011 (3:34 pm)
I do have one comment which is that I'm far more limited by lack of for example true multi-touch than I am FPS at this stage so I would definitely like to see progress in this area continue as a priority.I did also test this example using one full sprite sheet generated via unChaos (a 4 minute optimisation) and noted that there are no real performance gains even though it would have been possible to batch render all objects from a single texture so definitely some improvements to be made here. I think this bit of research was great as it's showed that we can significantly boost the performance of TGB through a relatively simple enhancement and it makes clear that physics isn't necessarily the major bottleneck when using a large number of sprites, which is very useful and as suggested I would prioritise this over Box2D for this reason.
#3
Michael: Thanks for your positive comments!
Alistair: If I'm understanding your test, you're saying that you took the art from the example project and put it all on one sprite sheet and then attempted to run the project?
I'm curious what you mean by 'true' multi-touch? I haven't gone too far with multi-touch on the ipad. As far as I can tell that was functioning as expected?
04/07/2011 (3:47 pm)
Thank you for the comments, guys!Michael: Thanks for your positive comments!
Alistair: If I'm understanding your test, you're saying that you took the art from the example project and put it all on one sprite sheet and then attempted to run the project?
I'm curious what you mean by 'true' multi-touch? I haven't gone too far with multi-touch on the ipad. As far as I can tell that was functioning as expected?
#4
04/07/2011 (4:06 pm)
Poking at the source code it looks like an attempt for batch rendering was started via t2dQuadBatch but abandoned for some reason.
#5
That's exactly it I placed everything into a giant sprite sheet. UnChaos (currently windows only) will cram as best as it can all sprites in a folder into one large sheet, it'll also generate a text document of all the co-ords of each image inside the giant sheet as global variables that you can copy and paste into Torque Script, this way if you refer to a sprite source rect as $ss_blackninja but he get moved around when you rebuild the spritesheet it wont affect your code as long as you paste in the new generated text.
By lack of true multitouch take a look at this post http://www.garagegames.com/community/forums/viewthread/125059 it's not just about Gestures. Currently there's no way to know for sure which finger is triggering the onMouse/iPhone touchdown callbacks and so it's difficult to develop multitouch code and depending on the gameplay style sometimes impossible without a change to the core C++ code.
04/07/2011 (4:13 pm)
@ChrisThat's exactly it I placed everything into a giant sprite sheet. UnChaos (currently windows only) will cram as best as it can all sprites in a folder into one large sheet, it'll also generate a text document of all the co-ords of each image inside the giant sheet as global variables that you can copy and paste into Torque Script, this way if you refer to a sprite source rect as $ss_blackninja but he get moved around when you rebuild the spritesheet it wont affect your code as long as you paste in the new generated text.
By lack of true multitouch take a look at this post http://www.garagegames.com/community/forums/viewthread/125059 it's not just about Gestures. Currently there's no way to know for sure which finger is triggering the onMouse/iPhone touchdown callbacks and so it's difficult to develop multitouch code and depending on the gameplay style sometimes impossible without a change to the core C++ code.
#6
Thanks for running that auxiliary test, that helps confirm what I was expecting. The real bottleneck on iOS devices is the batch calling, and part of the reason I picked Cocos2D as my comparison was because I know they have a solid implementation for batch calling.
Regarding multitouch, I totally agree with you that should be a priority. I think that aligns with what I was saying about having a higher quality of polish on these features. That definitely sounds like something that should be improved.
04/07/2011 (4:18 pm)
Hi Alistair,Thanks for running that auxiliary test, that helps confirm what I was expecting. The real bottleneck on iOS devices is the batch calling, and part of the reason I picked Cocos2D as my comparison was because I know they have a solid implementation for batch calling.
Regarding multitouch, I totally agree with you that should be a priority. I think that aligns with what I was saying about having a higher quality of polish on these features. That definitely sounds like something that should be improved.
#7
Glad you agree, as I just mentioned the guys at GarageGames must have agreed, there's an 80% complete implementation of batch rendering in t2dQuadBatch and mods to t2dStaticSprite but it was never finished and turned off, I wonder why.
04/07/2011 (4:23 pm)
@chrisGlad you agree, as I just mentioned the guys at GarageGames must have agreed, there's an 80% complete implementation of batch rendering in t2dQuadBatch and mods to t2dStaticSprite but it was never finished and turned off, I wonder why.
#8
Edit: Lol, I said Box2D instead of iTorque2D
04/07/2011 (4:27 pm)
Yes, I'm currently documenting the rendering pipeline of iTorque2D and cocos2d. After I complete that I will be looking into finishing that batch implementation, or creating my own. No promises.Edit: Lol, I said Box2D instead of iTorque2D
#9
Do you guys have more detailed documentation/diagrams on the rendering system etc? I'd be happy to work on a patch to enable batch rendering, same with multitouch and at least submit an attempt to try out, I get the feeling you guys have a lot more documentation on the under-workings of the engine I constantly feel uneasy making changes without truly knowing in what other ways that piece of code may be in use.
04/07/2011 (4:27 pm)
@MichaelDo you guys have more detailed documentation/diagrams on the rendering system etc? I'd be happy to work on a patch to enable batch rendering, same with multitouch and at least submit an attempt to try out, I get the feeling you guys have a lot more documentation on the under-workings of the engine I constantly feel uneasy making changes without truly knowing in what other ways that piece of code may be in use.
#10
Chris - great article and as a long time Torque user I agree that polishing what we've got and improving performance and eradicating bugs should be the priority right now.
04/07/2011 (11:41 pm)
Alistair - Full multi-touch functionality is provided by this enhancement: www.garagegames.com/community/blogs/view/19809. This is likely to be integrated into the engine soon.Chris - great article and as a long time Torque user I agree that polishing what we've got and improving performance and eradicating bugs should be the priority right now.
#11
I'm aware of this mod but I wouldn't go as far as to say 'full', I think it was mentioned that it doesn't track more than three fingers and most importantly it does not provide object level abstraction like the current Torque.
I teach TGB to ~ 50 students and both them and myself would not wish to use multi-touch in the manner implemented in that patch which is why we've been discussing it in the post linked above.
04/08/2011 (9:14 am)
Hi Conor,I'm aware of this mod but I wouldn't go as far as to say 'full', I think it was mentioned that it doesn't track more than three fingers and most importantly it does not provide object level abstraction like the current Torque.
I teach TGB to ~ 50 students and both them and myself would not wish to use multi-touch in the manner implemented in that patch which is why we've been discussing it in the post linked above.
#12
EDIT - and it tracks five fingers. Enough for a whole hand ;-)
04/08/2011 (2:12 pm)
That patch is an important first step, and without it it you have nothing but a bucket of confusion. However I agree with your desire for object level abstraction, all good. But the patch let's you get started now.EDIT - and it tracks five fingers. Enough for a whole hand ;-)
#13
It's not 100% object level gesture friendly yet but it's certainly the first phase and possible. From an end user point of view you simply record an id on touch down and release it on touch up and that way you can fully track any finger.
Required mods to the following files: iPhoneOGL.mm, iPhoneInput.mm, event.h, guiCanvas.cc, guiTypes.h, t2dSceneWindow.cc and t2dSceneObject.cc.
The mod itself is pretty simple, scrap most of the code in iPhoneInput.mm and pass events straight along to the event system, add an extra member in two event structures to hold the id of the finger event and pass it along to the last param on onMouseX as to not break any existing single finger functionality.
I don't know what the rules on posting source code are what's the best approach here?
Second phase would probably take another 15 minutes (but I'd like to test this out for now and it does what I need) where the scene window tracks which objects were clicked by which id and objects are aware of the order in which clicks took place, then we could go back to providing a click order as opposed to ID that was on the object level.
From there we have the raw data for implementing a gesture system.
Cheers
04/08/2011 (6:51 pm)
I've just spent 15minutes on my macbook and now have working object level abstraction. It's not perfect but it works. Simply adds an extra %id parameter to all onMouseX callbacks for window or objects that will contain an id unique to each finger (11 fingers A.O.K).It's not 100% object level gesture friendly yet but it's certainly the first phase and possible. From an end user point of view you simply record an id on touch down and release it on touch up and that way you can fully track any finger.
Required mods to the following files: iPhoneOGL.mm, iPhoneInput.mm, event.h, guiCanvas.cc, guiTypes.h, t2dSceneWindow.cc and t2dSceneObject.cc.
The mod itself is pretty simple, scrap most of the code in iPhoneInput.mm and pass events straight along to the event system, add an extra member in two event structures to hold the id of the finger event and pass it along to the last param on onMouseX as to not break any existing single finger functionality.
I don't know what the rules on posting source code are what's the best approach here?
Second phase would probably take another 15 minutes (but I'd like to test this out for now and it does what I need) where the scene window tracks which objects were clicked by which id and objects are aware of the order in which clicks took place, then we could go back to providing a click order as opposed to ID that was on the object level.
From there we have the raw data for implementing a gesture system.
Cheers
#14
I can't say much more, since nothing has solidified. The point is that your work was an eye opener and greatly appreciated. The flow of information at the new GG goes top to bottom, without any technical information getting lost in the process. This is why having a CEO who could be a technical director kicks ass =)
@Alistair -
04/09/2011 (8:14 am)
@Chris - I had a talk with Eric after he read my e-mail about your blog. The conversation went exactly how I expected. He wasn't happy about the results. He thinks an implementation of true batching would be a new feature, just as an important as any other new features planned for a release. He escalated it up to the owners and started talking both code and potential developers who could dive into the engine.I can't say much more, since nothing has solidified. The point is that your work was an eye opener and greatly appreciated. The flow of information at the new GG goes top to bottom, without any technical information getting lost in the process. This is why having a CEO who could be a technical director kicks ass =)
@Alistair -
Quote:I don't know what the rules on posting source code are what's the best approach here?You can share your code in the forums or as a resource. As long as you are not posting or sharing the entire engine, this practice is highly encouraged. Sounds like a good improvement.
#15
In that case guys, %clicks in the onMouseX callbacks, what determines these?
For example is normal behaviour on the PC that onMouseUp is called for every quick successful click but that the count is also provided or is it that during rapid clicks only a single onMouseUp is called and that we have to check the %clicks?
Knowing this will help me provide a consistent patch.
04/09/2011 (9:07 am)
@Michael - awesome.In that case guys, %clicks in the onMouseX callbacks, what determines these?
For example is normal behaviour on the PC that onMouseUp is called for every quick successful click but that the count is also provided or is it that during rapid clicks only a single onMouseUp is called and that we have to check the %clicks?
Knowing this will help me provide a consistent patch.
#16
Cheers
04/09/2011 (12:01 pm)
Ok I'm 80% through phase two of my plan but I've hit an annoying error, I'm using a std::map<> object but it's causing conflicts with the new operator used on the iPhone. I know issues like this have cropped up for you guys before, any ideas how I get around this?Cheers
#17
04/09/2011 (1:25 pm)
Good writeup and very consistent with the unwritten "rule of 30" many of us have learned as iT2D developers. (That is, don't exceed 30 sprites if you want to maintain good performance.)
#18
Additionally, I have completed documenting iTorque2D and Cocos2D's rendering pipelines, and have completed many more tests. The results are very, very surprising. My initial assumptions on batch calling seem to be wrong, and the bottleneck most likely lies elsewhere. Soon I will have a follow-up blog entry.
04/09/2011 (1:41 pm)
@mperry : I'm glad that my research and comments have been useful. I understand that this may not be easily prioritized, but that is one of the many great things about Torque: Even though we disagree on priority, I've got the source, so I'm not blocked by what you decide.Additionally, I have completed documenting iTorque2D and Cocos2D's rendering pipelines, and have completed many more tests. The results are very, very surprising. My initial assumptions on batch calling seem to be wrong, and the bottleneck most likely lies elsewhere. Soon I will have a follow-up blog entry.
#19
04/09/2011 (1:59 pm)
@chris : Thanks! Yeah, I've come across that rule myself! :) Rules were made to be broken. 
#20
04/09/2011 (2:12 pm)
Long time since I performance checked the Torque engine but from memory an awful lot of time was spent in the String table.
Employee Michael Perry
GarageGames