Plan for Sam Bacsa

So, after reviewing a conversation that happened between Noolness, Tim-MGT, and a few others in IRC about the performance of Codeweaver/TorqueDev, I decided to run some tests to see just how it measures up to similarly feature-filled IDEs (namely, SharpDevelop, an open-source C#/VB.Net IDE which is quite feature-complete, almost up to par with Visual Studio).

Statistcs follow. If you're not interested in these, stop reading :P

First I compared the new version of CW (pending delta 7) with the .Net 1.1 (delta 6) version in loading of files, rendering time, and tokenization time.

Here's how these stacked up:

- SharpDevelop (latest) loaded a 3.7 megabyte file (almost 1,000,000 lines of code) in about 25 seconds and took up way more memory than absolutely necessary (~500 megs) -- HOLY CRAP!

- D6 (.Net 1.1) loaded a 3.7 megabyte file in about 30 seconds and took up a boat-load of memory (~300 megs) -- also HOLY CRAP [but not quite as much]!

- D7 (.Net 2.0) loaded a 3.7 megabyte file in about 15 seconds and took up quite a bit of memory too (170 megs) -- also HOLY CRAP!

But see, that's unrealistic. 1 million lines of code in a file would be ridiculous. Here are some better, more realistic statistics:

- D6 (1.1) loaded a 380k file (about 6,500 lines of code) in about 7 seconds and took up about 40 megs of memory (be aware that it starts at around 32 when you load the whole IDE)

- D7 (2.0) loaded a 380k file in about 2 seconds and took up about 35 megs of memory (this time it started around 29)

- SharpDevelop (latest) loaded a 380k file in about 3 seconds and took up about 60 megs of memory (started at around 40).

Scroll and rendering speed on Codeweaver was about 50% faster. Insertion speed (adding text to the middle of the file) was instant on CW and lagged a few milliseconds on SharpDevelop. And, of course, as you can see from above, the improvement of loading the large files is immense (cut by more than 50%).

Now, running the garbage collector routines on Codeweaver after loading the file (these run intermittently through the process of the file being open, probably a few minutes in) cut down the usage like so:

- D6 (1.1) went from 40 megs usage to 35.
- D7 (2.0) went from 35 megs usage to 27 (less than when it was first loaded).
- SharpDevelop I left open for about 15 minutes and the memory usage only decreased by 6 megs

The overall rendering and usage speeds (during debugging, etc) between the D6 and D7 versions was about a 170% improvement in D7 (this is based on timing using a timer app and task manager, yay!).

Now, the average Torque programmer will not have 6,500 lines of code in one file. The usual averages run around 500 - 1,000. Any more than that should be split to separate files. Most files load in under one second, have no render delay, and take up a miniscule amount of memory. The jump in memory usually occurs only when the file is first loaded and not on subsequent loads of additional files (although, memory usage will increase by some).

The average idle memory usages of these programs, with 8 documents open, all of similar length and highlight intensity are as follows:

- SharpDevelop (latest) idled at around 70 megs.

- D6 (1.1) idled around 60 megs.

- D7 (2.0) idled around 55 megs.

- Visual Studio (2005) idled around 75 megs.


Improvements to debugging decrease processor load during a session. "Priority juggling" also allows for fast response time during the needed situations, and minimum presence during others.


In conclusion, delta release 7 stacks up against the very popular open-source SharpDevelop editor. This is certainly not Visual Studio grade, and never will be. Some have suggested to me that I focus on making a VS mod instead. While this would be an option, one must keep in mind that not many can afford Visual Studio, nor tolerate its enormous presence (over 4.5 gigs, if you have the MSDN library installed; ~2 gigs otherwise). For the .Net framework, this is squeezing a lot of performance out of the "virtual machine" architecture's overhead. As faster processors and more memory become increasingly available at inexpensive prices, this overhead will soon dwindle to a minimum.

For those concerned with having platform-native code execute for speed, there are other alternatives.

In case you're reading this, Noolness et al, I'm happy you reminded me of the issue. I've received many e-mails in the past about the performance of TorqueDev/CW (and it has greatly increased in the past versions), but I failed to address some of the core issues which I have done in the coming version. Additionally, the 2.0 framework optimizes code greatly, and runs much faster.

OK, statistics over. Comments always appreciated. I know many hundreds of developers are using Codeweaver on a regular basis, and this makes me very happy. To those who are keeping their distance because of the above-stated performance issues, I hope you at least give the new version more of a chance.

Take care,
Sam Bacsa



[Footnote]
Test machine was a 3 GHz Intel Pentium P4 (HT) laptop, with 768mb on-board memory, running Windows XP Professional at 1,500mb static swap space.