Persistent Characters and MMORPG Network Architectures

#81 Nothing about MMOGs is simple or free.

On our little ragtag team we having a saying/tool: "I don't believe you." and then "Try my product!".
The first forces the person to prove their statement and saves the rest of us the time of researching it for validity (which we have to do if it's important or a good idea).
Then when the person has proof they say "Try my product!" and present it for scrutiny.

So...
http://www.mysql.com/products/database/cluster/faq.html
http://www.brokeassgames.com/
"Try my product!"

As for MySQL working for a Torque MOG, YES.
As for MySQL working for a Torque MMOG*, YES, but you will need some money and it will not be easy (MMOG isn't an easy task anyways).
*I assume at least 10K+ players online at one time?

Martian has some great points for SQL flavors and I can vouch for SQLServer's high volume.
He also mentions things like concurrency and a few others that are a MUST for today's persistent games.
Your DB will need to be able to handle it.
I stated above that I was "green" to large scale deployments.
This is half truth, half "bait".
My goal for this thread is to see if the community can agree on one likely solution to use with the Torque engine to save newer members time and stress.
Unfortunately, price is nearly always a factor to indie developers.
However, most solutions can be scaled and switching DB structures is actually alot easier than it sounds.
This could be a pain and time consuming if you are live but it is pretty easy to do.
If we outgrow MySQL it will show our project (Ruin Online) is a "success" for us.
At that point if the MySQL clustering can't handle the transactions I will be migrating to SQLServer on Windows.
Postgre on ubuntu would work well too and be alot cheaper, but I'm not very familiar with that setup.
I use what I know works well and I have seen SQLServer on a Windows box do what we may need down the road.
I chose MySQL because it was the next DB that I was really familiar with and our budget is nearly non-existant.
We are about to setup our player registration to be automated instead of moderated, this will help us stress test our DB.
Also remember that your total users won't be on at the same time.
We have 2000+ players signed up, only about %10 are really active because we are still early Beta and not a completed free MMO, and only about %10 of that are online at any given time.
Realistically I see 1-30 players online at any given time.
For our project they seem to come in droves (WoW or CS server goes down? I dunno?).
We also use the DB for quest style NPCs, mob spawns, serial number registration and many other things.
So when you are researching and crunching numbers give that a thought.
My percentages may not be suitable for your project, but try and figure out how many transactions you will need to do in a given time period and not just for characters, there will be other data you will need to access.
How your game's DB communication is coded is also a HUGE factor.
Less is more in this case.
Hope this helps shed a little light.
Maybe we need a list of possible solutions and their pros and cons, I'll check some other game dev sites to see if it's allready been done.
I'm glad I didn't post my complete architecture because it has changed drastically.
I'll be posting again soon (need to hit up Visio or something to make an architecture graphic).

Stefan, you said the dirty word, so it's siggles for you, lol!

Ari
"Industry is about money, gaming is about entertainment. I worry that when the two are mixed the original idea is lost. I've seen money and success ruin so many good developers. I think the original product should be created first so that some guy in a cornflower blue shirt doesn't screw it up. After 8 years in the industry, I have chosen to farm hard working newbies who haven't been tainted and then back them up with a few retired industry heavies. It's a gamer's wet dream. Experienced developers being the instrument of the community. You can't buy innovation, but you can sure farm it. If I wanted to make money, I'd buy some cheap mobile phone game and re-work it. Personally I think "The Industry" is a dirty word. So back to the original question, what makes games fun for you? And try not to use the word industry, game titles are fine though, they give me reference." Excerpt from BrokeAss Games recruiting session.

#82 From a developer of the Plane Shift MMOG.
http://www.planeshift.it/main_01.html

"Regarding ORACLE being the "best" database: We used to use ORACLE for our MMO/VW platform, but we switched to MySQL because it performed better (with innodb tables). We even had an expensive ORACLE expert DBA on staff to optimize the ORACLE DB for a year before switching, but with MySQL, we don't need that kind of healing hands.

Of course, we implement our own clustering to scale horizontally across multiple database instances, so we don't need ORACLE's clustering support. Then again, most people I've heard of using ORACLE clustering complain about it being basically unusable as soon as a single node goes down.

For the original question: A flat vector of structs in memory, that you checkpoint every now and then, is not a bad solution for certain levels of scale. Make sure you checkpoint to a temporary file, call fsync(), and then do a rename() and sync() (in UNIX terms) to replace the old file, as otherwise the checkpointed version may be corrupted if you crash/lose power in the middle of checkpointing.

The step up from that would be to use something simple like SQLite, which comes in source and embeds right inside your server process. It doesn't scale well across multiple processes, but that's probably not needed. Regarding multi-threading, you could put SQLite in a single thread, that the other threads issued asynchronous requests to, very similar to how MySQL or ORACLE client libraries use sockets talking to a separate process. It'll probably just be bound on disk access anyway."

This guy is a MySQL fanboy but I'm just trying to find real scenarios.
It's true alot of the large MMOGs use ORACLE, but in my experience it was hell to use.
I could throw a rock from the Sega of America (part of my hatred for the industry? You bet!) building to the Oracle Tower in Redwood City, CA. back in the day, so I'm not totally alien to it.
But it was so different from the way I'm used to doing DBs that I never could get the "groove" with it.
I do think it's a bit out of reach for most indie developers price wise.
Correct me if I'm wrong and there is inexpensive way to run it.
Now, for developing, Microsoft has a developer license that would allow a team to release and deploy a BETA (be very clear on the license!) relatively cheap with SQLServer (god I hope I not confusing something else with MS-SQLServer, either way, I mean MS SQL).
Once you deploy it will cost a bit for commercial and not dev licensing, but it's not impossible if you are really intending to deploy an MMO.
I think we need 3 grades of DB setups for Torque.
Free, low cost, and enterprise.
Please correct me if I'm wrong on any of these, I'm very proficient with SQL but by no means a DB guru.
I'm pretty happy with my setup, but it may not work for others, and just maybe I'm completely wrong about MySQL.
I don't know much about SQLite, but it's a very interesting option.
Anyone here know it's pros and cons?

Ari
"I like being wrong, it means I'm learning."

#83 Hey Ari,

My post here is from a dogs age ago but I'll toss my 2bits in on this one more time... Not to disagree with you at all and I dont think you really came across as saying MS SQL is "Da Bomb" but ...

In my database experience (Which is limited. I am not a guru either but I have built my own database backend five or six times now from the ground up with different databases including flatfiles, SQLite, MySQL and MSAccess) it is clear to me that when we are dealing with a database connector for an MMO it's all about minimizing the request/memory footprint and maximizing the query performance. Now your mileage may vary but after working on network optimization for over 15 years professionally I am very familiar with doing what it takes to "tune" up a windows box and more recently an MS SQL server.

It is my opinion that even optimized to the 9s, a full blown MS SQL server will ALWAYS have a larger operational overhead than a strongly configured MySQL box.

Your question above about cheapest, midrange and enterprize level servers is a legitimate one. IMO MySQL wins cheapest(free) and midrange hands down. As to enterprise, well lets be honest I don't know the actual statistics on what MySQL can handle. (In practice, not theory. Sure it can handle a bajillion querries a second but seriously, in production with a concurrent player base of over 10k.. I cant even imagine what the performaance would be.) I know that MS SQL was used to drive some database engines in the past but the games that used MS SQL often ended up blaming it for their engine woes.

At this time, untill someone proves me wrong, I am going to place MySQL as the leader for all categories including enterprise. If my game ever hits 10k concurrent players I wont even give a damn about what I have to do to improve the engine I'll just do it ;)

Mark

#84

Quote:At this time, untill someone proves me wrong, I am going to place MySQL as the leader for all categories including enterprise. If my game ever hits 10k concurrent players I wont even give a damn about what I have to do to improve the engine I'll just do it ;)

Word!
I have had a few crashes but after checking the logs it was bad setups trying to handle brute force attacks that I trace routed to China, go figure.
I guess some youngster got denied for beta and wanted a serial pretty bad?
Needless to say (and being an ISP admin I feel dumb saying this) put your DB behind a firewall and only allow game servers and VPN connections to access it.
Otherwise you may get stress tested early or worse. ;)

[mmorant]
I've never made an MMOG so I'm praying I can bait out some heavies into some hard fact insight.
Worst case, I'll learn the hard way.
And truth be told, I get teary eyed everytime I think about what GG and this community have done for me/us.
Let's hope we get a solid solution to pass onto the newer members.
Maybe someday we'll have a resource for setting up a DB for large scale use in Torque to handle an MMOG.
I realize it's a bit out of scope and the GG guys are working really hard, so maybe we in the community can cut away the fat and create something tangible.
I don't mean basic info on stored procedures, hello world of DBA or anything, just some hard facts on what works, how much it can handle in a real scenario, how much it costs and where to get it.
Looking back I realize why so many ppl warn against creating an MMO.
But the truth is nobody is listening to that and a solution needs to be found.
Heheh, I was told Torque was not designed for an MMO and I bought it and tried anyway.
Glad I did.
Josh Ritter gave my team and I hope.
I tire of click'a'click'a RPGs quickly since SOE's SWG CU and NGE (and yes EQ), but the bottom line is... if you cross the finish line first, you take home the trophy.
I'm glad he did or I may have given up.
MoM is a great example of realistic goals and execution for an indie game.
I played for some time (40/40/40 I think?) when I got burned out on developing, it helped remind me what we love most about standard MMORPGs.
It'd be nice to hear some hard facts about what he's seen in the field but last I heard he ran around GG and deleted his posts after he "made it", so I'm afraid to even ask. ;)
Also Dreamer's debated MMORPG Kit is seriously cool stuff!
I design alot of my stuff from resources here or from scratch, but the MMORPG Kit is always there when I need or forget basic MMORPG style structures.
Alot of times I check the kit before I go off writing code to see if they allready have done it.
They use ODBC and (so far I have seen:) MySQL or SQLite.
I haven't considered changing to SQLite for more than 30 seconds, but I really should ask them more about it.
I love the kit and the team so much I sometimes hang around IRC and help users with basic coding questions.
I even donated my favorite piece of code to it, the RPGBrowser, a very cool DB driven RPGDialog replacement capable of executing torque script from a DB that is editable during the game ie. realtime.
Someday I hope to release the tiny little resource (3 scripts), but until then, these guys are the only ones who have it available.
The Dream team is also full of nice and helpful people, I see them get jerked around alot by newbs and they are still smiling so I'm guessing they love what they do (living the Dream, heheh).
Alot of possibilities are out there, most of us get busy using them and forget to post (I'm guilty), but I'm most interested in the evolution of gaming and not trade secrets that aren't secret at all because they are what the market is craving and screaming for allready, so when I find time I hand back what I find.
I think the future of gaming is in indie developers and not the industry (I'll drop ID Software's name as a reminder of all our roots).
I do hoard some code from time to time, but I at least leave bread crumbs for those to discover the path I followed.
Or worst case, point the direction.
I try and list the most important commercial resources we use on our web site too (if I forgot you during the frenzy of development, drop me an email and I'd be glad to pay homage with a banner on our links page).
Anyways, some live results (ie. deployed) on Postgre and SQLite would be helpful to this thread.
[/mmorant]

My current setup:
A MySQL database w/ offsite backup (All game, cluster and security key servers talk to this, clients are not allowed, currently on P4 Compaq box.).
Web page access to various aspects of the DB (player account and soon character info and inventory/bank transactions).
2 Master Servers w/ clients/servers using both for failover. (Disabled until we have more than one instance of our multizoned world, was using perl script, soon to use master server integrated into engine resource)
IRC Chat Server (modified TGE Lobby + Dream Kit's custom message.cs, game servers also have the same IRC client for ability to control bots and create chat events).
Nexus Server (hack of our game server that reads MySQL character data and says you need to be on this other server and has you connect to it *functional and online still needs some slimming, controls zone to zone travel and persistence).
6 Zone Servers (dedicated game servers on a single DELL box using multi-homed IPs)

Ari

#85 Not clear what points some posters are trying to make, but if mysql outperformed Oracle in your tests
then you were clearly doing something wrong. Like not restructuring your sql, tables, tablespaces, etc
they way you need to in oracle. I would suspect a plain vanilla mysql may outperform a plain vanilla and unconfigured Oracle DB.(not a surprise really)

Ive worked with both for many many years, and mysql (which I never suggested was bad, on the contrary its a very good choice for indie developers) has always taken a back seat to Oracle at the enterprise level.

In many of my posts I specifically point out that many of my comments are geared towards my experience with very large scale MMO's.

mysql is not an enterprise solution by the way, sorry thats complete bunk. will it be someday? perhaps.

Quote:
@ The Martian : I define success on how pleased your developers, artists, and community is with your game. And just a footnote, there is nothing wrong with MySQL as a database solution for any torque product. One of the tricks is spreading your server load, which Broke Ass Games clearly outlines in the first post. Knowing when to open/close databases, what databases need to be open at all times, and periodically scheduled backups and comparisons should prevent any anomalies from occuring.

believe me I understand the many "tricks" of using a database in a MMO game. like I said mysql is fine, I didnt say it was wrong, I just brought up some issues I had run into using it and examples of problems others have faces attempting to use it on large scale sytems.

obviously you schedule backups and such while down for maintenance or during non peak times. you should also not be doing an deletes while the game is up and running (offline them or process them during non-peak times) there are tons and tons of tricks like that, those can be used on any DB you choose.

good for you that you found a decent free solution that works for you (at least your using a rdbms so kudos : )

my comments have always been directed at the big scale, millions of records, thousands of transactions per second. so you have to put all my comments into perspective. most of them do not apply to you, but that doesnt mean you can not learn from what we did to make things fast.

this dicussion shouldnt turn into a pissing contest about which DB is better. Its more of a question of which DB is better for what you are trying to do.

for me if I wanted to use a free DB I would pick postgres, in all our tests it easily outperformed mysql, but it requires a bit more knowledge and configuration then mysql and so isnt for everyone.

I noticed someone had mentioned clustering and how you only need to use oracle if you wanted to cluster and everyone complains about clustering.

so a couple things, one, i would never recommend anyone use the rack clustering from oracle, its problematic, and way too hard to work with for most people. But its certainly not the only reason you would pick something like Oracle.

tools, availability of knowledeable people, documentation, scalability, replication, advanced backup ability, and performance are some of the reasons an enterprise level company would pick such a product as oracle.

and your right everyone that I know that went rack cluster regretted it, and from what I can tell its because they thought you could just rack cluster and get all this super performance. The problem is you need to write your application with clustering in mind, you cant just through any application on a rack cluster and expect it to work correctly.

Quote:
At this time, untill someone proves me wrong, I am going to place MySQL as the leader for all categories including enterprise. If my game ever hits 10k concurrent players I wont even give a damn about what I have to do to improve the engine I'll just do it ;)

I have to completely disagree of course, sorry, but mysql is definately not a enterprise solution (depends on your definition of enterprise) but its by no means that type of product.

Ive worked at enough large companies to know how untrue that statement is. Just an example, how many financial insituitions(banks, sotck market exchanges, etc) do you think run on mysql?

out of all the ones Ive been too the number is 0, most run on DB2, Oracle, or SQLServer.
how many MMO games do you think(larger scale, over 200k subscriptions) run on mysql?
out of all the ones Ive contracted on, worked on, or shipped(thats 10 now), the number is 0.
(might be one out there is you know which one then please post it)

at one point in time I know daoc was considering using it but never heard if they moved to it or not, i believe they were still a custom flat file solution that they were hating.

I could go on but I think you get the point, when you talk about Enterprise level stuff, mysql is not even in the picture.

But in terms of web sites, small scale projects, indie MMO's etc, its probably and excellent choice, and your right if you ever get that 10k subs you mentioned then you can worry about it when that time comes.

which is pretty much my point as well, go with what works for now, what is free, and get out there and get players, using whatever you need, whatever works, your an indie developer, no one expects you to spend a zillion dollars on a mainstream enterprise wide RDBMS solution. Your expected to do it on the cheap and use your success to improve any weak areas later on.

thats actually an excellent strategy!

use what you feel comfortable with, but please dont make unfounded comments about your techonolgy choices. Comment on whether it worked, what problems you ran into, and what kind of load you are seeing is all useful to everyone.

but dont cheapen your success with unsubstantiated blasting of the other choices that other people right at this moment might be making, especially when you dont have anything to back it up.

all the RDBM's can all be good choices for the right project, but dont assume that because it works for a couple hundred to a couple thousand players that its automatically good for 20k, 50k, 100k or more.

Ive seen too many hardships over the years with people trying to use smaller scaled software on very large systems (which all different types of architectures and tricks) to know that not all RDBM's are good for every situation.

but in the end use what works and keep us in the loop on how it works out for you. and dont be afraid to ask questions if you run into trouble even if it reveals problems with your choice. Im not going
to give you the told you so junk, id rather help you out and get you back on track should you need it.

so no need to defend your choice, just make it happen, ask questions, and be open to new ways to do things.

and as always, good luck :)

#86 @TheMartian
Thanks for sharing your insight!

Quote:
mysql is not an enterprise solution by the way, sorry thats complete bunk. will it be someday? perhaps.

http://www.mysql.com/products/database/cluster/faq.html
I thought MySQL had grown up allready?

Quote:
most run on DB2, Oracle, or SQLServer

I just googled SQLServer to make sure it was just a case of tomatoe, tomatoe.
SQLServer = MS SQL Server, correct?

Quote:
this dicussion shouldnt turn into a pissing contest about which DB is better. Its more of a question of which DB is better for what you are trying to do.

Hehe, nope, just finding various views and challenging them to learn more.
The more I push, the more clear the picture becomes.
I have nothing negative to say about any DB.
Except that Oracle was hell for me to use, I also mention it was odd for me to get a good feel for.
So if the comment about Oracle seemed negative it just meant that I had trouble understanding it.

Quote:
but dont cheapen your success with unsubstantiated blasting of the other choices that other people right at this moment might be making, especially when you dont have anything to back it up.

Quote:
I noticed someone had mentioned clustering and how you only need to use oracle if you wanted to cluster and everyone complains about clustering.

Those were both from the post that I quoted from gamedev.net from a developer on the Plane Shift project.
I re-posted his comments to "stir the dragon" and find out more.
I thought that switching from Oracle to MySQL was a bit odd but didn't know enough about Oracle to fully understand why.
It seems that for enterprise level MS SQL Server may be the way to go for an indie like me (ie. cheaper than Oracle), but up to that point MySQL should handle just fine.
I'll research more about the new MySQL features and clustering technology, maybe by the time I need an enterprise solution, it will be ready for me.
Anyways, thanks for the input, I'm just trolling around for solid input from experienced users and every bit helps. :)
Also, the post way up near the top had some very good info about DB structures for MMOs.
For anyone who is new to DBs and MMOs read it very carefully.
Ok, the picture (at least in my case) is really starting to take shape. =D

Ari

#87 This would apply I think. Mythic Enterainment (Dark Age of Camelot):

Quote:
Because running Camelot would require a considerable amount of data management, we initially planned on using Oracle to store account and character information. However, Oracle's quoted license fee of more than $900,000 quickly removed them from contention. Once we got over our shock and amusement at Oracle's pricing, we turned to a Linux-based freeware solution, MySQL, to manage Camelot's data storage, which so far has worked admirably.


Everyone developing games should at least investigate open source solutions for their servers. It's saved us a pile of money and has been stable and reliable. In fact, prior to Camelot's launch, it was axiomatic that MMORPGs were unstable and prone to crashing during their first month or so. From the outset, we were determined to buck this trend. We co-located our servers directly at UUNET, on the network backbone, which ensured a wide network pipe to the Internet. With this Internet connection, we can increase our band-
width with just a few hours' notice to UUNET.

www.mysql.com/customers/customer.php?id=254 -> www.gamasutra.com/features/20020213/firor_pfv.htm

#88 No need to get your back up Martian.. Your opinions are free to be posted as everyone elses.. Can you point out which indie developer here at GG has 10k to get into the entry level Oracle license BTW? or God forbid they release a product and have to suddebly pay 100k, 250k or 1 million in licensing to Oracle?

If you like Oracle and you own or can afford a license then Bravo.. I wish you all the best with it. As has been pointed out, MySQL handled DAoC very well. Not sure what your definition of enterprise is, nor do I care.

Here is a fact.

MySQL performs it's querries more efficiently (memory wise, processor cycle wise and millisecond wise) than Oracle. substantially more so than MS SQL. MySQL is also free to develop with and (within the confies of the license) free to release with which suits most indie developers just nicely.

If you disagree with people using MySQL thats fine but dont bash people for making a choice based on what best suits their needs. I've worked in the gaming industry for over 7 years and I couldnt be happier with MySQL.

I tell you what... If my game ever gets bigger than DAoC and I'm unable to handle my DV with MySQL any longer I'll gladly concider some other DB engine. Till then I think you need to chill bro.

#89 He's saying a well configured Oracle database is more efficent than mySQL. That's not bashing. It's no coincidence that most big companies use Oracle or some other enterprise package for their realtime simulations.

Quote:
Can you point out which indie developer here at GG has 10k to get into the entry level Oracle license BTW? or God forbid they release a product and have to suddebly pay 100k, 250k or 1 million in licensing to Oracle?

If you like Oracle and you own or can afford a license then Bravo..

You changed direction. It was mentioned that mySQL was faster/whatever than Oracle. TheMartian argued the opposite. And now you're talking about price. Seems like you're getting a little bit worked up.

#90 The post about DAOC and that oracle quote is pretty old, currently Ive priced Oracle out and received the same price as SQLServer. It doesnt cost that kind of money anymore to run Oracle. Especially if you going to use dual or quad boxes. Its still not really "Indie" affordable.

I remember that DOAC thing, they wound up writing there own flat file database and they really wanted to get rid of it and move back to a RDBMS. Not sure if they ever did but they certainly wished they had gone with one in the first place.

oh, thats another reason to not use oracle rack cluster, the licensing on it is outrageous.

as for being happy with mysql, thats wonderful, but you missed my point, ive seen it work great on small systems, smaller data sets, smaller loads. So sure its definately possible for you over the past 7 years
to enjoy working with mysql.

But I just had another large scale company contact me and present the same problem Ive seen over and over, you ramp mysql up with a ton of data and high transaction rates and it falls flat on its face, again.
we ran the same application with Postgres and it handled the same dataset and the same transaction
rates perfectly fine. so yes another company is migrating off of mysql and onto postgres.(see its not always about oracle lol)

as for mysql coming into its own, it sure is, they are doing more and more enterprise level things, but for mysql its all brand new and as of yet unproven. Other RDBMS's have had similar technology for over a decade.

but it is coming along, at some point ill get to do some more testing of there latest technology updates and see if all the problems have been flushed out.

Quote:
I tell you what... If my game ever gets bigger than DAoC and I'm unable to handle my DV with MySQL any longer I'll gladly concider some other DB engine. Till then I think you need to chill bro.

guess you didnt read my whole post. but thats ok. Like I said im happy if it works for you, im not mad or upset and saying you cant make it work, of course it can work. My point was Ive seen way too many instances where mysql failed as an enterprise solution. so to say its ready for the big time is questionable in my opinion.

Is it ready for an indie mmo game? certainly. so no argument there, never was arguing this point anyway.

Plenty of other DB's work fine on big systems, ive worked with huge SQLServer systems that handled it fine as well as large Postgres DB's(this is a really nice free DB btw).

Quote:
Can you point out which indie developer here at GG has 10k to get into the entry level Oracle license BTW?

again you didnt read my whole post, I already stated several times that Oracle was not an "indie" product.

although oracle is free for 1 cpu. (check there free version out). But im not so much promoting oracle, I am actually a much bigger proponent of Postgres anyway especially since it has similar characteristics of Oracle and its free.(and if you want oracle features for less money check out enterpriseDB that uses postgres as its base).

the only thing I have trouble agreeing with is the statement that mysql is a enterprise level solution that outperforms oracle.

Quote:
MySQL performs it's querries more efficiently (memory wise, processor cycle wise and millisecond wise) than Oracle. substantially more so than MS SQL

and thats based on what? some labatory test done at mysql headquarters? my tests always involved millions of records, hundreds of tables, and gigabytes of data, you run a comparison of mysql and oracle in that type of environment? how about with thousands of transactions a second? how about on 2, 4, 8, 16 cpus? mysql scale as well as Oracle on bigger hardware?

those are actually honest questions and not intended to be inflammatory, I can answer some of those questions for you but im curious if anyone else has ever run large scale serious tests or not.

(my results)in some of the tests mysql does indeed do better, but there are plenty of other tests where it was very weak.

but its all academic really, as indie developers its not really a concern, just about anything will work for you, crin out loud flat files are the fastest solution and for small scale systems works prefectly fine.
so it really depends what you are comfortable with.

no need to get mad at me, use whatever you want, if it works for you for now thats awesome, and if its free on top of that wonderful. sharing your experiences is how everyone can learn, it benefits everyone to hear what we have all learned, experienced, and tried out. Which is sort of the whole point of an indie forum site right?

#91 Fwiw, our project actually could and did afford oracle, and we ended up scrapping it for mySql. I'm a 3D guy, not a DB or backend guy, so i'm not entirely clear about the reasons, but i think it's because it was difficult to find a quality DBA to run the thing, and it tended not to play nicely with whatever other packages we had linked up to it, ala hibernate. i believe we've currently got mySql going on a cluster of machines each with multi CPUs. at least, i know it's a cluster, and i'm assuming multi CPU's. ;)

#92 Hey cool, yeah i see that alot, oracle is more complicated to setup, configure, and tune and so it takes a bit more knowledge to get it right, so for example default installations of oracle are almost useless, you really have to get under the covers and tweak it. which I admit makes it a bit harder to work worth.

Id be really interested to hear how the clustering with mysql works out (would be awesome if they get it working well, clustering is some very powerful functionality and to have it free with a free DB is even better).

very cool, keep us up to date on how it does as you load increases would be nice to see some real world
data on its use in a production environment.

#93

Quote:I remember that DOAC thing, they wound up writing there own flat file database and they really wanted to get rid of it and move back to a RDBMS. Not sure if they ever did but they certainly wished they had gone with one in the first place.

Er, they never went flatfile with DAoC, you must be thinking of something else - it's as it says, they tried Oracle, found it too expensive, switched to MySQL and have stuck with it ever since, DAoC has been running on MySQL since pre-beta with absolutely no problems.

As BrokeAss Games said, MySQL HAS grown up already, in fact it grew up a long, long time ago. MySQL has a proven track record as the persistent storage backend for an MMO as above in terms of DAoC.

Quote:mysql is not an enterprise solution by the way, sorry thats complete bunk.

HP would disagree with you and seeing as they're one of the biggest tech companies in the world and this is a core area of their business then frankly, you're wrong. Oh and uh, we probably shouldn't forget that Google uses MySQL as their DB backend for most things (if not everything?), certainly for Adwords and such, I'm hoping Google fits into your definition of "enterprise"?

Sorry but if MySQL is good enough for Google, good enough for HP and has a proven track record of being used for a very successful MMOG for over 5 years (including beta there) then it's good enough for any indie wanting to produce an MMOG and there's really no reason to go with anything else when it seems to have the widest ranging set of resources available and doesn't cost a penny.

#94

Ok, well, I kept telling myself not to interject as I usually come across very strong and sometimes it can start a flame war, but enough is enough.

Why is it that some of you are taking tones such as:

Quote:Oh and uh, we probably shouldn't forget...

Just because a forum is a place for opinions doesn't mean you need to take negative tones with people because you disagree with them. Act like an adult and state your opinions in a nice fashion for the love of Tolkien people!

It pains me to see someone like martian stating his OPINION based on his EXPERIENCE with the subject at hand and have people completely misinterpret what he says. He can't state "mySQL is great for indies" any plainer than he has can he? Why do people keep going back to:

Quote:Sorry but if MySQL is good enough for google.. then it's good enough for an indie wanting to produce an MMOG....

Haven't you been listening? HE SAID IT'S A GOOD CHOICE FOR INDIES. He mentioned other free RDBMS's that were as good or better, IN HIS OPINION, than MySQL, but he NEVER stated that MySQL wasn't good enough to use in an indie invironment.

As far as companies like google using MySQL, try reading the articles where they discuss this. They BUILT adwords on MySQL because it was free and even so, "it did take some extra programming to work around one of these missing features."

Lots of enterprise applications start out using free software and upgrade when the time comes. That doesn't make it an enterprise solution. The guy who helped build AdWords even claimed that, although now MySQL is "nearly as full-featured as the best commercial databases", he said it was nowhere near the other RDB's at the time he used it. Try posting links to where you get your information from now on or state it as OPINIONS: xooglers.blogspot.com/2005/12/lets-get-real-database.html

In reference to DAOC, they never really topped 250k active subscriptions. The fact that they have roughly 22 servers with at most 1k people on the server, it wouldn't be hard to imagine that each shard has it's own mysql database, which would mean that the total transactions PER SECOND is nowhere near hte millions that martian is talking about. This is all on the DAOC game website for verification.

Stop acting like just because some press release or news article by MySQL comes out, that it's shi* doesn't stink. Anyone can make ANY database look like it's being used in a spot where it really can't perform.

Most of all, STOP acting like people who say "hey, this is what I experienced, and I disagree with you" are scum and deserve to get flamed!

EDIT: Oh yea, and the HP bit? You serious? HP is a BUSINESS who sells COMPUTER SOLUTIONS. It's their JOB to find ways to make more money, and that's all that article shows, them "partnering" with MySQL so they can get people to buy more of their crappy computers with MySQL on them thinking it's a good alternative for their needs. (maybe it is, maybe it isn't, but HP could care less).

Just because you are a really large company, does NOT imply you make good decisions or know what the hell you are doing. Dell is EXTREMELY profitable and large now and they make absolute CRAP computers. Or at least they did when I was in college and worked at CompUSA's tech department. We got more Dell computers into the shop than ALL other computer manu's put together.



edited RDMS to RDBMS (doh)

#95 Martian: Sorry mate. I took a little offence to the part of your post about people shouldnt be suggesting MySQL over Oracle and then that we werent basing our suggestions on tests..

Anyways we differ on opinion on some things. You dont think MySQL is a viable enterprise solution.. thats cool. I dont really know or care what a defined "enterprise" solution is but I know that the guys I have been working with for the last couple years on Database work, worked with NASA on their Oracle DB and my tests were based on their feedback on Oracles performance parameters. My MySQL parameters are of my own doing.

Anyways I didnt mean to bash you up and one thing about the GG forums that I appreciate is the massive ammount of real world experience that people share here. So I certainly dont want to be party to gagging someones experience =) Reversly though I will fight to the death to back up my own experiences and testing if for no other reason that I want to learn where I might be going wrong. I spent the last 2.5 years working on MySQL backends and optimizing my querries and I love it.

Again, sorry for any hard feelings.

#96 Amen to that Jonathon.

Also, Google and especially not HP do not fall under the category of "realtime simulations".
The comparasion of these two against MMORPG's is IMO silly. No one will notice a slight delay from the database on a website.

#97 Hey guys, its cool, we are hopefully all here to learn from each other experiences.

as for the DAOC thing, the reason I mentioned there solution is because I sat next to the lead database guy when we did our presentation "database challenges facing MMO games" a few years ago at a game conference.

He clearly stated to the whole audience that they had built a custom home grown flat file database system
and were looking to migrate to something like mysql.

So thats where my comments come from.

and frankly Im actually reall excited about the possibility that mysql has finally grown up, if it can do clustering well, and improve there scalability for really large highly active database then I would be all over it.
Who wouldnt considering its free :)

for my next game im certainly going to give it a fair shake.

but I just noticed recently that Oracle was end of lifing 8i, which I believe means you can use 8i for free on any hardware. Have to look into it, if thats the case Ill probably stick with Oracle for the next project, but it not the case ill definately re-review mysql and postgres again.

oh and as for google, very little of there stuff is in mysql. they also are not concerned about reliability thats why they put all there stuff on cheap hardware that fails all the time, they just keep copies all over the place so they dont lose any data, very different architecture then you would be looking to do with a MMO persistent world game.

bummer cant find the link to the article, (but it was on slashdot) a link to a really nice write up of how google works and how they architectured their systems and how its changed over time.

almost there entire system is flat file.(very complex distributed flat file system, very little rdbms).

and I whole heartedly agree, big companies dont often make very good technology decisions(at least not all the time) I lost count of how many comanies Ive seen or worked at that selected techonolgy based on what the marketing guy said and not based on what any of the technology/programming/software architects said.

companies make decisions for market position, public relations, exposure, stock price, to get bought out, etc.
they will buy good techonologies only to put the purchased place out of business and replace the good tech with there own crappy tech.
just the annoucement of using a technology, merging with someone, etc can cause the stock to jump up.

but by no means is that an indication of good techonolgy decisions lol, boy do I have some stories in that area.

some companies even make annoucements like that to draw interest and dont even use the tech they mentioned in their annoucements. and thats just the beginning.

so I wouldnt recommend using other companies PR press releases as justification for anything.

lets stick to real world hands on testing, implementations, and experiementations that we the worker bee's actually do and stay away from the marketing hype if we can :)

#98

Quote:Lots of enterprise applications start out using free software and upgrade when the time comes. That doesn't make it an enterprise solution. The guy who helped build AdWords even claimed that, although now MySQL is "nearly as full-featured as the best commercial databases", he said it was nowhere near the other RDB's at the time he used it.

Yes but then he also goes on to state that when they did try another commercial RDBMs that it was a bad decision and since then Google has gone back to MySQL. I'm not really sure how you can bitch at me when you spout half-truths like that to try and somehow prove me wrong. The fact is if it could serve Adwords what, 5, 6 years ago? then it's not hard to see how well it can do now (particularly as it's back serving Adwords). Back then MySQL was the weaker database yet it still performed

Quote:In reference to DAOC, they never really topped 250k active subscriptions. The fact that they have roughly 22 servers with at most 1k people on the server, it wouldn't be hard to imagine that each shard has it's own mysql database, which would mean that the total transactions PER SECOND is nowhere near hte millions that martian is talking about. This is all on the DAOC game website for verification.

Wrong again, DAoC servers have during the game's peak often served successfully 4k players per server, yes the game is dieing now because it's an old game, however that doesn't somehow magically mean that it never hosted those numbers - it did, I was one of them.

Quote:Just because you are a really large company, does NOT imply you make good decisions or know what the hell you are doing.

Obviously, these companies got large, rich and famous by magic, not like they'd ever need to know what they're doing of course.

Quote:Also, Google and especially not HP do not fall under the category of "realtime simulations".
The comparasion of these two against MMORPG's is IMO silly. No one will notice a slight delay from the database on a website.

I agree to an extent, I was merely pointing out how wrong TheMartian was in his comment about how MySQL isn't an enterprise solution, he couldn't be further from the truth nowadays, if he'd came and said that years ago though I'd have probably agreed. The part I disagree with you on though is your comment about comparisons, websites have the potential to handle many more transactions than any real time simulation so there are some parallels there at least because a popular website has to handle a much larger load than just about any online game out there currently.

Anyhow, Jonathon, why are you questioning people's tones when your tone is extremely aggressive in itself? It's generally a good idea to practice what you preach.

Still, I'm going to bow out here, I only posted originally to post some counters to what are frankly dated views of MySQL. No one's ever going to agree on what the best RDBMS is - it's like the old OpenGL vs. DirectX argument or the Linux vs. Windows argument, it's down to personal choice at the end of the day so it would probably be more sensible if people kept their personal opinions about different RDBMS out of the thread in the first place (and yes I'm probably just as guilty) but anyhow it seems too late for that now so I'll leave this thread to the zealots.

#99

Quote:Yes but then he also goes on to state that when they did try another commercial RDBMs that it was a bad decision and since then Google has gone back to MySQL.

I misquoted nothing in my statement about Google. You, however, just did. The guy who made that comment doesn't even work for Google and hasn't for many years. He said he HEARD that they went back, but wasn't sure. Practice what I preach? Yea.

Google moved to Oracle from MySQL. The porting was probably done by people who aren't efficient in Oracle, so set it up incorrectly by simply mocking what they had set up in MySQL. Moving from anything outside of postgres to Oracle is a huge undertaking and requires people to really know their Oracle stuff to accomplish effectively. Had they started with Oracle, maybe we'd be hearing a different story about it's speed. I'm sure Martian can vouch for that.

Quote:Wrong again, DAoC servers have during the game's peak often served successfully 4k players per server, yes the game is dieing now because it's an old game, however that doesn't somehow magically mean that it never hosted those numbers - it did, I was one of them.

The funny thing is, you keep telling everyone they are wrong, yet you have no proof of your accusations. Where's your proof of their workload per server? I showed you mine. It's not hard to imagine, with the current number of servers, that DAOC had attempted to keep the same player/server ratio that they have now as most MMOs do. I made an assumption based on current information that I linked you to. Stop spouting that you know what you are saying when you have no proof to back it up.

I've done many searches the past several days to attempt to find numbers for DAOC and have come up with nothing solid. Where do you get yours from? Saying 'I know because I was there' doesn't count as fact. Opinions vary.

Quote:Obviously, these companies got large, rich and famous by magic, not like they'd ever need to know what they're doing of course.

You are right, Enron did everything correctly and knew what they were doing. What was I thinking? People wouldn't possibly be in business to make money with the least cost to the company.

Quote:Anyhow, Jonathon, why are you questioning people's tones when your tone is extremely aggressive in itself? It's generally a good idea to practice what you preach.

My tone was aggressive, not disrespectful. There is a HUGE difference. I also warned at the top of my post that I'm like that =P.

Quote:it would probably be more sensible if people kept their personal opinions about different RDBMS out of the thread in the first place

The whole point of this thread is to show your personal feelings about the different DB's, so why would we stop posting them?

#100 This thread needs to die.