by date
S3 will revolutionize content distribution
S3 will revolutionize content distribution
| Name: | Dreamer | |
|---|---|---|
| Date Posted: | Mar 15, 2006 | |
| Rating: | 4.0 out of 5 | |
| Public: | YES | |
| Comments: | YES | |
| RSS Feed: | or Subscribe with . | |
| Profile Page: | View profile page for Dreamer |
Blog post
Hello everyone,
I've been evaluating methods for easily and cost effectively pushing updates to clients applications.
My options thus far have been SVN, FTP, Bittorrent, and a whole host of others.
FTP was my original thought, but my own experience with FTP in terms of controlling access, and the amount of load it places on the server has shown that this would be at best a suboptimal solution.
Next in line is Bittorrent, this is a fantastic protocol, basically everyone who wants your content has to share that content with others to recieve more content, while it does save you from placing excess load on the server, again access control is limited, also if you have to push multiple updates/bug fixes you may end up with stale patches and/or incomplete downloads. Furthermore setting up a proper bittorrent tracker or seed has shown to be a little problematic in my experience, and some hosts or even ISP, will even shape your traffic or possibly cut you off if you try to host one.
Thus far SVN has looked to be the natural winner, since each update is really only a diff from the previous revision, thereby reducing bandwidth on the server. And yes SVN can be used for any collection of files.
However controlling access to SVN could be time consuming and/or buggy and if a large number of people all need updates at once, your server will still be swamped.
Recently I read an article on a new service from Amazon called S3.
If you haven't heard about it before, this awesome new service allows you to store unlimited amounts of data for $0.15 per gig per month, and then $0.20 per gig of bandwidth for access to those files.
While at first this may not sound like a great deal, or even of any relevance or importance to you check this out.
You can make the content readable by anyone, or only a select list of people, through an easy to use ACL (access control list), which you can update on the fly.
Furthermore bittorrent is an integral option and using it will drastically reduce the amount of money you spend on bandwidth, since you only spend for actual data transfered and not network overhead.
This essentially allows you to upload any file or set of files, and have automatic bit-torrent capability. Without any of the drawbacks.
But what about the versioning capabilities of SVN?
Well here's the interesting thing, the S3 API descriptions mention that you can associate metadata of up to 4k with the file, and part of this metadata is accessible directly through the filename via a key value pair.
What does all of this mean?
It means that you could upload your data to your S3 storage service and use it for an update and distribution server.
For instance lets say you modified the root animation file player_root.dsq
After it is uploaded you associate some information with it for instance a time stamp.
You would wind up with a file you could easily distribute to any of your customers via any bittorent client, and you would have the added security of an access control list. You would also have the advantage of SVN in that if you later decided you made a mistake in the animation or whatever, you could roll back to the earlier version of the file simply by deleting it.
For me anyways this is great news, and as soon as I launch the MMORPG Enhancement Kit and the newest version of http://www.mydreamrpg.com/ I intend to begin creating tools to facilitate this.
I hope you find this exciting.
Regards,
Dreamer
p.s. For more information on S3 click here
I've been evaluating methods for easily and cost effectively pushing updates to clients applications.
My options thus far have been SVN, FTP, Bittorrent, and a whole host of others.
FTP was my original thought, but my own experience with FTP in terms of controlling access, and the amount of load it places on the server has shown that this would be at best a suboptimal solution.
Next in line is Bittorrent, this is a fantastic protocol, basically everyone who wants your content has to share that content with others to recieve more content, while it does save you from placing excess load on the server, again access control is limited, also if you have to push multiple updates/bug fixes you may end up with stale patches and/or incomplete downloads. Furthermore setting up a proper bittorrent tracker or seed has shown to be a little problematic in my experience, and some hosts or even ISP, will even shape your traffic or possibly cut you off if you try to host one.
Thus far SVN has looked to be the natural winner, since each update is really only a diff from the previous revision, thereby reducing bandwidth on the server. And yes SVN can be used for any collection of files.
However controlling access to SVN could be time consuming and/or buggy and if a large number of people all need updates at once, your server will still be swamped.
Recently I read an article on a new service from Amazon called S3.
If you haven't heard about it before, this awesome new service allows you to store unlimited amounts of data for $0.15 per gig per month, and then $0.20 per gig of bandwidth for access to those files.
While at first this may not sound like a great deal, or even of any relevance or importance to you check this out.
You can make the content readable by anyone, or only a select list of people, through an easy to use ACL (access control list), which you can update on the fly.
Furthermore bittorrent is an integral option and using it will drastically reduce the amount of money you spend on bandwidth, since you only spend for actual data transfered and not network overhead.
This essentially allows you to upload any file or set of files, and have automatic bit-torrent capability. Without any of the drawbacks.
But what about the versioning capabilities of SVN?
Well here's the interesting thing, the S3 API descriptions mention that you can associate metadata of up to 4k with the file, and part of this metadata is accessible directly through the filename via a key value pair.
What does all of this mean?
It means that you could upload your data to your S3 storage service and use it for an update and distribution server.
For instance lets say you modified the root animation file player_root.dsq
After it is uploaded you associate some information with it for instance a time stamp.
You would wind up with a file you could easily distribute to any of your customers via any bittorent client, and you would have the added security of an access control list. You would also have the advantage of SVN in that if you later decided you made a mistake in the animation or whatever, you could roll back to the earlier version of the file simply by deleting it.
For me anyways this is great news, and as soon as I launch the MMORPG Enhancement Kit and the newest version of http://www.mydreamrpg.com/ I intend to begin creating tools to facilitate this.
I hope you find this exciting.
Regards,
Dreamer
p.s. For more information on S3 click here
Recent Blog Posts
| List: | 01/18/07 - Shameless Plug 11/29/06 - Infuzor Installer and Autopatcher now shipping! 11/07/06 - We Got A New Look! 11/01/06 - MMORPG Enhancement Kit 1.5 Halloween Release 05/13/06 - MMORPG Kit Full Release Day is Here! 05/12/06 - MMORPG Kit Release is Imminent! 05/01/06 - MMORPG Enhancement Kit Screenies (56k Warning) 03/15/06 - S3 will revolutionize content distribution |
|---|
Submit your own resources!| Phil Carlisle (Mar 15, 2006 at 13:08 GMT) |
I was going to write a blog entry about using SVN for updating customers, as I see this as the quickest route to keeping people synchronised initially.
There *must* be a better option though.
Phil.
| Brian Marshall (Mar 15, 2006 at 14:20 GMT) |
So fire away, what would people be looking for in a distribution-handler?
-Brian
| BigPapa (Mar 15, 2006 at 14:28 GMT) |
| Prairie Games (Mar 15, 2006 at 14:33 GMT) |
| Dreamer (Mar 15, 2006 at 15:00 GMT) Resource Rating: 5 |
@Everyone, This is just a much, much cheaper way IMHO to distribute infrequently updated content like game patches etc.
Think about it, if you have 200+ users who all need a 50MB patch thats several gigs of data your server has to handle, you are either going to need a server farm to handle it or offload that stress to someone else.
Also your customers may take a break from playing for a week or two and come back, they may need 2-3 patches to get brought up to date, this allows you to get those to them for very little cost.
By my estimation if you used this it would save the need to purchase extra servers just to handle updates, which may be something that is only pushed once every week or so but would otherwise be sitting idle.
| Phil Carlisle (Mar 15, 2006 at 18:40 GMT) |
Of course your idea of the S3 thing would work equally well, but it doesnt currently have that functionality, whereas SVN does.
It depends I guess on wether you already have servers and extra bandwidth on them. I've got spare capacity on current provision, so SVN is ideal for me. Your case might be different (and yeah, I take your point about the cost of the data transfer being pretty small for S3).
I just dont want to have to develop anything to have this system work, SVN does it as a solution with no effort at all on my part (at least initially).
Phil.
You must be a member and be logged in to either append comments or rate this resource.


4.0 out of 5


