Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
Games Hardware

An Inside Look At Warhammer Online's Server Setup 71

Posted by Soulskill from the hate-to-see-the-power-bill dept.
An article at Gamasutra provides some details on the hardware Mythic uses to power Warhammer Online, courtesy of Chief Technical Officer Matt Shaw and Online Technical Director Andrew Mann. Quoting: "At any given time, approximately 2,000 servers are in operation, supporting the gameplay in WAR. Matt Shaw commented, 'What we call a server to the user, that main server is actually a cluster of a number of machines. Our Server Farm in Virginia, for example,' Mann said, 'has about 60 Dell Blade chassis running Warhammer Online — each hosting up to 16 servers. All in all, we have about 700 servers in operation at this location.' ... 'We use blade architecture heavily for Warhammer Online,' Mann noted. 'Almost every server that we deploy is a blade system. We don't use virtualization; our software is somewhat virtualized itself. We've always had the technology to run our game world across several pieces of hardware. It's application-layer clustering at a process level. Virtualization wouldn't gain us much because we already run very close to peak CPU usage on these systems.' ... The normalized server configuration — in use across all of the Mythic-managed facilities — features dual Quad-Core Intel Xeon processors running at 3 GHz with 8 GB of RAM."
This discussion has been archived. No new comments can be posted.

An Inside Look At Warhammer Online's Server Setup More

An Inside Look At Warhammer Online's Server Setup

Comments Filter:

  • 70 * 16 ? Are they kidding me? (Score:2, Insightful)

    by Anonymous Coward

    They have like eight servers (worlds) up, tops. They closed all the other ones down. There's no way they have 1120 servers running.

    • Re: (Score:1, Informative)

      by Anonymous Coward

      I just checked. Server status page shows 7 servers, down from the 15 that remained in the middle of March 2009 after they shut down 63 servers.

    • Re: (Score:3, Insightful)

      by SCPRedMage ( 838040 )
      That's PHYSICAL servers. A single game "server" is made of multiple PHYSICAL servers.

      • You don't need a shitload of *physical* servers when you have no gameplay servers.

        • Re: (Score:1, Insightful)

          by Anonymous Coward

          Sure you do, if you're using archaic server software from the days where DAoC was born.

  • At least gamasutra labels their paid placements (Score:5, Informative)

    by sleeper0 ( 319432 ) on Thursday December 31, 2009 @02:52AM (#30603456)
    Sponsored Feature: Restless Entities Never Sleep -- The Back End of Warhammer Online: Age of Reckoning

    New features enhance the overall gaming experience, such as Intel Turbo Boost Technology (to maximize speed for demanding applications), Intel Hyper-Threading Technology (for advanced multi-tasking and support for up to eight threads), and Intel Smart Cache (to provide a higher performing, more efficient cache subsystem). Experience Warhammer Online in its best light with the processor that has become the gold standard in the gaming world, the Intel Core i7 processor Extreme Edition.

    • Re: (Score:3, Insightful)

      by _Sprocket_ ( 42527 )

      From the beginning of the article:

      [In this Intel-sponsored feature, part of the Visual Computing section, the technical experts behind Mythic and EA's Warhammer Online discuss the mechanics of keeping the MMO running across multiple servers and data centers.]

      So yeah - if you start picking up on the glossy brochure language... it's because it is, in fact, marketingspeek. But burried in all the "gosh golly isn't this product swell" blather is some interesting little tidbits about the environment; wheat from chaff.

      • Re: (Score:3, Informative)

        by sleeper0 ( 319432 )
        Ahh, I was trying to figure out how I missed that - turns out the link to the article starts you on the second page where the disclaimer is nowhere to be found. Not that it's a dig on gamasutra as it's clearly stated in the title. Just wish /. had mentioned it before I read most of the way through and was wondering why I felt like I was reading and intel ad.

    • I though it was just my cynicism kicking in at first. Glad someone else noticed that this whole thing reads like a big advertisement for Intel... It is.

  • Article summary (Score:3, Insightful)

    by chanrobi ( 944359 ) on Thursday December 31, 2009 @03:13AM (#30603546)
    Blah blah blah, intel cpus are the best, blah blah blah. Reads like one giant intel advertisement

    • Re: (Score:3, Funny)

      by iamapizza ( 1312801 )
      Be thankful it isn't another iphone advert.

    • Re: (Score:2)

      by Bengie ( 1121981 )

      I love my i7 to death, but wouldn't AMD's new 6-core CPU be better since it uses overall less power to do the same amount of work, even if it takes a bit longer to complete the work.

  • Virtualization (Score:5, Insightful)

    by Fractal Dice ( 696349 ) on Thursday December 31, 2009 @03:34AM (#30603610) Journal
    I've always felt virtualization was pushed too heavily as a concept, making up for failings of the operating systems and how they are used. Many admins have become so used to consolidating existing legacy servers together that we start thinking every app should be wrapped in its own private copy of an OS - as if the whole point of the concepts of users and groups and limits have been forgotten.

    • Re: (Score:2, Interesting)

      by PhrstBrn ( 751463 )

      Sometimes virtualization makes sense for an app where you don't really know exactly what the usage requirements are going to be. You know at first, your app not going to need a full machine to run. So you wrap it in a VM, and throw it onto a shared server. But you think, in the future, you're going to need to scale up to bigger and better hardware. But you're not sure.

      If the app is already contained in a VM, it's trivial to just move it from Server A to a Bigger Server B if you need more power. The pro

      • Re: (Score:2, Insightful)

        by Anonymous Coward

        It is also great when you have an old server that has been running for a long time on piece of crap hardware that is about to die. You can just do P2V and not worry about trying to transfer data over, especially those ones that have been in production for years and reinstalling software is not feasible.

      • But you don't need virtualization to do that really. If you've got a smaller app then put several of them on a single server. If it needs a bigger server later, then put the app on a bigger server. The advantage of migration with VMs is that the apps can be dumb and not know how to save/restore their state if you move them. When you get to the point where you've got one process per VM, you end up wasting a lot of resources; more memory usage, longer context switches, more paging.

        What it really does is s

    • Re: (Score:2, Interesting)

      by Opportunist ( 166417 )

      I was thinking the same.

      Basically, enabling legacy applications to survive by giving them a slice of a real machine and running them that way is a great crutch. But not more. It would be more efficient to revamp the system and bring it up to contemporary code, but often that's not possible. I blame closed source and companies that wrote it going out of business, but that's me... I could ramble about shortsighted management decisions and putting the life of a company on the line and dependent on the existanc

      • Re:Virtualization (Score:5, Insightful)

        by dkf ( 304284 ) <donal.k.fellows@manchester.ac.uk> on Thursday December 31, 2009 @06:46AM (#30604034) Homepage

        Basically, enabling legacy applications to survive by giving them a slice of a real machine and running them that way is a great crutch. But not more. It would be more efficient to revamp the system and bring it up to contemporary code, but often that's not possible. I blame closed source and companies that wrote it going out of business, but that's me... I could ramble about shortsighted management decisions and putting the life of a company on the line and dependent on the existance of another company, but ... I won't.

        The open/closed status of the program code has got nothing to do with it. We use virtualization with plenty of code where we have the source (either open source or written in-house) because that lets us greatly improve the utilization of hardware. Why take up a whole rack of servers to do what one modern blade can cope with? (Most servers aren't CPU-bound.) What's more, it can do this without you having to figure out how to get all those silly deployments to work together nicely.

        The other good thing about virtualization is that it lets many people have control over their own machines without needing lots of "servers" under desks. That means you can do things like ensuring that everything that the business really depends on has UPS power and sane networking. (I know. Critical stuff shouldn't be put under someone's desk. Virtualization makes it easier to bring reality closer to that ideal.)

        Virtualization isn't perfect at all, but it does cure a bunch of problems that crop up in reality and at far less cost than "doing it properly". (For one thing, it's not cheap to build a new datacenter. Even fitting out a new server room isn't something that you want to have to spend on every day.)

        • The next question might be, why does each instance of your in-house code need a private OS to function? At least, that's what I wonder each day at my own workplace. Virtualizing UNIX systems doesn't make a whole lot of sense to me. Containers make a little more sense, but why is it so hard to run multiple applications on one UNIX system now?

    • Re:Virtualization (Score:5, Informative)

      by mlts ( 1038732 ) * on Thursday December 31, 2009 @08:10AM (#30604250)

      Virtualization gives some advantages:

      1: You can move the VM between physical hardware with little trouble. Power off VM, robocopy the files, power it on. For older Windows operating systems that required a reinstall if the underlying HAL changed, this is a large lifesaver.

      2: Fast backups with the snapshot functionality.

      3: Cloning -- need more instances, grab more hardware, fire up Hyper-V or ESXi, slap the VM on and go to town.

      4: Clustering -- several physical machines can host one VM through a SAN and if one box fails, the failover can pick up where the main machine left off on the machine (not the app) level. This means you don't need to worry about how apps will deal with jumping MACs or hardware changes unexpectedly.

      5: Security. If a VM got infected, it can be powered off and rolled back to a safe snapshot, and also a snapshot taken of its dirty state for forensics.

      6: Ability to run on future hardware. Say everyone ditches x86 and amd64 and decides to go to IBM's POWER architecture and emulate legacy stuff. The stuff in the VM won't care that is is actually isn't running on a different CPU.

      Of course, virtualization's disadvantage is performance losses due to the added overhead of more context switching.

      For a MMO, virtualization isn't really needed except at the database core. If a zone server [1] goes down, there will be people nerd raging on the forums, but in reality if someone gets to it in 24 hours or so, people won't be pulling their subscriptions. The only real thing that would cause people to bail is a large player database rollback, so days to weeks of playing are lost. However if you have a good database cluster, this isn't going to happen.

      Virtualization is just one of many IT tools. Sometimes it is an excellent thing to have. Other times, there isn't any real need to have it, especially for CPU intensive stuff on a server that can be cloned or easily reimaged with the apps on it.

      [1]: I'm assuming zone servers handle the combat mechanics, only sending updates to the core player database when a player loots an item, dies, logs out, disconnects, or at a periodic interval if nothing else changes.

      • Re:Virtualization (Score:5, Insightful)

        by TheRealFixer ( 552803 ) on Thursday December 31, 2009 @09:18AM (#30604468)
        6: Ability to run on future hardware. Say everyone ditches x86 and amd64 and decides to go to IBM's POWER architecture and emulate legacy stuff. The stuff in the VM won't care that is is actually isn't running on a different CPU.

        This is not true. Hardware virtualization is not emulation, which is what you're talking about here. Processes in the VM are run directly on the host processor, they're just managed by the hypervisor. There's no emulation layer, since that would make performance pretty atrocious. So, the stuff in the VM absolutely will care about what processor you've moved to, especially if you've suddenly changed instruction sets. Binaries compiled for x86 won't magically run on PowerPC just because it's running on a VM.

      • Re: (Score:2)

        by jon3k ( 691256 )
        First off, what the hell is a "database core" ? Why would you virtualize it? You just cluster them if you need more than 1 physical hosts resources and if you need less than that you just install multiple instances of whatever DBMS you're working with.

        And I don't know who told you your guests could run on another type of hardware but they lied to you. There's no emulation happening. If the OS is designed to run on x86 hardware it won't magically run on another architecture.
      • Nice summary of VM benefits. What I find amazing is that the developers decided not to use off the shelf virtualization technology. How much development time would have been saved? Were tests conducted to see whether efficiency would be gained? Or did they just say "screw it, we don't know VMware, we're going to write it ourselves, from scratch!" The backup/restore/dynamic allocation/other benefits above are no joke. For a game, they are especially interesting, but so many firms ignore them.

        • Re: (Score:2)

          by Bengie ( 1121981 )

          I guess I would use World of Warcraft as as example. They have their own custom DB engine that keeps track of just about everything. Blizz use to have a link that let you look up some interesting info like

          What mob was the most killed
          what items dropped from which mobs
          average gold per mob
          average damage done per mob
          fastest time a mob died
          zone with the most average players
          how long the average player was in the zone..

          I could go select to filter by level 50 players, then find out their most played class, what lev

      • There isn't anything in there that you can't do with properly designed unix applications to which you have source access. Virtualisation is useful, but not for that.

    • No shit, I wish more people would see that the added complexities of virtualization sometimes outweigh the advantages, especially in resource-intensive apps that have never been tested in virtualization.

    • Re: (Score:2)

      by alen ( 225700 )

      say you have a few dozen apps relying on 5 different versions of Oracle BEA weblogic. you can't upgrade every WL box to the latest version since it will break another app on there and you can't upgrade every app since it take months of dev time. so you scale out and have one or a few WL instances on each OS instance. in this case you virtualize it because it reduces server sprawl and it's a stealth way of upgrading the hardware to something an ancient OS like Windows 2000 cant support. HP doesn't support Wi

  • Dell ad? (Score:3, Insightful)

    by Degro ( 989442 ) on Thursday December 31, 2009 @03:53AM (#30603656)
    This game failed in big part because of their extremely poor server performance. Who cares how they did it?

    • Re:Dell ad? (Score:4, Insightful)

      by Vrallis ( 33290 ) on Thursday December 31, 2009 @04:14AM (#30603720) Homepage

      This game failed in big part because of their extremely poor server performance. Who cares how they did it?

      Well, both poor server performance (Fortress battles were completely unplayable when I quit) plus they opened WAY too many servers at the start. If they'd started with 1/3 as many servers the game would probably be in far better shape today. The server transfers that they opened up as a last-ditch effort prior to merging servers was the straw that broke the camel's back for me, destroying my server's population.

      WTB a fantasy (okay, I'll say it...WoW-like) MMO modeled after EVE's economy, industry, PVP, territory control, etc.

      • Re: (Score:3, Funny)

        by Opportunist ( 166417 )

        You enjoy squaring circles?

        You want an MMO that is approachable and very easy to play, where learning a pattern of repeated keystrokes ensures success that runs along the creed of "insert time, get reward", where items and their stats are the be-all, end-all defining element when it comes to the strength of your character without too much dependency on your playing skill (i.e. a WoW-like game) with a complex player driven economy, where almost all equipment is entirely player created, with a complex resourc

        • Re: (Score:1)

          by jhoegl ( 638955 )
          So what you are saying is
          1.) Invent time sink
          2.) Make people pay for wasting their time on something that wont matter
          3.)???
          4.) Profit.

          No wonder most games are boring to me now.
        • Word War II online had a neat idea. It was a MMO that was played in campaigns. Within each campaign many players would just log in and fight first person style. But there was also other players that made strategic decisions within a campaign that effected the game play of all players.

          For example the leadership players could decide to reduce fighter plan construction in favor of having more tanks available. When a player logged in they might find that all the plans were already in use so they would ha

    • Umm... to avoid repeating it?

      Nothing is too bad to serve as an example how NOT to do it.

  • The increased proportion of slashvertising is a direct result of the holidays? Or the holidays are slow news days and that only leaves the omnipresent advertising.

    i.e.: In a hypothetical fast news holiday...day, would we have a quick and constant flow of mostly ad news? Or just the normal proportion.

    • Re: (Score:3, Interesting)

      by Darinbob ( 1142669 )
      Actually, it is kind of interesting to see what's in the back room. I know in some MMOs if a "server" is down there's inevitably some wiseguy who says "they should buy a better machine", or "I'm an IT dude and I could run the place better than those bozos". Other times people are confused why some regions of the game are working but others are not. Or why it takes so long to reboot the "server" to apply the next game patch. Or why there's scheduled maintenance. I think there are a lot of players who re

      • Re: (Score:1)

        by jhoegl ( 638955 )
        Blades take the same amount of time to boot as most servers. The reason for the downtime is data transfers.
        • Yes, but my point is that it isn't a single computer that is a server. The shutdown, patch, database updates, reboot, and verification involves tens of computers.

      • Re: (Score:2)

        by mlts ( 1038732 ) *

        What cracks me up is people who say that, especially the IT dude thing, then when asked what they would do differently, it seems they get the deer in the headlights thing going.

        If one wants to know what running a MMO is like on a small scale, some Neverwinter Nights and NWN2 persistant worlds used a core database (MySQL + NWNX) and zone servers so they could deploy an immense world spanning a sizable number of PCs. It gets pretty scary how much CPU a zone server which just handles the mechanics of gameplay

        • Yeah, when I ran a MUD it was on a "mini-super". While CPU was an issue (especially during the work day when the computer had other jobs to do), bigger concerns were with handling a large number of active network connections and RAM space. And this was with only about 50 players at once.

      • I think there are a lot of players who really believe that the companies just go out and buy a single computer (probably a tower) for each game server.

        Thats exactly what Codemasters did to Operation Flashpoint :D

  • Gamasutra can be added to that list of tech brands that have failed. Once upon a time, Gamasutra had technical articles with technical content. It's becoming increasingly difficult to find real information anymore. Programmers used to write detailed treatises on techniques they've used. Now the money managers tie everything up in NDAs and the state of the art has failed to advance for a decade.

    By squinting hard and using a calculator, it's possible to decide that a "server" in Warhammer is actually 3 c

  • I was going to complain about this, ("come on, slashdot editors, pay attention"), but then I realized that the first page of the advert-article is essentially content-free. Was this odd link a feature or a bug that's a feature anyway?

Slashdot Top Deals

Two can Live as Cheaply as One for Half as Long. -- Howard Kandel

Close