A first look at Doom 3 Mac benchmarks

For more Doom 3 benchmarks, please read Doom 3 Benchmarks Revisited.

On Friday Aspyr Media Inc. announced that its Macintosh conversion of id Software’s first person shooter Doom 3 had gone gold master and was expected to begin shipping in mid-March. We’ve spent the past few days with that GM, playing it on one of the fastest Mac gaming rigs you buy right now, and we’ve got some numbers to share with you. The results are eye-opening.

So what’s the big deal?

If you’re not a hardcore gamer, you may be wondering why we’re making such a big deal out of this one game. It’s a good question, and the answer is because it’s not just about this one game.

Doom 3 is bound to sell a lot of copies — id Software is hugely respected by gamers and the PC version of this game has had a lot of exposure during its development over the past few years. In fact, you may remember that id Software’s own John Carmack several years ago took the Macworld Expo keynote stage with Steve Jobs, in Tokyo, Japan, to demonstrate the core technology that powers this game.

But more importantly than that, id’s gaming technology is licensed to other game developers. So by bringing Doom 3 to the Macintosh, Aspyr Media has paved the way for other games using the same technology to come to the Mac more easily and inexpensively in the future. And a look at Doom 3’s performance may be a good indicator of how those future games will run on the Mac.

Our test setup

First, a little bit on our gaming rig: It’s a Power Mac G5 2.5GHz dual processor model, equipped with 2.5GB of RAM and ATI’s new Radeon X800 XT Mac Edition graphics card — the fastest ATI card you can get on the Mac, equipped with 256MB VRAM.

We tried the game on an ADC-equipped 22-inch Apple Cinema Display and a 17-inch CRT-based Apple Studio Display connected with a DVI to VGA adapter. We used the Studio Display to report 1600 x 1200 performance, a higher resolution than the Cinema Display could produce, and we double-checked its performance at the same resolutions as the Cinema Display to make sure there wasn’t any difference.

The system was running Mac OS X10.3.8 — the minimum operating system requirement for Doom 3, since that particular release features some improvements to Apple’s OpenGL drivers that help improve performance for games like Doom 3. All tests were performed in full-screen mode, with no other applications running in the background.

All tests were made using Doom 3’s timedemo demo1 command, accessible through its built-in console. The tests were performed at resolutions from 640 x 480 to 1600 x 1200, running with and without Full-Scene Anti Aliasing (FSAA).

FSAA a technique that helps smooth the jagged edges of polygons, providing a more realistic look. Many gamers like to run with FSAA turned on, especially at lower resolutions. FSAA causes a performance penalty, as you can see, and frankly, we think it has diminishing returns in Doom 3 — so much of the game is in dark and dimly lit spaces, anti-aliasing jagged edges don’t make it look any better.

Anisotropic filtering — which improves the visual quality of textures on surfaces that are viewed at an angle — was left at its default setting (for this video card), 8x. We found that fiddling with this setting had a negligible effect on frame rates, so we left it at 8x for our tests.

Here’s what we saw:

The demo really pushes the hardware, and it’s not the same as what you’ll go through in much of the game. There’s a lot of movement, a lot of texture data and a lot of animation and complex geometry. When we ran Doom 3 itself with a frame counter, it often stayed in ranges much higher than these numbers suggest, and the game itself is throttled to 60 frames per second — regardless of how fast the demo runs, 60 FPS is as fast as you’d ever see the actual game go.

Aspyr tells us that if you have an Nvidia GeForce 6800 card in your Power Mac, chances are you’ll see modestly higher numbers than what we saw here. Nvidia’s 6800 tracks higher than the X800 on the PC as well, although the difference in performance isn’t as great as it is on a PC.

Apples to Oranges

So the bottom line is this: If you were hoping that your dual-processor G5 was going to be a Pentium or Athlon-killer when it comes to Doom 3 framerates, you’re going to be disappointed. But there are some important factors to consider.

Numerous Web sites and magazines have benchmarked Doom 3’s performance on the PC since the game was released last summer, and these numbers are bound to draw comparisons. You’ll discover that our tests produced lower frame rate averages than those PC benchmarks you’ll see elsewhere — in some cases by a little, in other cases by a lot, depending on the methodologies used and the systems tested. One way or the other, Mac frame rates seem off by 20 percent or more in many cases in the comparisons we’ve seen posted online.

We’ve often heard about the “Megahertz Myth” — the idea that Macs can perform as well as or better than PCs despite their lower clock speeds, and that’s true for some operations. Games don’t work that way, though: there’s no substitute for CPU cycles when it comes to games like this. Many tests we’ve seen are based on Pentium systems working at higher clock speeds than this Power Mac G5, so that should be considered — scaling up the G5’s framerate numbers proportionally helps reduce that gap.

Also consider that there are core differences between Macs and PCs like bus architectures, memory speed, different graphics chips that support faster memory and clock speeds, and other factors that make a direct apples-to-apples comparison exceedingly difficult.

Finally, it’s worth noting that despite the lower numbers compared to PCs, we are seeing a very playable frame rate of 29.5 FPS running at 1600 x 1200 pixels. And at that resolution, FSAA has really diminishing returns — Doom 3 is spectacular-looking even without it.

Higher-end penalty

Doom 3 has very demanding system requirements — the highest we’ve seen on a Mac game to date. A 1.5GHz G4 is the minimum, along with an ATI Radeon 9600 or Nvidia GeForce FX 5200 graphics system with at least 64MB VRAM. These specs aren’t wildly out of sync with Doom 3’s PC counterpart, either — this game really asks a lot of hardware regardless of platform.

It appears that the performance disparity we’ve discussed is happening mainly when comparing the performance of high-end Macs to high-end PCs. On Macs that are closer to the minimum system requirements and on mid-range systems, the performance difference between a Mac and a PC is much smaller, and in some cases, negligible.

Year-old PowerBooks and iMac G5’s are just scraping the minimum requirements. In a quick test on a 17-inch PowerBook G4/1.5GHz system with 1GB RAM, ATI Mobility Radeon 9700 graphics with 64MB VRAM and Mac OS X v10.3.8, we saw framerates in the low 20’s with Doom 3’s default options turned on (resolution at 640 x 480).

A 1.6GHz iMac G5 with sufficient RAM will pull default numbers in the mid-20’s, about the same as a similarly equipped Dell PC running at the same clock speed, according to Aspyr. Aspyr has also benchmarked a Power Mac G5 1.8GHz system equipped with an ATI Radeon 9800 Pro Mac Special Edition card — considered a “mid-range” system for this game — operating at about 29.6 frames per second, compared to 32 frames per second for a similarly equipped Dell.

With this information in mind, it’s pretty clear that the big performance difference between Macs and PCs is happening on the high end of the system configurations, not the low end.

Where’s the slowdown?

Aspyr tells us they worked diligently with Apple to make sure “no stone was left unturned” when it came to optimizing Doom 3 for the PowerPC’s Velocity Engine registers, so you’ll be happy to know that Doom 3 is well-tuned for your G4 and G5. A caveat, however: Early on in development, id Software determined it had very little to gain in performance by optimizing the game engine for dual-processor configurations, as it had with later builds of Quake 3 Arena. As a result, Mac gamers using dual-processor systems like ours will see no direct benefit.

Running the same demo at 640 x 480 with 3D rendering turned off — which can be done by setting the r_skiprender value to 1 in the Doom 3 console, then running the timedemo — returns an FPS average of better than 106 frames per second. Setting that command gets all the 3D stuff out of the way, telling us how fast the core game engine itself is running. And that number compares pretty well from Macs to PCs. That implies that the performance difference we’re seeing has something to do with the way the 3D graphics are rendered, as opposed to any specific shortcoming in the game engine or overall slowdown that would account for a difference.

So where does this performance drop come from? Some of it, at least, can be explained by the way the Mac works. Mac game developers agree that it’s much harder to get the Mac to pay attention exclusively to a game, even if it’s the only application running, than it is on a PC running Windows XP. That has some payoffs — you can play this game in a window, for example, seeing only a minor performance drop — because “OpenGL’s tendrils run much deeper” in the Mac OS X kernel than they do in Windows, according to one source with whom we spoke.

Will further driver optimization help? It’s entirely possible, although it’s hard to say how much. We expect such improvements will be iterative and gradual. But if Quake 3 Arena’s history is any indicator, we’ll see continued improvements in both driver speed and graphics chip speed until both Macs and PCs are on a level playing field. At least until the next great game engine comes along.