IBM's Roadrunner smashes 4-minute mile of supercomputing

June 9, 2008 (Computerworld) IBM's Roadrunner supercomputer has smashed the high-tech equivalent of the four-minute mile by breaking the lofty petaflop barrier.

IBM executives announced today that the company's latest supercomputer -- a hybrid system running AMD Opteron processors and Cell chips -- sustained a speed of 1.026 quadrillion calculations per second. That's about twice as fast as the next-fastest supercomputer, IBM's BlueGene/L, which is based at the Lawrence Livermore National Laboratory.

The new machine would need a single week to run a calculation that the fastest supercomputer 10 years ago would have needed 20 years to complete.

Click on these links to see a video featuring Don Grice, chief engineer of the Roadrunner project, talking about its development, and to see a photo gallery featuring the supercomputer and its developers at IBM's Poughkeepsie, N.Y., research facility

A petaflop is a quadrillion floating-point operations per second. The petaflop barrier, which has been the golden ring of supercomputing since the teraflop barrier was broken 11 years ago, is a goal that many companies, including Cray Inc., Hewlett-Packard Co., Sun Microsystems Inc. and SGI, all have been shooting for. IBM beat them to the punch, but the other contenders are still at its heels. Now IBM is on to the next goal, creating an exascale system.

The Roadrunner machine broke the barrier on May 25 on its fourth attempt, according to Grice. He told Computerworld that the first test of the machine as a whole simply was geared to make sure it could launch and complete a problem on such a large number of cores. The second test suffered a failure in a memory module. In the third test, technicians discovered they weren't using a big enough application.

Then with the fourth test, the machine passed the petaflop barrier in the middle of the night on May 25, said Grice. Most everyone had gone home because it was so late, but two technicians were on hand to watch the supercomputer run through the application and read the final speed report.

"Making the number has caught everyone's imagination," said Grice. "But it's really what you can do with that number."

The supercomputer will be used at Los Alamos National Laboratory to work on national security problems, test nuclear stockpiles, run annual testing of various nuclear weapons systems and predict long-term climate change, as well as studying the universe and trying to find an HIV vaccine, according to John Morrison, leader of the high-performance computing division at Los Alamos.

"We're dealing with nuclear weapons," said Thomas D'Agostino, administrator of the National Nuclear Security Administration. "Speed is of critical importance here. If our labs can solve problems faster, that's a good thing because we are dealing with nuclear weapons. If we could have solved a problem in six months with an already fast supercomputer, then we can solve it in a month now."

Roadrunner's accomplishment marks the first time that BlueGene, which runs at 478 teraflops, hasn't held the highest position in the Top 500 supercomputer list since November 2004, according to Jack Dongarra, a co-creator of the Top 500 list.

Roadrunner uses 3.9 megawatts of power, which Grice noted is enough to power 39,000 100-watt light bulbs. It has 6,948 dual-core Opterons on IBM LS21 Blades, as well as 12,960 Cell processors on IBM QS22 blades. The machine, which has 80 terabytes of memory, has 296 IBM BladeCenter H racks. It takes up 6,000 square feet, uses 57 miles of fiber optic cable and weighs in at 500,000 pounds.

The new IBM supercomputer is the second incarnation of Roadrunner. The original Roadrunner, a cluster machine that can hit 70 teraflops, is in use at the Los Alamos lab. The older Linux-based system runs AMD Opteron chips.

This new version of Roadrunner also runs Linux and gets its hefty power boost by adding the Cell chips, originally designed jointly by IBM, Toshiba and Sony for the latter's PlayStation 3 game console, to the Opteron base. The hybrid supercomputer will use the Cell chips for massive calculations.