Scheduling a 100,000 core supercomputer for maximum utilization and capability

Andrews Phil, Kovatch Patricia, Hazlewood Victor, Baer Troy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations

Abstract

In late 2009, the National Institute for Computational Sciences placed in production the world's fastest academic supercomputer (third overall), a Cray XT5 named Kraken, with almost 100,000 compute cores and a peak speed in excess of one Petaflop. Delivering over 50% of the total cycles available to the National Science Foundation users via the TeraGrid, Kraken has two missions that have historically proven difficult to simultaneously reconcile: providing the maximum number of total cycles to the community, while enabling full machine runs for "hero" users. Historically, this has been attempted by allowing schedulers to choose the correct time for the beginning of large jobs, with a concomitant reduction in utilization. At NICS, we used the results of a previous theoretical investigation to adopt a different approach, where the "clearing out" of the system is forced on a weekly basis, followed by consecutive full machine runs. As our previous simulation results suggested, this lead to a significant improvement in utilization, to over 90%. The difference in utilization between the traditional and adopted scheduling policies was the equivalent of a 300+ Teraflop supercomputer, or several million dollars of compute time per year.

Original languageEnglish
Title of host publicationProceedings - 2010 39th International Conference on Parallel Processing Workshops, ICPPW 2010
Pages421-427
Number of pages7
DOIs
StatePublished - 2010
Externally publishedYes
Event2010 39th International Conference on Parallel Processing Workshops, ICPPW 2010 - San Diego, CA, United States
Duration: 13 Sep 201016 Sep 2010

Publication series

NameProceedings of the International Conference on Parallel Processing Workshops
ISSN (Print)1530-2016

Conference

Conference2010 39th International Conference on Parallel Processing Workshops, ICPPW 2010
Country/TerritoryUnited States
CitySan Diego, CA
Period13/09/1016/09/10

Keywords

  • Component
  • High-performance computing
  • Scheduling
  • Systems software

Fingerprint

Dive into the research topics of 'Scheduling a 100,000 core supercomputer for maximum utilization and capability'. Together they form a unique fingerprint.

Cite this