The Spack package manager: Bringing order to HPC software chaos

Todd Gamblin; Matthew Legendre; Michael R. Collette; Gregory L. Lee; Adam Moody; Bronis R. De Supinski; Scott Futral

doi:10.1145/2807591.2807623

The Spack package manager: Bringing order to HPC software chaos

Todd Gamblin, Matthew Legendre, Michael R. Collette, Gregory L. Lee, Adam Moody, Bronis R. De Supinski, Scott Futral

School of Electronics, Electrical Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

168 Citations (Scopus)

Abstract

Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools. Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible. However, managing many configurations is difficult because the configuration space is combinatorial in size. We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity. Spack provides a novel, recursive specification syntax to invoke parametric builds of packages and dependencies. It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment. We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.

Original language	English
Title of host publication	Proceedings of SC 2015
Subtitle of host publication	The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1
Volume	SC '15
ISBN (Electronic)	9781450337236
DOIs	https://doi.org/10.1145/2807591.2807623
Publication status	Published - 15 Nov 2015
Event	International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015 - Austin, United States Duration: 15 Nov 2015 → 20 Nov 2015

Conference

Conference	International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015
Country/Territory	United States
City	Austin
Period	15/11/2015 → 20/11/2015

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications
Hardware and Architecture
Software

Access to Document

10.1145/2807591.2807623

Cite this

Gamblin, T., Legendre, M., Collette, M. R., Lee, G. L., Moody, A., De Supinski, B. R., & Futral, S. (2015). The Spack package manager: Bringing order to HPC software chaos. In Proceedings of SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis (Vol. SC '15, pp. 1). Article a40 Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/2807591.2807623

@inproceedings{7c43fdf08a094b5fb2a17a6e1ed5cab9,

title = "The Spack package manager: Bringing order to HPC software chaos",

abstract = "Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools. Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible. However, managing many configurations is difficult because the configuration space is combinatorial in size. We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity. Spack provides a novel, recursive specification syntax to invoke parametric builds of packages and dependencies. It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment. We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.",

author = "Todd Gamblin and Matthew Legendre and Collette, {Michael R.} and Lee, {Gregory L.} and Adam Moody and {De Supinski}, {Bronis R.} and Scott Futral",

year = "2015",

month = nov,

day = "15",

doi = "10.1145/2807591.2807623",

language = "English",

volume = "SC '15",

pages = "1",

booktitle = "Proceedings of SC 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015 ; Conference date: 15-11-2015 Through 20-11-2015",

}

Gamblin, T, Legendre, M, Collette, MR, Lee, GL, Moody, A, De Supinski, BR & Futral, S 2015, The Spack package manager: Bringing order to HPC software chaos. in Proceedings of SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis. vol. SC '15, a40, Institute of Electrical and Electronics Engineers Inc., pp. 1, International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015, Austin, United States, 15/11/2015. https://doi.org/10.1145/2807591.2807623

The Spack package manager: Bringing order to HPC software chaos. / Gamblin, Todd; Legendre, Matthew; Collette, Michael R. et al.
Proceedings of SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis. Vol. SC '15 Institute of Electrical and Electronics Engineers Inc., 2015. p. 1 a40.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - The Spack package manager

T2 - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015

AU - Gamblin, Todd

AU - Legendre, Matthew

AU - Collette, Michael R.

AU - Lee, Gregory L.

AU - Moody, Adam

AU - De Supinski, Bronis R.

AU - Futral, Scott

PY - 2015/11/15

Y1 - 2015/11/15

N2 - Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools. Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible. However, managing many configurations is difficult because the configuration space is combinatorial in size. We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity. Spack provides a novel, recursive specification syntax to invoke parametric builds of packages and dependencies. It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment. We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.

AB - Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools. Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible. However, managing many configurations is difficult because the configuration space is combinatorial in size. We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity. Spack provides a novel, recursive specification syntax to invoke parametric builds of packages and dependencies. It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment. We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.

UR - http://www.scopus.com/inward/record.url?scp=84966472143&partnerID=8YFLogxK

U2 - 10.1145/2807591.2807623

DO - 10.1145/2807591.2807623

M3 - Conference contribution

AN - SCOPUS:84966472143

VL - SC '15

SP - 1

BT - Proceedings of SC 2015

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 15 November 2015 through 20 November 2015

ER -

Gamblin T, Legendre M, Collette MR, Lee GL, Moody A, De Supinski BR et al. The Spack package manager: Bringing order to HPC software chaos. In Proceedings of SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis. Vol. SC '15. Institute of Electrical and Electronics Engineers Inc. 2015. p. 1. a40 doi: 10.1145/2807591.2807623

The Spack package manager: Bringing order to HPC software chaos

Abstract

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Bronis de Supinski

Cite this

The Spack package manager: Bringing order to HPC software chaos

Abstract

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Profiles

Bronis de Supinski

Cite this