RESTful Web Services for Scientific Computing

Location: D133
Average rating: **...
(2.33, 3 ratings)

The National Energy Research Scientific Computing Center (NERSC) houses some of the worlds most powerful supercomputers. NERSC awards HPC cycles via an application process, among the scientific applications consuming cycles are climate change and quantum mechanics simulations. At NERSC several scientific portal efforts are underway in order to open up HPC resources to a wider user community.

The typical NERSC user will remotely log into an HPC system, and in a shell environment using command-line tools create an application’s input files, and a PBS script which will be submitted to the batch scheduler. Depending on the queue, the scheduler may take hours to move the job to run status. After the job has completed, which may take hours or weeks, the various output files generated by the application may need to be moved and analyzed. The record of the job run is the working directory where all the input/output files are kept. This is a typical workflow for an HPC user at NERSC.

There are various limitations of a command-line centric world. First many users, especially those in emerging or growing HPC Scientific Application communities, don’t like and don’t want to learn command-line. Their comfort zone is the World Wide Web, learning to use a new web site is preferable than reading and rereading documentation on how to submit and monitor HPC jobs at NERSC. A portal environment provides a single point of access, which is extremely advantageous for recording a history, canning workflows, and combining the power of underlying resources into a unified interface. Secondly, command-line tools are computer-centric while user’s want application-centric tools. The NERSC Web Toolkit (NEWT) provides RESTFul HTTP interfaces to HPC resources, scientific application users and portal developers can utilize these services to create new user facing web applications.

In this talk I will present the NEWT API, implemented as a python Django project, and discuss NERSC portals that are taking advantage of it.

Photo of Joshua Boverhof

Joshua Boverhof

Lawrence Berkeley National Lab

Joshua is an engineer at LBNL, his expertise includes web and cloud technologies. He is currently developing RESTful web services for the NERSC supercomputing center, and works on “” a hosted managed file transfer service.

Photo of Shreyas Cholia

Shreyas Cholia

Lawrence Berkeley National Laboratory

Shreyas Cholia leads the Usable Software Systems group at Lawrence Berkeley National Laboratory (LBNL), which focuses on making scientific computing more transparent and usable. He is particularly interested in how web APIs and tools can facilitate this. Shreyas also leads the science gateway, web, and grid efforts at the National Energy Research Scientific Computing Center (NERSC) at LBNL. His current work includes a project that enables Jupyter to interact with supercomputing resources, and NEWT, a REST API for high-performance computing. He holds a degree from Rice University, where he studied computer science and cognitive sciences.