SLURM is coming! MPI threading, Open Source, reprogramming, and Futurama.

SLURM: “It’s highly addictive!

Why are we changing?

The current scheduler used by the DSCR, Sun Gridengine (SGE), is past the end of its life.  In the software world, this means that it no longer gets bug fixes, and it will work with less and less efficiency and more and more errors as operating systems are updated and hardware is refreshed.  There are several candidates for replacement, but the Duke Shared Cluster Resource (DSCR) has decided to go with SLURM which has been widely adopted by the research computing community. A large portion of the TOP500 use SLURM, including current Number One Tianhe-2 (Milky Way-2).

What is it?

SLURM is Open Source, meaning that it is community-supported and takes contributions to its code from the community. Users with problems or issues or questions can go to the SLURM community to discuss it and get help. It was built from the beginning with MPI threading in mind, and support for multi-core jobs is one of the main focuses of the SLURM community. Further, the logging and tracking features are much more comprehensive, which will allow greater insight into use of the cluster to guide future development and enhancements.

Several groups on campus are already using SLURM, including IGSP, and  the transition from SGE is reportedly moderately easy, although the switch will require all scripts to be modified.  For many commands, simply type an “s” where you would have put a “q” will work.  The Research Computing group will be hosting classes and training sessions both before and after the shift, and a test cluster will be available soon.  NB: Your previous jobs and scripts will NOT work on the new cluster.  You will need to re-program them.

When are we changing?

The test cluster is currently being built, and the goal is to have it available by the end of January or mid-February. Testing will commence at this time, and every user of the DSCR will need to log in to test their account and their jobs. These will not be production machines, results will not be guaranteed and content will not be saved or archived.  You will receive reminders that this testing is needed.

The production cluster will come on line gradually, as machines and accounts are migrated over. Your account on the new production cluster will be activated after testing is completed.  The aim is for this to be completed during the summer of 2014, but we realize that much of that timeline depends on the tests run on the programs and code that will be imported.

How is this transition being handled?

The transition will be done in a systematic way. At an announced date, all queues on the current cluster will be disabled. As machines become free of jobs, they will be moved to the new cluster. All machines will be re-imaged, and they will get new names and the new software. User accounts will be suspended on the old cluster, and as a user’s jobs complete, the account will be migrated to the new cluster. However, the account will not be enabled until the user has completed testing on the test cluster. No data will be lost in the migration, and storage will attach to the new login boxes the same way as it does now.

Also during this transition, stale accounts will be examined and their contents shared with the PI for that account’s group. Once a user has moved on from Duke, their account automatically becomes invalid on the new system, requiring sponsorship from a PI for temporary continuation of NetID privileges.

Who is doing all of this?

The process for upgrading software is handled by OIT, with a project manager, a formal Request For Change, and a regular team that meets biweekly and discusses changes and problems.  Mark DeLong is the project owner and Bryn Smith is the lead technician.  Updates will be posted regularly to this blog and sent out to the hpc-announce@duke.edu list.  Please send questions, concerns, or input to scsc@duke.edu.

What’s in a name?

The animated TV series Futurama has an ongoing product-placement campaign for a soft drink called Slurm. The acronym SLURM stands for Simple Linux Utility for Resource Management.

– Bryn Smith


This entry was posted on Monday, January 27th, 2014 at 9:27 am and is filed under high-performance computing, service update. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

2 thoughts on “SLURM is coming! MPI threading, Open Source, reprogramming, and Futurama.

  1. From Jacob Jenson, at SchedMD LLC, comes this comment:
    The SLURM link in the first paragraph is pointing to an old llnl.gov
    page (https://computing.llnl.gov/linux/slurm/). LLNL has not worked on
    slurm in over three years and the site content is very out of date.
    SchedMD has been the driving force behind Slurm for the past three
    years. If you want to update this link the current Slurm code repository
    is found at http://schedmd.com/#repos and the current documentation is
    located at http://slurm.schedmd.com/

    In addition to driving Slurm development forward SchedMD also offers
    professional services for Slurm users, these services include training,
    support contracts and custom development. If you or your team would like
    to know more about any of these services please let me know.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>