SLURM: reconfig

Reconfigure all Slurm daemons on all nodes. This should be done after changing the Slurm configuration file.

scontrol reconfig

Print the current Slurm configuration. This also reports if the primary and secondary controllers (slurmctld daemons) are responding. To just see the state of the controllers, use the scontrol subcommand ping.

  scontrol show config

example restart-related commands on most systems

  ulimit -a
  # make sure limits are high enough
  # are they a cap for what is spawned?
  set -x -v

  scontrol reconfig

  systemctl stop slurmd 
  systemctl stop slurmctld

  systemctl start slurmctld
  systemctl start slurmd 

  systemctl status slurmctld
  systemctl status slurmd 

  echo log files are in /var/log/*slurm*

     4 drwxr-xr-x 2 slurm slurm    4096 Jan 16  2020 /var/log/slurm-llnl
  2368 -rw------- 1 slurm slurm 2416774 Dec  2 16:53 /var/log/slurmctld.log
   212 -rw------- 1 root  root   210845 Dec  2 16:53 /var/log/slurmd.log

slurmctld -D -vvvv


category: index