NAMD

NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems.

• Version: 3.0.3
• Machines: HPC clusters 
• Location: /uufs/chpc.utah.edu/sys/installdir/r8/namd

Information:

To get details of the functionality of VMD visit the website: http://www.ks.uiuc.edu/Research/namd/

We have the smp cpu version (ml namd/3.0.verbs-smp) as well as a GPU build (ml namd/3.0.verbs-smp-CUDA). On the Lonepeak cluster, which does not have the InfiniBand network, use the namd/2.14.netlrts.

To Use:

The builds that we provide have support the InfiniBand network through the ibverbs interface. NAMD uses charm++ parallel objects for parallelization and as such the program launch uses the charmrun interface. One also has to create a specific host file that lists the nodes on which to run.

The SLURM script written in tcsh to run on notchpeak or kingspeak needs to include the following:

#load NAMD module
module load namd/3.0.3.verbs-smp
set NAMD3_FULLPATH = `which namd3`

#set up hostfile
srun hostname | sort > nodefile
echo group main > hostfile
set NODES = `cat nodefile`
foreach node ( $NODES )
 echo host $node >> hostfile
end

# run NAMD via charmrun, substituting in your input and output file names
charmrun $NAMD3_FULLPATH ++n $SLURM_NNODES ++ppn 1 +setcpuaffinity ++nodelist hostfile ubq_ws.namd > ubq_ws.log

Note that in this case we run $SLURM_NTASKS processes per node (++n), each with one PEs (processing elements, +ppn). However, we encourage to try a few combinations of processes per node and PEs per process (e.g. 2 processes per node, $SLURM_CPUS_ON_NODE/2 PEs, etc.) , to see what combination gives the best performance, as this combination may be CPU type specific.

For Lonepeak, we recommend to run only on a single node, since the Ethernet network is relatively slow. That is:

module load namd/3.0.netlrts-smp
set NAMD3_FULLPATH = `which namd3`

# prepare hostfile for charmrun
srun hostname | sort > nodefile
echo group main > hostfile
set NODES = `cat nodefile`
foreach node ( $NODES )
 echo host $node >> hostfile
end

# run NAMD via charmrun
charmrun $NAMD3_FULLPATH +p$SLURM_NTASKS +ppn1 ++nodelist hostfile +setcpuaffinity in.namd > in.log

GPU use:

CHPC clusters don't have efficient network setup for multi-node GPU communication, therefore use only GPUs within a single node. Furthermore, even intra-node GPU communication on many GPUs is not very performant, therefore the bulk of NAMD simulations will be the most efficient using a single GPU. For that, we have module namd/3.0.3.multicore-CUDA. Ask for single process, ++p1, and use multiple cores with the ++ppn option. SLURM script would look like:

#SBATCH --account=notchpeak-shared-short
#SBATCH --time=4:00:00
#SBATCH --nodes=1
#SBATCH --ntasks=4
#SBATCH --gres=gpu:t4:1
#SBATCH -o slurm-%j.out-%N
#SBATCH -e slurm-%j.err-%N
#

#load NAMD module
module load namd/3.0.3.multicore-CUDA
set NAMD3_FULLPATH = `which namd3`

# run NAMD via charmrun
charmrun $NAMD3_FULLPATH ++local ++ppn $SLURM_NTASKS ++n 1 +setcpuaffinity in.namd > in.log