Creating a login node with a Raspberry Pi for the miniHPC

minihpc

Start with an update

sudo apt update -y
sudo apt upgrade -y

Install required packages.

sudo apt install -y nfs-kernel-server lmod ansible slurm munge nmap \
nfs-common net-tools build-essential htop net-tools screen vim python3-pip \
dnsmasq slurm-wlm iptables iptables-persistent libmunge-dev libmunge2 \
libpmix2 libpmix-bin libpmix-dev git

A dialog block will appear on the screen. Answer yes to both questions.

Enable IP forwarding

  • As sudo, edit the file /etc/sysctl.conf. Find the line that reads #net.ipv4.ip_forward=1 and remove the # at the beginning of the line. Save the file. The change will become active after a reboot.

Configure IP-tables

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT
sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo netfilter-persistent save

Configure the network interfaces

Place the following into /etc/network/interfaces

auto eth0
allow-hotplug eth0
iface eth0 inet static
  address 192.168.5.101
  netmask 255.255.255.0
source /etc/network/interfaces.d/*

Modify the hostname

echo pixie001 | sudo tee /etc/hostname

Configure DNS masquerading

  • Configure dnsmasq by entering the following in the file /etc/dnsmasq.conf. Replace the MAC address on the sixth line with the MAC address of your compute node.
interface=eth0
bind-dynamic
domain-needed
bogus-priv
dhcp-range=192.168.5.102,192.168.5.200,255.255.255.0,12h
dhcp-host=ab:cd:12:34:ab:cd,192.168.5.102
dhcp-option=3,192.168.0.1 # default route
dhcp-option=6,192.168.0.1 # default route

Create a shared directory.

sudo mkdir /sharedfs
sudo chown nobody:nogroup -R /sharedfs
sudo chmod 777 -R /sharedfs

Configure NFS

  • Configure shared drives by adding the following at the end of the file /etc/exports
/sharedfs    192.168.5.0/24(rw,sync,no_root_squash,no_subtree_check)

Configure hosts

  • The /etc/hosts file should contain the following. Make sure to change all occurences of pixie in the script to the name of your cluster:
127.0.0.1   localhost
::1     localhost ip6-localhost ip6-loopback
ff02::1     ip6-allnodes
ff02::2     ip6-allrouters

# This login node's hostname
127.0.1.1   pixie001

# IP and hostname of compute nodes
192.168.5.102   pixie002

Configure Slurm

Add the following to /etc/slurm/slurm.conf. Change all occurences of pixie in this script to the name of your cluster.

SlurmctldHost=pixie001(192.168.5.101)
MpiDefault=none
ProctrackType=proctrack/cgroup
#ProctrackType=proctrack/linuxproc
ReturnToService=1
SlurmctldPidFile=/run/slurmctld.pid
SlurmctldPort=6817
SlurmdPidFile=/run/slurmd.pid
SlurmdPort=6818
SlurmdSpoolDir=/var/lib/slurm/slurmd
SlurmUser=slurm
StateSaveLocation=/var/lib/slurm/slurmctld
SwitchType=switch/none
TaskPlugin=task/affinity
InactiveLimit=0
KillWait=30
MinJobAge=300
SlurmctldTimeout=120
SlurmdTimeout=300
Waittime=0
SchedulerType=sched/backfill
SelectType=select/cons_res
SelectTypeParameters=CR_Core
AccountingStorageType=accounting_storage/none
# AccountingStoreJobComment=YES
AccountingStoreFlags=job_comment
ClusterName=pixie
JobCompType=jobcomp/none
JobAcctGatherFrequency=30
JobAcctGatherType=jobacct_gather/none
SlurmctldDebug=info
SlurmctldLogFile=/var/log/slurm/slurmctld.log
SlurmdDebug=info
SlurmdLogFile=/var/log/slurm/slurmd.log
PartitionName=pixiecluster Nodes=pixie[002-002] Default=YES MaxTime=INFINITE State=UP
RebootProgram=/etc/slurm/slurmreboot.sh
NodeName=pixie001 NodeAddr=192.168.5.101 CPUs=4 State=IDLE
NodeName=pixie002 NodeAddr=192.168.5.102 CPUs=4 State=IDLE
  • Restart slurm
sudo systemctl restart slurmctld

Configure munge

  • Create munge key
sudo mkdir /etc/munge
dd if=/dev/urandom bs=1 count=1024 | sudo tee -a /etc/munge/munge.key
  • Set ownership
sudo chown munge: /etc/munge/munge.key
sudo chmod 400 /etc/munge/munge.key

Install ESSI

mkdir essi
cd essi
wget https://raw.githubusercontent.com/EESSI/eessi-demo/main/scripts/install_cvmfs_eessi.sh
sudo bash ./install_cvmfs_eessi.sh

source /cvmfs/software.eessi.io/versions/2023.06/init/lmod/bash
Back to top