Lustre file system – simple deploy

OS: CentOS 6.3
MGS server: lustre-mgs IP 192.168.0.1
OSS server: lustre-oss-01 IP 192.168.0.2
Storage: local drive on OSS server

Download Lustre packages: http://downloads.whamcloud.com/public/lustre/latest-maintenance-release/
Download modified e2fsprogs packages: http://downloads.whamcloud.com/public/e2fsprogs/latest/

Deploy MGS server

rpm -ivh kernel-2.6.32-279.14.1.el6_lustre.x86_64.rpm kernel-firmware-2.6.32-279.14.1.el6_lustre.x86_64.rpm
rpm -Uvh e2fsprogs-1.42.3.wc3-7.el6.x86_64.rpm libcom_err-1.42.3.wc3-7.el6.x86_64.rpm e2fsprogs-libs-1.42.3.wc3-7.el6.x86_64.rpm libss-1.42.3.wc3-7.el6.x86_64.rpm
rpm -ivh lustre-modules-2.1.4-2.6.32_279.14.1.el6_lustre.x86_64.x86_64.rpm lustre-ldiskfs-3.3.0-2.6.32_279.14.1.el6_lustre.x86_64.x86_64.rpm
yum install net-snmp-libs
rpm -ivh lustre-2.1.4-2.6.32_279.14.1.el6_lustre.x86_64.x86_64.rpm

Disable SELinux
Add selinux=0 to /boot/grub/grub.conf at the end of kernel line

title CentOS (2.6.32-279.14.1.el6_lustre.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-279.14.1.el6_lustre.x86_64 ro root=/dev/mapper/vg_lustremgs01-lv_root rd_LVM_LV=vg_lustremgs01/lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_lustremgs01/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet selinux=0
    initrd /initramfs-2.6.32-279.14.1.el6_lustre.x86_64.img

Disable firewall and reboot

chkconfig iptables off
reboot

Create file /etc/modprobe.d/lustre.conf, add line:

options lnet networks=tcp0(eth0)

Create a combined MGS/MDT file system on a block device

mkfs.lustre --fsname=lustre --mgs --mdt --index=0 /dev/mapper/vg_lustremgs01-lv_lustre
mkdir /mdt
mount -t lustre /dev/mapper/vg_lustremgs01-lv_lustre /mdt

Add line to /etc/fstab

/dev/mapper/vg_lustremgs01-lv_lustre /mdt lustre defaults 0 0

Deploy OSS server

The same steps as for MGS

  1. Install the same packages
  2. Disable SELinux and firewall
  3. Create /etc/modprobe.d/lustre.conf
  4. Add line to /etc/fstab

Create and mount filesystem

mkfs.lustre --fsname=lustre --mgsnode=192.168.0.1@tcp --ost --index=1 /dev/mapper/vg_lustreoss01-lv_lustre
mkdir /lustre
mount -t lustre /dev/mapper/vg_lustreoss01-lv_lustre /lustre

Deploy client

rpm -ivh lustre-client-2.1.4-2.6.32_279.14.1.el6.x86_64.x86_64.rpm lustre-client-modules-2.1.4-2.6.32_279.14.1.el6.x86_64.x86_64.rpm
mkdir /lustre
mount -t lustre 192.168.0.1@tcp:/lustre /lustre

If client has another kernel version, you need compile client

Download client source: http://downloads.whamcloud.com/public/lustre/latest-maintenance-release/

yum install lustre-client-source-2.1.4-2.6.32_279.14.1.el6.x86_64.x86_64.rpm
cd /usr/src/lustre-2.1.4
./configure
make
make install
mkdir /lustre
reboot
mount -t lustre 192.168.0.1@tcp:/lustre /lustre