Lustre file system - simple deploy

Posted on Sun 13 January 2013

OS: CentOS 6.3
MGS server: lustre-mgs IP
OSS server: lustre-oss-01 IP
Storage: local drive on OSS server

Download Lustre packages:
Download modified e2fsprogs packages:

Deploy MGS server

rpm -ivh kernel-2.6.32-279.14.1.el6_lustre.x86_64.rpm kernel-firmware-2.6.32-279.14.1.el6_lustre.x86_64.rpm
rpm -Uvh e2fsprogs-1.42.3.wc3-7.el6.x86_64.rpm libcom_err-1.42.3.wc3-7.el6.x86_64.rpm e2fsprogs-libs-1.42.3.wc3-7.el6.x86_64.rpm libss-1.42.3.wc3-7.el6.x86_64.rpm
rpm -ivh lustre-modules-2.1.4-2.6.32_279.14.1.el6_lustre.x86_64.x86_64.rpm lustre-ldiskfs-3.3.0-2.6.32_279.14.1.el6_lustre.x86_64.x86_64.rpm
yum install net-snmp-libs
rpm -ivh lustre-2.1.4-2.6.32_279.14.1.el6_lustre.x86_64.x86_64.rpm

Disable SELinux
Add selinux=0 to /boot/grub/grub.conf at the end of kernel line

title CentOS (2.6.32-279.14.1.el6_lustre.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-279.14.1.el6_lustre.x86_64 ro root=/dev/mapper/vg_lustremgs01-lv_root rd_LVM_LV=vg_lustremgs01/lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_lustremgs01/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet selinux=0
    initrd /initramfs-2.6.32-279.14.1.el6_lustre.x86_64.img

Disable firewall and reboot

chkconfig iptables off

Create file /etc/modprobe.d/lustre.conf, add line:

options lnet networks=tcp0(eth0)

Create a combined MGS/MDT file system on a block device

mkfs.lustre --fsname=lustre --mgs --mdt --index=0 /dev/mapper/vg_lustremgs01-lv_lustre
mkdir /mdt
mount -t lustre /dev/mapper/vg_lustremgs01-lv_lustre /mdt

Add line to /etc/fstab

/dev/mapper/vg_lustremgs01-lv_lustre /mdt lustre defaults 0 0

Deploy OSS server

The same steps as for MGS

  1. Install the same packages
  2. Disable SELinux and firewall
  3. Create /etc/modprobe.d/lustre.conf
  4. Add line to /etc/fstab

Create and mount filesystem

mkfs.lustre --fsname=lustre --mgsnode= --ost --index=1 /dev/mapper/vg_lustreoss01-lv_lustre
mkdir /lustre
mount -t lustre /dev/mapper/vg_lustreoss01-lv_lustre /lustre

Deploy client

rpm -ivh lustre-client-2.1.4-2.6.32_279.14.1.el6.x86_64.x86_64.rpm lustre-client-modules-2.1.4-2.6.32_279.14.1.el6.x86_64.x86_64.rpm
mkdir /lustre
mount -t lustre /lustre

If client has another kernel version, you need compile client

Download client source:

yum install lustre-client-source-2.1.4-2.6.32_279.14.1.el6.x86_64.x86_64.rpm
cd /usr/src/lustre-2.1.4
make install
mkdir /lustre
mount -t lustre /lustre