One of the applications that I’m working on uses archived documents, there is no NoSQL here, just plain tiff files with indexes in Oracle database. Everything related to document access, permissions, conversions, watermarking, security, encryption … is managed by the application itself. So I had to keep my cluster permanently connected to a SAN storage via HBA fiber cards.
In the beginning I opted for NFS as shared file system, then decided to go for OCFS2 which is open source, maintained by Oracle, and available under GPLv2.
Some Pros/Cons of NFS, GFS2 and OCFS2 (from Dublin OSS barcamp) :
NFS
- Pro: standard, cross-platform, easy to implement
- Con: Poor performance, single point of failure (single locking manager, even in HA)
GFS2
- Pro: Very responsive on large data files, works on physical and virtual, quota and SE-Linux support, faster than EXT3 when I/O operations are on the same node
- Con: Only supported with Red Hat, Performance issues on accessing small files on several subdirectory on different nodes
OCFS2
- Pro: Very fast with large and small data files on different node with two types of performance models (mail, data file). Works on a physical and virtual.
- Con: Supported only through contract with Oracle or SLES, no quota support, no on-line resize
First we need to install OCFS2 tools :
sudo apt-get install ocfs2-tools
There is another package ocfs2console that you want to install to configure the cluster via GUI, but since I’m using ubuntu server I’m skipping this to configure my cluster manually.
Create on every node attached to storage /etc/ocfs2/cluster.conf
sudo vi /etc/ocfs2/cluster.conf
With the content below, only replace node1 and node2 with their respective names and IP for each node :
node:
name = node1
cluster = ocfs2
number = 0
ip_address = 10.10.0.0
ip_port = 7777
node:
name = node2
cluster = ocfs2
number = 1
ip_address = 10.10.0.1
ip_port = 7777
cluster:
name = ocfs2
node_count = 2
Now you reconfigure ocfs2-tools with the default values :
sudo dpkg-reconfigure ocfs2-tools
then restart services :
sudo /etc/init.d/o2cb restart
sudo /etc/init.d/ocfs2 restart
If your fiber card connected to your host/storage, and virtual disks created and presented you should run fdisk to see it :
$ sudo fdisk -l
Disk /dev/sda: 1073.7 GB, 1073741824000 bytes
255 heads, 63 sectors/track, 130541 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x02020202
the result have been truncated to show only one virtual disk, and you might see multiple /dev/sda, /dev/sdb, /dev/sdc… according to your configuration in addition to your local hard disks. What I have done is creating a 1TB partition that I will share between my two nodes :
$ sudo fdisk /dev/sda
In fdisk menu choose “n” for new partition, and choose your partition size according to your requirements. Then use “w” to write changes and exit.
Finally we create a ocfs2 partition table :
$ mkfs.ocfs2 /dev/sda
mount your partition :
$ sudo mkdir /archives
$sudo mount -t ocfs2 /dev/sda /archives
or you can add it to fstab to mount automatically on boot :
$ /dev/sda /archives ocfs2 _netdev 0 0
The _netdev option is used here to prevent the system from attempting to mount these file systems until the network has been enabled on the system.
You want to test your new partition, and you will notice that every file/folder created on node1 is automatically available on node2, and vise-versa.
Enjoy !