Each server within the cluster should have:
- a dedicated fast storage on SSD storage
- a dedicated slow storage on traditional hard drives (HDD).
Optinally, the system can be connected to an archive storage.
Manage using LVM
Assuming there is:
- an HDD drive at
- an SSD drive at
# Run as ROOT pvcreate /dev/sdb /dev/sdd vgcreate hdd_vg /dev/sdb vgcreate ssd_vg /dev/sdd lvcreate -l 100%FREE -n hdd_lv hdd_vg lvcreate -l 100%FREE -n ssd_lv ssd_vg mkdir -p /data/ssd /data/hdd mkfs.xfs /dev/mapper/hdd_vg-hdd_lv mkfs.xfs /dev/mapper/ssd_vg-ssd_lv
Add these lines into
/dev/mapper/hdd_vg-hdd_lv /data/hdd xfs defaults 0 0 /dev/mapper/ssd_vg-ssd_lv /data/ssd xfs defaults 0 0
# Run as ROOT mount -a
Fast storage contains data, that are needed to be quickly accessed, such as persisted Kafka logs/messages and data for ElasticSearch master and hot data nodes.
The fast storage has to be mounted on
Inside, each folder is named according to the Docker container in the LogMan.io Deployment:
/data/ssd/kafka /data/ssd/zookeeper /data/ssd/influxdb /data/ssd/es-master /data/ssd/es-hot1 ...
Size and redundancy
The fast storage must be located at mirrored dist space (aka RAID 1). That means that you need twice the physical capacity for a given logical capacity.
We recommend to have 1TB of the logical fast storage capacity.
- 1x Swap partition
- 1x LVM partition
- LVM provides one logical volume (LV) that is mirrored over two physical devices
- This logical volume uses Ext4 filesystem and it is mounted at
noatimeflag is recommended to be set
Important: The total swap size should always be the same size of RAM memory, stored on fast SSD disk storage. When two SSD drives are used, half of the swap size should be allocated on the each drive for a swap.
The slow storage contains data, that does not have to be quickly accessed, and usually contain older logs and events, such as warm and cold indices for ElasticSearch.
The slow storage has to be mounted to
Inside, each folder is named according to the Docker container:
/data/ssd/es-warm1 /data/ssd/es-cold1 /data/hdd/docker ...
/data/hdd/docker is sym-linked so to
Size and redundancy
We recommend to apply disk storage redundancy such as RAID6 on the slow storage space. This means that the physical storage size will be bigger than a logical size.
A single big partition with Ext4 or XFS.
noatime flag is recommended to be set.
The very old data could be moved into the archive for a long-term storage. The archive could be network attached storage (NAS), cloud-base storage such as AWS S3 etc.
The offline archive is optional component.
The data in the archive are not readilly available in the system and has to be restored back prior their use.