Manual Installation

TeskaLabs LogMan.io could be installed manually on compute resources. It is a common approach in large organizations or somehow specific deployments. Compute resources include physical servers, virtual servers, private and public cloud compute/VM instances and so on.

IMPORTANT: TeskaLabs LogMan.io CANNOT BE operated under root user (superuser). Violation of this rule may lead to a significant cybersecurity risks.

Prerequisities

  • Hardware (physical or virtualized)
  • OS Linux: RedHat 8 and 7, CentOS 7, Ubuntu 20.04 LTS (for others, kindly contact our support)
  • Outgoing access to the Internet (could be restricted after the initial installation)
  • SMTP server
  • IP connectivity
  • DNS domain, even internal for HTTPS setup
  • Credentials to docker.teskalabs.com (contact our support if you don’t have one)

Prepare the Operating System

The OS could be delivered by dedicated team, using company standard for Linux OS. The maintenance responsibility could be clearly divided between teams responsible for OS maintenance and LogMan.io support setup.

Install packages

  • Docker
  • Docker Compose
  • git command
    • For management of site specific configs, declarations, docker-compose files live in Git repositories

Docker is necessary for deployment of all subsequent applications in containers, namely Apache Kafka, ElasticSearch, NGINX and individual streaming pumps.

Docker Compose serves to pull and deploy multiple images at once using a configuration file named docker-compose.yml.

Mount data disks

Fast storage (SSD) needs to be mounted at /data/ssd. Slow storage (HDD) needs to be mounted at /data/hdd. Make sure that the disk configuration is persisted at /etc/fstab.

The recommended filesystem for LogMan.io data storage is xfs due to its simple extendability.

For more details about a Disk Organization, continue here .

Create basic folder structure

mkdir -p \
  /data/ssd/zookeeper/data \
  /data/ssd/zookeeper/log \
  /data/ssd/kafka/data \
  /data/ssd/elasticsearch/es-master/data \
  /data/ssd/elasticsearch/es-h01/data \
  /data/ssd/elasticsearch/es-w01/data \
  /data/hdd/elasticsearch/es-c01/data \
  /data/ssd/influxdb/data \
  /data/hdd/nginx/log

Change ownership to elasticsearch data folder:

chown -R 1000:0 /data/ssd/elasticsearch
chown -R 1000:0 /data/hdd/elasticsearch

Increase the VM max_map_count parameter

Write this contents into file /etc/sysctl.d/01-vm.conf

vm.max_map_count=262144

Reboot the system.

reboot

Deploy and Run

Clone the site configuration files into the /opt folder:

cd /opt
git clone https://gitlab.com/TeskaLabs/<PARTNER_GROUP>/<MY_CONFIG_REPO_PATH>

Login to docker.teskalabs.com.

cd <MY_CONFIG_REPO_PATH>
docker login docker.teskalabs.com

Enter the repository and deploy the server specific Docker Compose file:

docker-compose -f docker-compose-<SERVER_ID>.yml pull
docker-compose -f docker-compose-<SERVER_ID>.yml build
docker-compose -f docker-compose-<SERVER_ID>.yml up -d

Check that all containers are running:

docker ps

Scale up

Multiple instances of the same image are created using Docker Compose Scale option, which will thus ensure horizontal scaling of the container.

To scale a specific container, use the following command:

docker-compose scale syslog-lmio-parser=3

There are going to be three containers with the same name, only differing in the instance number (see above).

Utilities