Manual Installation of LogMan.io

Prerequisites

Docker is necessary for deployment of all subsequent applications in containers, namely Kafka, ElasticSearch, NGINX and individual pumps.

Docker Compose serves to pull and deploy multiple images at once using a configuration file named docker-compose.yml.

Prepare the System

Create basic folder structure

mkdir -p \
  /data/ssd/zookeeper/data \
  /data/ssd/zookeeper/log \
  /data/ssd/kafka/data \
  /data/ssd/elasticsearch/es-master/data \
  /data/ssd/elasticsearch/es-h01/data \
  /data/ssd/elasticsearch/es-w01/data \
  /data/hdd/elasticsearch/es-c01/data \
  /data/ssd/influxdb/data \
  /data/hdd/nginx/log

Change ownership to elasticsearch data folder:

# Run as root
chown -R 1000:0 /data/ssd/elasticsearch

Increase max virtual memory areas (for elasticsearch container)

# Run as root
sysctl -w vm.max_map_count=262144

Persist the configuration. Write this contents into file /etc/sysctl.d/01-vm.conf

vm.max_map_count=262144

Deploy and Run

Clone the site configuration files into the /opt folder:

cd /opt
git clone https://gitlab.com/TeskaLabs/<PARTNER_GROUP>/<MY_CONFIG_REPO_PATH>

Login to docker.teskalabs.com.

cd <MY_CONFIG_REPO_PATH>
docker login docker.teskalabs.com

Enter the repository and deploy the server specific Docker Compose file:

docker-compose -f docker-compose-<SERVER_ID>.yml pull
docker-compose -f docker-compose-<SERVER_ID>.yml build
docker-compose -f docker-compose-<SERVER_ID>.yml up -d

Check that all containers are running:

docker ps

Scale

Multiple instances of the same image are created using Docker Compose Scale option, which will thus ensure horizontal scaling of the container.

To scale a specific container, use the following command:

docker-compose scale syslog-lmio-parser=3

There are going to be three containers with the same name, only differing in the instance number (see above).

Utilities