Skip to content

LogMan.io Commander

LogMan.io Commander allows to run the following utility commands via command line or API.

encpwd command

Passwords used in configurations can be protected by encryption.

Encrypt Password command encrypts password(s) to LogMan.io password format using AES cipher.

The passwords are then used in LogMan.io Collector declarative configuration in the following format:

!encpwd "<LMIO_PASSWORD>"

Configuration

The default AES key path can be configured in the following way:

[pwdencryptor]
key=/data/aes.key

Usage

Docker container

Command Line
docker exec -it lmio-commander lmiocmd encpwd MyPassword
API

LogMan.io Commander also serves an API endpoint, so the encpwd command can be reached via HTTP call:

curl -X POST -d "MyPassword" http://lmio-commander:8989/encpwd

library command

Library command serves to insert library folder structure with all YAML declarations into ZooKeeper, where other components such as LogMan.io Parser and Correlator may dynamically download it from.

The folder structure can be located in the filesystem (mounted to the Docker container) or on GIT url.

This is how to initiate loading of the library into ZooKeeper cluster:

Configuration

It is necessary to properly configure the source folder and ZooKeeper output.

[source]
path=/library

[destination]
urls=zookeeper:12181
path=/lmio

The source path can be a GIT repository path prefixed with git://:

[source]
path=git://<username>:<deploy_token>@<git_url_path>.git

In this way, the library will be automatically cloned from GIT into a temporary folder, uploaded to ZooKeeper and then the temporary folder deleted.

Usage

Docker container

Command Line
docker exec -it lmio-commander lmiocmd library load

Using explicitly defined configuration:

docker exec -it lmio-commander lmiocmd -c /data/lmio-commander.conf library load
API

LogMan.io Commander also serves an API endpoint, so the library command can be reached via HTTP call:

curl -X PUT http://lmio-commander:8989/library/load

See Docker Compose section below.

iplookup command

The iplookup command processes IP range databases and generates IP lookup files ready for use with lmio-parser IP Enricher. It has two subcommands: iplookup from-csv for processing general CSV files and iplookup from-ip2location for processing IP2LOCATION CSV files.

Configuration

The source and destination directories can be set in a config file 

[iplookup]
source_path=/data
destination_path=/data

iplookup from-csv

Reads a generic CSV file and produces an IP Enricher lookup file. The first row of the file is expected to be the header containing the column names. The first two columns need to be ip_from and ip_to.

Command line interface

lmiocmd.py iplookup from-csv [-h] [--separator SEPARATOR] [--zone-name ZONE_NAME] [--gzip] [--include-ip-range] file_name

Positional arguments:

  • file_name: Input CSV file

Optional arguments:

  • -h, --help: Show this help message and exit.
  • -g, --gzip: Compress output file with gzip.
  • -i INPUT_IP_FORMAT, --input-ip-format INPUT_IP_FORMAT: Format of input IP addresses. Defaults to 'ipv6'. Possible values:
  • ipv6: IPv6 adrress represented as string, e.g. ::ffff:c000:02eb,
  • ipv4: standard quad-dotted IPv4 adrress string, e.g. 192.0.2.235,
  • ipv6-int: IPv6 adrress as a 128-bit decimal integer, e.g. 281473902969579,
  • ipv4-int: IPv4 address as a 32-bit decimal integer, e.g. 3221226219.
  • -s SEPARATOR, --separator SEPARATOR: CSV column separator.
  • -o LOOKUP_NAME, --lookup-name LOOKUP_NAME: Name of output lookup. It is used as lookup zone name. By default, it is derived from input file name.
  • --include-ip-range: Include ip_from and ip_to fields in the lookup values.
  • --force-ipv4: Prevent mapping IPv4 addresses to IPv6. This is incompatible with IPv6 input formats.
Example usage
lmiocmd iplookup from-csv \
--input-ip-format ipv6 \
--lookup-name ip2country \
--gzip \
my-ipv6-zones.CSV

iplookup from-ip2location

This command is similar to the iplookup from-csv command above, but is tailored specifically for processing IP2Location™ CSV databases. In case of IP2LOCATION LITE databases, the command can infer the input IP format and the column names from the file name. However, it is possible to specify the column names explicitly

Command line interface

lmiocmd.py iplookup from-csv [-h] [--separator SEPARATOR] [--zone-name ZONE_NAME] [--gzip] [--include-ip-range] file_name

Positional arguments:

  • file_name: Input CSV file

Optional arguments:

  • -h, --help: Show this help message and exit.
  • -g, --gzip: Compress output file with gzip.
  • -s SEPARATOR, --separator SEPARATOR: CSV column separator. Defaults to ','.
  • -c COLUMN_NAMES, --column-names COLUMN_NAMES: Space-separated list of column names to use. By default, it is inferred from IP2LOCATION file name.
  • -i INPUT_IP_FORMAT, --input-ip-format INPUT_IP_FORMAT: Format of input IP addresses. By default, it is inferred from IP2LOCATION file name. Possible values:
  • ipv6-int: IPv6 adrress as a 128-bit decimal integer, e.g. 281473902969579,
  • ipv4-int: IPv4 address as a 32-bit decimal integer, e.g. 3221226219.
  • -o LOOKUP_NAME, --lookup-name LOOKUP_NAME: Name of output lookup. It is used as lookup zone name. By default, it is derived from input file name.
  • -e, --keep-empty-rows: Do not exclude rows with empty values (indicated by '-').
  • --include-ip-range: Include ip_from and ip_to fields in the lookup values.
  • --force-ipv4: Prevent mapping IPv4 addresses to IPv6.
Example usage

With automatic column names and input IP format:

lmiocmd iplookup from-ip2location \
--lookup-name ip2country \
--gzip \
IP2LOCATION-LITE-DB1.IPV6.CSV

With explicit column names and input IP format (the result will be equivalent to the example above):

lmiocmd iplookup from-ip2location \
--lookup-name ip2country \
--gzip \
--column names "ip_from ip_to country_code country_name" \
--input-ip-format ipv6-int
IP2LOCATION-LITE-DB1.IPV6.CSV

Docker Compose

File

The following docker-compose.yml file pulls the LogMan.io Commander image from TeskaLabs' Docker Registry and expects the configuration file in ./lmio-commander folder.

version: '3'
services:
  lmio-commander:
    image: docker.teskalabs.com/lmio/lmio-commander
    container_name: lmio-commander
    volumes:
      - ./lmio-commander:/data
      - /opt/lmio-library:/library
    ports:
      - "8989:8080"

The /opt/lmio-library path leads to LogMan.io Library repository.

Run the container

docker-compose pull
docker-compose up -d