Skip to content

LogMan.io Depositor

TeskaLabs LogMan.io Depositor is a microservice responsible for storing events in Elasticsearch and setting up Elasticsearch artifacts (like index templates and ILM policies) based on event lane declarations. LogMan.io Depositor stores the successfully parsed or correlated events and other events in their proper Elasticsearch indices.

Note

LogMan.io Depositor replaces LogMan.io Dispatcher.

Important notes

Prerequisites and configuration

  • Depositor requires a specific Elasticsearch setting with node roles provided, see Prerequisites
  • Depositor's default lifecycle policy requires node roles to be set in Elasticsearch's configuration, see Prerequisites
  • Depositor by default stops sending data to Elasticsearch if cluster health is below 50 %, see Configuration
  • Depositor considers all event lane files regardless of if they are disabled for the given tenant in the UI or not

Index management

  • Depositor creates its own index template and lifecycle policy (ILM) for each index specified in the events and others sections within the event lane declaration, see Event Lane
  • Depositor's default index template has 6 shards and 1 replica
  • The field mapping (types of the fields) in the index template are based on the schema, which by default is /Schemas/ECS.yaml, unless specified in the configuration or event lane, see Event Lane

Lifecycle details

  • Depositor's default lifecycle policy has limit of 16 GB per primary shard per index (the default maximum index size is thus 6 shards * 16 GB * 2 for replica = 192 GB)
  • Depositor's default lifecycle policy has shrinking enabled when entering the warm phase
  • Depositor's default lifecycle policy deletes data after 180 days

Migration

  • When migrating LogMan.io Dispatcher to LogMan.io Depositor, see the Migration section