Baseline

The declarations for baselines are loaded from the library from the folder specified in the configuration, such as /Baseliners.

Note

The Baseliner uses /Schemas/ECS.yaml by default, so it also must be present in the library.

Declaration

This is the example of the baseline definition, located in /Baseliners folder in the library:

---
define:
    name: Dataset
    description: Creates baseline for each dataset and trigger alarms if the actual number deviates
    type: baseliner

baseline:
    region: Czech Republic
    period: day
    learning: 4  # samples (till the baseliner is "learned" for a given "key")
    classes: [workdays, weekends, holidays]

evaluate:
    key: event.dataset
    timestamp: "@timestamp"

analyze:
    test:
        !AND
        - !LT
            - !ARG VALUE
            - 1
        - !GT
            - !ARG MEAN
            - 10.0

trigger:
    - event:
            # Threat description
            # https://www.elastic.co/guide/en/ecs/master/ecs-threat.html
            threat.framework: "MITRE ATT&CK"
            threat.indicator.sightings: !ITEM EVENT value
            threat.indicator.confidence: "High"
            threat.indicator.name: !ITEM EVENT dimension

    - notification:
            type: email
            to: ["myemail@example.co"]
            template: "/Templates/Email/Notification_baseliner_dimension.md"

            variables:
                name: "Logs are not coming to the dataset within the given UTC hour."
                dimension: !ITEM EVENT dimension
                hour: !ITEM EVENT hour

Sections

baseline

Defines how the given baseline is going to be build.

The region defines in which region the activity is happening (for calculating holidays and so on).

The period says which timespan is relevant for the baseline, it can be either day or week.

The classes include which days in the week we want to monitor, it can be workdays, weekends and holidays.

predicate (optional)

Filters incoming events to be considered an activity in the baseline.

The filter is specified using SP-Lang.

evaluate

This section specifies which attributes from the event are going to be used in the baseline build.

There are two options.

The attribute key specifies the attribute/entity to monitor.

The timestamp specifies the attribute in which the time dimension of the event activity is stored in.

analyze

The test section in analyze specifies, when to run trigger, if the actual activity (!ARG VALUE) deviates from the baseline.

The following attributes are available, used in the SP-Lang notation:

TENANT: "str",
VALUE: "ui64",
STDEV: "fp64",
MEAN: "fp64",
MEDIAN: "fp64",
VARIANCE: "fp64",
MIN: "ui64",
MAX: "ui64",
SUM: "ui64",
HOUR: "ui64",
KEY: "str",
CLASS: "str",

trigger

Which trigger to run after a successful analysis. The example of the trigger is:

trigger:
    - notification:
            type: email
            to: ["myemail@example.co"]
            template: "/Templates/Email/Notification_baseliner_dimension.md"

            variables:
                name: "Logs are not coming to the dataset within the given UTC hour."
                dimension: !ITEM EVENT dimension
                hour: !ITEM EVENT hour

The argument EVENT provided in the !ARG expression contains the following attributes.

tenant

The name of the tenant the baseline belogs to.

dimension

The dimension the baseline belongs to, as specified in evaluate.

class

The class the baseline was calculated from.

Options include: workdays, weekends and holidays

hour

The number of the UTC hour the analysis happend in.

value

The value of the current counter of events for the given UTC hour.

Analysis in UI

By default, LogMan.io UI provides the following display of analysis: user and host

The analysis to use needs to be specified in the schema (default: /Schemas/ECS.yaml) in the following way:

  host.id:
    type: "str"
    analysis: host

  user.id:
    type: "str"
    analysis: user

  ...

If then the tenant is configured to use this schema (ECS by default), the host.id and user.id fields in Discover screen will show a link to the given baseline.

Analysis host will utilize the baseline, which has name Host, by default:

---
define:
  name: Host

Analysis user will utilize the baseline, which has name User, by default:

---
define:
  name: User

If the analysis cannot find its associated baseline, it will show an empty screen.

Note

Both baselines needed for analysis are distributed as part of LogMan.io Common Library.