Skip to content

Declarations for defining baselines

The declarations for baselines are loaded from the Library from the folder specified in the configuration, such as /Baseliners.

Note

The Baseliner uses /Schemas/ECS.yaml by default, so /Schemas/ECS.yaml must also be present in the Library.

Declaration

This is an example of a baseline definition, located in the /Baseliners folder in the Library:

---
define:
    name: Dataset
    description: Creates baseline for each dataset and trigger alarms if the actual number deviates
    type: baseliner

baseline:
    region: Czech Republic
    period: day
    learning: 4
    classes: [workdays, weekends, holidays]

evaluate:
    key: event.dataset
    timestamp: "@timestamp"

analyze:
    test:
        !AND
        - !LT
            - !ARG VALUE
            - 1
        - !GT
            - !ARG MEAN
            - 10.0

trigger:
    - event:
            # Threat description
            # https://www.elastic.co/guide/en/ecs/master/ecs-threat.html
            threat.framework: "MITRE ATT&CK"
            threat.indicator.sightings: !ITEM EVENT value
            threat.indicator.confidence: "High"
            threat.indicator.name: !ITEM EVENT dimension

    - notification:
            type: email
            to: ["myemail@example.co"]
            template: "/Templates/Email/Notification_baseliner_dimension.md"

            variables:
                name: "Logs are not coming to the dataset within the given UTC hour."
                dimension: !ITEM EVENT dimension
                hour: !ITEM EVENT hour

Sections

baseline

baseline: # (1)
    region: Czech Republic #(2)
    period: day # (3)
    learning: 4  # (4)
    classes: [workdays, weekends, holidays] # (5)
  1. Defines how the given baseline is built.
  2. Defines in which region the activity is happening (for calculating holidays and so on).
  3. Defines the timespan for the baseline. The period can be either day or week.
  4. Defines the number of periods (here day) that occur from the baseliner beginning to receive input until the user can see the baseline analysis. Additional details below.
  5. Define which days in the week we want to monitor. Classes can include any/all: workdays, weekends, and holidays.

learning
The learning field defines the learning phase.
The learning phase is the time from the first occurrence of the dimension value in the input of the Baseliner instance until the point when the baseline is shown to the user and the analysis takes place. In the declaration, learning is the number of periods. The learning phase is calculated separately for holidays, weekends and working days. Baselines are rebuilt overnight (housekeeping).

In this example, the period is day, so learning is 4 days. Considering the calendar, a learning phase of 4 days beginning on Friday means 4 working days, and thus ends on Wednesday night.

predicate (optional)

The predicate section filters incoming events to be considered as activity in the baseline.

Write filters with TeskaLabs SP-Lang. Visit Predicates or the SP-Lang documentation for details.

evaluate

This section specifies which attributes from the event are going to be used in the baseline build.

evaluate:
    key: event.dataset # (1)
    timestamp: "@timestamp" # (2)
  1. Specifies the attribute/entity to monitor.
  2. Specifies the attribute in which the time dimension of the event activity is stored in.

analyze

The test section in analyze specifies when to run the trigger, if the actual activity (!ARG VALUE) deviates from the baseline. Write tests in SP-Lang.

analyze:
    test:
        !AND #(1)
        - !LT #(2)
            - !ARG VALUE # (3)
            - 1
        - !GT # (4)
            - !ARG MEAN # (5)
            - 10.0
  1. All expressions nested under AND must be true for the test to pass. Here, if the value is less than 1 and the mean is greater than 10, the trigger is run.
  2. "Less than"
  3. Get (!ARG) the value (VALUE). If the value is less than 1 as specified, the !LT expression is true.
  4. "Greater than"
  5. Get (!ARG) the mean (MEAN). If the mean is greater than 10.0 as specified, the !GT expression is true.

The following attributes are available, used in SP-Lang notation:

TENANT: "str",
VALUE: "ui64",
STDEV: "fp64",
MEAN: "fp64",
MEDIAN: "fp64",
VARIANCE: "fp64",
MIN: "ui64",
MAX: "ui64",
SUM: "ui64",
HOUR: "ui64",
KEY: "str",
CLASS: "str",

trigger

The trigger section defines the activity that is triggered to run after a successful analysis. (More about triggers.)

Baseliner creates events

Upon every analysis (every hour), Baseliner creates an event to summarize its analysis. These Baseliner-created events are available to use (as EVENT) with expressions such as !ARG and !ITEM, meaning you can pull values from the events for your trigger activities.

These Baseliner-created events include the fields:

tenant
The name of the tenant the baseline belogs to.

dimension
The dimension the baseline belongs to, as specified in evaluate.

class
The class the baseline was calculated from.
Options include: workdays, weekends, and holidays

hour
The number of the UTC hour the analysis happend in.

value
The value of the current counter of events for the given UTC hour.

Notification trigger

A notification trigger sends a message, such as an email. See Email notifications for more details about sending email notifications and using email templates.

An example of a notification trigger:

trigger:
    - notification:
            type: email #(1)
            to: ["myemail@example.co"] # (2)
            template: "/Templates/Email/Notification_baseliner_dimension.md" # (3)

            variables: # (4)
                name: "Logs are not coming to the dataset within the given UTC hour."
                dimension: !ITEM EVENT dimension # (5)
                hour: !ITEM EVENT hour
  1. Specifies an email notification
  2. Recipient address
  3. Filepath to the email template in the LogMan.io Library
  4. Begins the section that gives directions for how to fill the blank fields from the email template. The blank fields in the template being used in this example are name, dimension, and hour.
  5. Uses SP-Lang to get information (!ITEM) from the Baseliner-created EVENT (detailed below). In this case, the template field dimension will be filled with the value of dimension taken from the Baseliner-created event.

Event trigger

You can use an event trigger to create a log or event, which you'll be able to see in the TeskaLabs LogMan.io UI.

Example of an event trigger:

    - event: # (1)
            threat.framework: "MITRE ATT&CK"
            threat.indicator.sightings: !ITEM EVENT value
            threat.indicator.confidence: "High"
            threat.indicator.name: !ITEM EVENT dimension
  1. This new event is a threat description using threat fields from Elasticsearch

Analysis in UI

By default, the LogMan.io UI provides displays of analyses for user and host.

Specify the analysis in the schema (default: /Schemas/ECS.yaml) like this:

  host.id:
    type: "str"
    analysis: host

  user.id:
    type: "str"
    analysis: user

  ...

If then the tenant is configured to use this schema (ECS by default), the host.id and user.id fields in Discover will show a link to the given baseline.

Analysis host uses the baseline named Host by default:

---
define:
  name: Host

Analysis user uses the baseline named User by default:

---
define:
  name: User

If a specific analysis cannot locate its associated baseline, the UI will display an empty screen for that analysis.

Note

Both baselines needed for analysis are distributed as part of the LogMan.io Common Library.