LogMan.io: Window Correlator

LogMan.io Manual Page

Example

define:
  name: Example of the Window Correlator
  description: |
    Long multi-line descriptor
    That really goes to another linw
  type: correlator/window

predicate:
  !EQ
    - !ITEM EVENT Type
    - UseIt

evaluate:
  dimension: [CustomerName, DestinationAddress, DestinationHostname]  
  by: Timestamp
  resolution: 5  # seconds

analyze:
  window: hopping
  aggregate: sum
  span: 12
  test:
    !GT
    - !ARG
    - 5

trigger:
  - event:
      !DICT
      set:
        DeviceEventClassID: 281
        CategorySignificance: "/Compromise"

Section define

This section contains the common definition and meta data.

Item name

Shorter human-readable name of this declaration.

Item type

The type of this declaration, must be correlator/window.

Item description (optional)

Longed, possibly multiline, human-readable description of the declaration.

Section predicate (optional)

The predicate filters incoming events using an expression. If the expression returns True, the event will enter evaluate section. If the expression returns False, then the event is skipped.

Other returned values are undefined.

Section evaluate

The evaluate section specifies primary key, resolution and other attributes that are applied on the incoming event. The evaluate function is to add the event into the two dimensional structure, defined by a time and a primary key.

Item dimension

Specifies simple or compound primary key (or dimension) for the event. The dimension is defined by names of the input event fields.

Example of the simple primary key:

evaluate:
  dimension: CustomerName

Example of the compound primary key:

evaluate:
  dimension: [CustomerName, DestinationAddress, DestinationHostname]

Item by

Specified the name of the field of the input event that contains a date/time information, which will be used for evaluation.

Item resolution (optional)

Specifies the resolution of the time aggregation of the correlator. The unit is seconds.

evaluate:
  resolution: 3600  # 1 hour

Default value: 3600

Item saturation (optional)

Specifies the duration of the ‘silent’ time interval after the trigger is fired. It is specific for the dimension. The unit is resolution.

Default value: 3

Section analyze (optional)

The section analyze contains the configuration of the time window that is applied on the input events. The result of the time window analysis is subjected to the configurable test. When the test is successful (aka returns True), the trigger is fired.

Note: The section is optional, the default behavior is to fire the trigger when there is at least one event in the tumbling of the span equals 2.

Item window (optional)

Specifies what kind of time window to use.

Options:

  • tumbling: Fixed span (duration), non-overlapping, gap-less contiguous time intervals
  • hopping: Fixed span (duration), overlapping windows contiguous time intervals

Default value: hopping

Item span

Specifies the width of the window. The unit is resolution.

Item aggregate (optional)

Specifies what aggregation functions to be applied on events in the window.

Aggegate functions

Default value: sum

Example of the unique count:

analyze:
  window: hopping
  aggregate: unique count
  dimension: SourceAddress
  span: 6
  test:
    !GE
    - !ARG
    - 5

Trigger when 5 and more unique Source Addresses are observed.

Item test (optional)

The test is an expression that is applied on the output of the aggregate calculation. If the expression returns True, the trigger will be fired if a dimension is not already saturated. If the expression returns False, then no action is taken.

Other returned values are undefined.

Section trigger

The trigger section specifies what kinds of actions to be taken when the trigger is fired by test in the analyze section.