Skip to content

Stashing Correlator

In the field of cybersecurity, a stashing correlator is a tool that collects and temporarily stores related events based on specific identifiers, such as message IDs. This process helps track and analyze sequences of events over time, allowing for the identification of patterns or anomalies that may indicate security threats. By grouping related events, the stashing correlator enhances the ability to monitor and respond to potential cyber incidents, providing a comprehensive view of activities that may not be apparent when examining individual events alone.

Analogy for Better Understanding

Think of the stashing correlator as a detective piecing together clues from different sources. Imagine a detective gets two notes: one with a person's name and another with their location. Individually, these notes don't give much information, but the detective notices both notes have the same case number (ID). The detective combines them to understand who is where. Similarly, the stashing correlator joins related events using a common attribute, like an ID, to create a complete picture for further analysis.

Example Scenario

Imagine you receive two pieces of mail: one telling you who sent a letter and the other telling you who received it. Individually, these pieces of mail don't provide the full picture. The stashing correlator is like a smart assistant that notices both pieces have the same tracking number (message ID) and combines them into one comprehensive record, showing both the sender and the recipient. This helps in understanding the full context of the communication.

Consider the following logs from a Postfix event lane for a given tenant:

June 14 09:19:21 alice postfix/qmgr[59833]: F3710A248D: from=<alice@example.com>, size=304, nrcpt=1 (queue active)
June 14 09:19:21 alice postfix/local[60446]: F3710A248D: to=<bob@example.com>, orig_to=<alice>, relay=local, delay=0.04, delay>

These logs are processed separately by LogMan.io Parsec. The first log indicates who sent the email (alice@example.com), and the second log shows the recipient (bob@example.com). The parsed logs look like this:

# Log #1
{
    "email.message_id": "F3710A248D",
    "email.from.address": ["alice@example.com"],
    ...
}

# Log #2
{
    "email.message_id": "F3710A248D",
    "email.to.address": ["bob@example.com"],
    ...
}

To connect all the information for future analysis, it is necessary to consolidate the events into a single log that contains both the sender and recipient information. The stashing correlator performs this task by joining the parsed events using a common attribute, such as the message ID (F3710A248D). If no other event with the same message ID arrives within the send_after_seconds period, a new event is created that includes all the gathered information:

# Stashed log
{
    "email.message_id": "F3710A248D",
    "email.from.address": ["alice@example.com"],
    "email.to.address": ["bob@example.com"],
    ...
}

This consolidated log can then be analyzed by other detection correlators, such as the Window Correlator, to further investigate and respond to potential security incidents.

Sample

The following sample stashes events by their message IDs and sends them after 10 seconds of no activity:

---
define:
  name: "Stashing events by message ID"
  description: "Example for stashing events by message ID"
  type: correlator/stashing

logsource:
  vendor: [WietseVenama]

predicate:
  !IN
  what: message.id
  where: !EVENT

stash:
  dimension: message.id
  send_after_seconds: 10  # Send the stashed message after seconds with no activity

Section define

This section contains the common definition and metadata.

Item name

Shorter human-readable name of this declaration.

Item type

The type of this declaration, must be correlator/stashing.

Item description (optional)

Longer, possibly multiline, human-readable description of the declaration.

Section logsource

Specifies the sources of the logs, indicating the vendors or products whose events coming from event lanes will be processed.

Section predicate

The predicate filters incoming events using an expression. If the expression returns !!true, the event will enter the stash section. If !!false, the event is skipped.

Include of nested predicate filters

Predicate filters are expressions located in a dedicated file, that can be included in many different predicates as their parts.

To include an external predicate filter:

!INCLUDE /predicate_filter.yaml

where /predicate_filter is the path of the file in the library.

Section stash

Specifies the stashing behavior, including the dimension for grouping events and the timeout for sending stashed events after inactivity.

Item dimension

Specifies the primary key (or dimension) for the event. Defined by names of the input event fields.

Example of a primary key:

stash:
  dimension: [message.id]

Item send_after_seconds

Specifies the timeout for sending stashed events after a period of inactivity. The unit is seconds.

stash:
  send_after_seconds: 10  # 10 seconds

Summary

In summary, the stashing correlator acts as a smart organizer, collecting and joining related events to create a full picture for further analysis. This helps in identifying patterns and anomalies in cybersecurity, providing a more effective way to monitor and respond to potential incidents.