Cascade Parser

Example

---
define:
  name: Syslog RFC5424
  type: parser/cascade
  field_alias: field_alias.default
  encoding: utf-8  # none, ascii, utf-8 ... (default: utf-8)
  target: parsed # optional, specify the target of the parsed event (default: parsed)

predicate:
  !CONTAINS
  what: !EVENT
  substring: 'ASA'

parse:
  !REGEX.PARSE
  what: !EVENT
  regex: '^(\w{1,3}\s+\d+\s\d+:\d+:\d+)\s(?:(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})|([^\s]+))\s%ASA-\d+-(.*)$'
  items:
    - rt:
        !DATETIME.PARSE
        value: !ARG
        format: '%b %d %H:%M:%S'
        flags: Y
    - dvchost

Section define

This section contains the common definition and meta data.

Item name

Shorter human-readable name of this declaration.

Item type

The type of this declaration, must be parser/cascade.

Item field_alias

Name of the field alias lookup to be loaded, so that alias names of event attributes can be used in the declaration alongside their canonical names.

Item encoding

Encoding of the incoming event.

Item target (optional)

Default target pipeline of the parsed event, unless specified differently in context. The options include: parsed, lookup, unparsed

Item description (optional)

Longed, possibly multiline, human-readable description of the declaration.

Section predicate (optional)

The predicate filters incoming events using an expression. If the expression returns True, the event will enter parse section. If the expression returns False, then the event is skipped.

Other returned values are undefined.

This section can be used to speed-up parsing by skipping lines with obviously non-relevant content.

Section parse

This section specifies the actual parsing mechanism. It expects a dictionary to be returned or None, which means that the parsing was not successful.

Typical statements in parse section

!FIRST statement allows to specify a list of parsing declarations, which will be evaluated in the order (top-down), the first declaration which returns non-None value stops the iteration and this value is returned.

!REGEX.PARSE statement allows to transform the log line into a dictionary structure. It also allows to attach sub-parsers to further decompose substrings.

Output routing

To indicate that the parser will not parse the event it received so far, an attribute target needs to be set to unparsed within the context. Then, other parsers in the pipeline may receive and parse the event.

In the same way, the target can be set to different destination groups, such as parsed.

To set the target in the context, the !CONTEXT.SET is used:

- !CONTEXT.SET
  what: <... expression ...>
  set:
    target: unparsed

Example of use in the parser. If no regex matches the incoming event, event is posted to unparsed target, so other parsers in the row may process it.

!FIRST
- !REGEX.PARSE
what: !EVENT
regex: '^(one)\s(two)\s(three)$'
items:
  - one
  - two
  - three
- !REGEX.PARSE
what: !EVENT
regex: '^(uno)\s(duo)\s(tres)$'
items:
  - one
  - two
  - three
# This is where the handling of partially parsed event starts
- !CONTEXT.SET
set:
  target: unparsed
- !DICT
set:
  unparsed: !EVENT