Parsing key terms¶
Important terms relevant to LogMan.io Parsec and the parsing process.
Event¶
A unit of data that moves through the parsing process is referred to as an event. An original event comes to LogMan.io Parsec as an input and is then parsed by the processors. If parsing succeeds, it produces a parsed event, and if parsing fails, it produces an error event.
Original event¶
An original event is the input that LogMan.io Parsec recieves - in other words, an unparsed log. It can be represented by a raw (possibly encoded) string or structure in JSON or XML format.
Parsed event¶
A parsed event is the output from successful parsing, formatted as an unordered list of key-value pairs serialized into JSON structure. A parsed event always contains a unique ID, the original event, and typically the information about when the event was created by the source and received by Apache Kafka.
Error event¶
An error event is the output from unsuccessful parsing, formatted as an unordered list of key-value pairs serialized into JSON structure. It is produced when parsing, mapping, or enrichment fails, or when another exception occurs in LogMan.io Parsec. It always contains the original event, the information about when the event was unsuccessfully parsed, and the error message describing the reason why the process of parsing failed. Despite unsuccessful parsing, the error event will always be in JSON format, key-value pairs.
Library¶
Your TeskaLabs LogMan.io Library holds all of your declaration files (as well as many other types of files). You can edit your declaration files in your Library via Zookeeper.
Declarations¶
Declarations describe how the event will be transformed. Declarations are YAML files that LogMan.io Parsec can interpret to create declarative processors. There are three types of declarations in LogMan.io Parsec: parsers, enrichers, and mappings. See Declarations for more.
Parser¶
A parser is the type of declaration that takes the original event or a specific field of a partially-parsed event as input, analyzes its individual parts, and then stores them as key-value pairs to the event.
Mapping¶
A mapping declaration is the type of declaration that takes a partially parsed event as input, renames the field names, and eventually converts the data types. It works together with a schema (ECS, CEF). It also works as a filter to leave out data that is not needed in the final parsed event.
Enricher¶
An enricher is the type of declaration that supplement a partially parsed event with additional data.