Skip to content

Streams in LogMan.io Elman

Detection of new streams in LogMan.io Elman

LogMan.io Elman periodically detects received.* topics in Kafka cluster. If a new received topic without an existing event lane is found, a new event lane is created using event lane templates.

For known sources, there is an event lane template, which is used primarily for choosing parsing rules for that event lane. LogMan.io Elman seeks an appropriate template in /Templates/EventLanes/. One of the following cases can happen:

  1. The event lane template is found in the library. Parsing rules and other specific content is then copied to the event lane. After that, a model for LogMan.io Parsec is created in /Site/ and LogMan.io Elman sends UP command to ASAB Remote Control. Finally, all the content listed in the event lane is enabled in Library.

  2. The event lane template is not found in Library. Then an event lane is created, but the path for parsing rules is not fulfilled as well as other event lane specific content. The model is not created automatically. User action is then required. A user can fulfill suitable parsing rules manually in the event lane declaration. Finally, it is possible to update all models manually from the terminal using curl:

curl --location --request PUT 'localhost:8954/model'

Finally, one or more instances of LogMan.io Parsec are created, and the process of parsing starts.

Kafka topics

LogMan.io Elman updates properties of Kafka received.*, events.*, and others.* topics. In particular, the number of partitions for each topic is increased and the retention policy is set. See Configuration for more details.

Library content

LogMan.io Elman disables every content from all event lane templates in Library. When a new event lane is created, its content is automatically enabled for the event lane tenant.