Skip to content

Forwarding to Kafka

If the Apache Kafka is configured in the configuration, every received log event is forwarded by LogMan.io Receiver to Kafka topic. The topic is created automatically, when the first message is forwarded.

The name of the Apache Kafka topic is derived from the stream name: received.<tenant>.<stream>, it is the same as the name in the archive.

Example of Kafka topics

received.mytenant.udp-8889
received.mytenant.tcp-7781

The raw log event is sent in the Kafka message body.

Following information is added to a message header:

  • tenant: The name of the tenant which received this message
  • row_id: Row Id, the globally unique identifier of the event in the archive. Not present if the archive is disabled.
  • collected_at: The event collection date&time, Unix timestamp in microseconds
  • received_at: The event receival date&time, Unix timestamp in microseconds
  • source: The event source

Roadmap

Automated setup of the Kafka topic (such as number of Kafka partitions) will be implemented in the future releases.