Collecting lookups

Files

To periodically collect lookups from files such as CSV, use the input:FileBlock: input with following configuration:

path:  # Specify the lookup folder, where the file lookup will be stored (f. e. /data/lookups/mylookup/*)
chilldown_period:  # Specify how often in seconds to check for new files (default: 5)

FileBlock reads all files in one block (one event is the entire file content) and passes it to configured output, which is usually output:WebSocket.

In such a way, the lookup is passed to LogMan.io Ingestor, and, eventually, to LogMan.io Parser, where the lookup can be processed and stored to ElasticSearch. Please see the “Parsing lookups” section in LogMan.io Parser documentation.

Sample configuration

input:FileBlock:MyLookupFileInput:
  path: /data/lookups/mylookup/*
  chilldown_period: 10
  output: LookupOutput

output:WebSocket:LookupOutput:
  url: https://lm1/files-ingestor-ws
  ...