Skip to content

Collecting into lookups

Files

To periodically collect lookups from files such as CSV, use the input:FileBlock: input with following configuration:

path:  # Specify the lookup folder, where the file lookup will be stored (f. e. /data/lookups/mylookup/*)
chilldown_period:  # Specify how often in seconds to check for new files (default: 5)

FileBlock reads all files in one block (one event is the entire file content) and passes it to configured output, which is usually output:WebSocket.

In such a way, the lookup is passed to LogMan.io Receiver, and, eventually, to LogMan.io Parser, where the lookup can be processed and stored in Elasticsearch. See Parsing lookups for more information.

Sample configuration

input:FileBlock:MyLookupFileInput:
  path: /data/lookups/mylookup/*
  chilldown_period: 10
  output: LookupOutput

output:WebSocket:LookupOutput:
  url: https://lm1/files-ingestor-ws
  ...