Collecting lookups


To periodically collect lookups from files such as CSV, use the input:FileBlock: input with following configuration:

path:  # Specify the lookup folder, where the file lookup will be stored (f. e. /data/lookups/mylookup/*)
chilldown_period:  # Specify how often in seconds to check for new files (default: 5)

FileBlock reads all files in one block (one event is the entire file content) and passes it to configured output, which is usually output:WebSocket.

In such a way, the lookup is passed to Ingestor, and, eventually, to Parser, where the lookup can be processed and stored to ElasticSearch. Please see the “Parsing lookups” section in Parser documentation.

Sample configuration

  path: /data/lookups/mylookup/*
  chilldown_period: 10
  output: LookupOutput

  url: https://lm1/files-ingestor-ws