To periodically collect lookups from files such as CSV,
input:FileBlock: input with following configuration:
path: # Specify the lookup folder, where the file lookup will be stored (f. e. /data/lookups/mylookup/*) chilldown_period: # Specify how often in seconds to check for new files (default: 5)
FileBlock reads all files in one block (one event is the entire file content) and passes it to configured output,
which is usually
In such a way, the lookup is passed to LogMan.io Ingestor, and, eventually, to LogMan.io Parser, where the lookup can be processed and stored to ElasticSearch. Please see the “Parsing lookups” section in LogMan.io Parser documentation.
input:FileBlock:MyLookupFileInput: path: /data/lookups/mylookup/* chilldown_period: 10 output: LookupOutput output:WebSocket:LookupOutput: url: https://lm1/files-ingestor-ws ...