Skip to content

Discover configuration

LogMan.io Discover overview

Discover screen setup

The Discover screen is used for displaying and exploring the data (not only) in ElasticSearch.

The configuration of Discover screen can be loaded from Library or from static file in public folder the same way, as it is in case of Dashboards.

The type of filtered data relies on the specification which must be defined together with datetimeField. Those are crucial values without which the filtering is not possible.

Discover configuration

Library configuration

Library configuration is stored in the Library node. It must be of JSON type.

To get the configuration from Library, asab_config service must be running with the configuration pointing to the main node of Library. For more info, please refer here: http://gitlab.teskalabs.int/lmio/asab-config

The configuration from Library is editable

In the Discover Library node, there can be multiple configuration files within each of it only one discover configuration screen can be set. Another discover screen has to be configured in new Library configuration node.

All configuration files from Discover Library node are loaded in one API call.

Library configuration structure

Config structure in Library

- main Library node
    - config
        - Discover
            - **config**.json
    - type
        - Discover.json
            - schema
Where

  • config is the name of the particular Discover configuration, it must be of json type.

In Library, the path to the config file looks like:

/<main Library node>/config/Discover/<discoverConfig>.json

Schema path will be as following:

/<main Library node>/type/Discover.json

Example of above described Library structure for multiple Discover config file case:

- logman
    - config
        - Discover
            - declarative.json
            - default.json
            - speed.json
    - type
        - Discover.json

IMPORTANT NOTE

Schema (type) and config file (config) must be set in Library, otherwise the discover will not be loaded correctly.

All configuration files from Discover Library node are loaded in one API call.

Example of the configuration:
{
    "Discover:datasource": {
        "specification": "declarative*",
        "datetimeField": "@timestamp",
        "type": "elasticsearch"
    }
}

Where

  • object key serves the purpose of the naming the object. It must be named as Discover:datasource.
  • type is the type of the searching engine
  • specification is the url with the ElasticSearch index pattern. If one would like to seek for all the data, the url must end with an asterisk *. This is a mandatory parameter.
  • datetimeField is the index of the item's datetime. It is a mandatory parameter since it is needed for searching/scrolling with ElasticSearch.

Schema (optional setup)

Don't confuse with Library schema

Set up the name to obtain schema from the library (if present), which is then applied to values defined within the schema. With the schema, we can apply actions to values corresponding the defined type, e.g. using the ASAB-WebUI's DateTime component for time values.

{
    ...

    "Discover:schema": {
        "name": "ECS"
    }

    ...
}

Example of schemas structure in the library:

- library
    - Schemas
        - Discover.yaml
        - ECS.yaml
        ...

Example of the schema in the library:

---

define:
  name: Elastic Common Schema
  type: common/schema
  description: https://www.elastic.co/guide/en/ecs/current/index.html

fields:
  '@timestamp':
    type: datetime
    label: "Datetime"
    unit: seconds
    docs: https://www.elastic.co/guide/en/ecs/current/ecs-base.html#field-timestamp

Authorization (optional setup)

Discover configuration can be limited for access only with specific tenant(s). This means, that users without particular tenant(s) are not able to access the discover configuration with its data source. This is convenient e.g. when administrator wants to limit access to discover configuration with sensitive data to particular group(s) of users.

If configuration is being set directly in the Library (and not via Configuration tool), its recommended to add Authorization section and leave tenants key as an empty string (if no limitation is required). This will help to keep up the same structure of configuration across the Discover configurations:

{
    ...

    "Authorization": {
        "tenants": ""
    }

    ...
}

Example of Authorization settings within the configuration where limited access is required:

{
    ...

    "Authorization": {
        "tenants": "tenant one, tenant two"
    }

    ...
}

Where key tenants serves the purpose of displaying and using the configuration only by specific tenant(s). Multiple tenants can be specified, separated by comma. The type of the tenants key is string.

Prompt settings (optional setup)

Prompt settings section provides additional option to setup Discover prompt or change its defaults.

Example of Discover:prompts section within the configuration:

{
    ...

    "Discover:prompts": {
        "dateRangePicker:datetimeStart": "now-15m",
        "dateRangePicker:datetimeEnd": "now+15s"
        ...
    },

    ...
}
Setup custom datetime range periods

Sometimes its desired to setup custom datetime period for data display, because data are laying e.g. outside of default period set for Discover. The default period is now-1H, which should seek for data within now and 1 hour back. For example, this could be set in the Discover:prompts as following:

{
    ...

    "Discover:prompts": {
        "dateRangePicker:datetimeStart": "now-1H",
        "dateRangePicker:datetimeEnd": "now"
    },

    ...
}

Where the dateRangePicker:datetimeStart and dateRangePicker:datetimeEnd are the periods which sets up the range to the starting period (initial) and to the ending period (final).

The setup possibilities for both periods are:

  • now-ns
  • now-nm
  • now-nH
  • now-nd
  • now-nw
  • now-nM
  • now-nY
  • now
  • now+ns
  • now+nm
  • now+nH
  • now+nd
  • now+nw
  • now+nM
  • now+nY

Where - n is the number, e.g. 2, - s indicate seconds, - m indicate minutes, - H indicate hours, - d indicate days, - w indicate weeks, - M indicate months, - Y indicate years

Other values will be ignored.

It is possible to e.g. setup only one period as in this example, the second period will remain default:

{
    ...

    "Discover:prompts": {
        "dateRangePicker:datetimeStart": "now-2H"
    },

    ...
}

Another datetime range setup example, where data are displayed 15 hours to the past and sought 10 minutes into the future:

{
    ...

    "Discover:prompts": {
        "dateRangePicker:datetimeStart": "now-15H",
        "dateRangePicker:datetimeEnd": "now+10m"
    },

    ...
}

Library schema

To set up a discover screen manually in the Library, one must set the discover schema in valid JSON format.

Schema must be provided and stored in /<main Library node>/type/<discoverType>.json

The schema can look as following:

{
    "$id": "Discover schema",
    "type": "object",
    "title": "Discover schema",
    "description": "The Discover schema",
    "default": {},
    "examples": [
        {
            "Discover:datasource": {
                "specification": "declarative*",
                "datetimeField": "@timestamp",
                "type": "elasticsearch"
            }
        }
    ],
    "required": [],
    "properties": {
        "Discover:datasource": {
            "type": "string",
            "title": "Discover source",
            "description": "The data specification for Discover screen",
            "default": {},
            "examples": [
                {
                    "specification": "declarative*",
                    "datetimeField": "@timestamp",
                    "type": "elasticsearch"
                }
            ],
            "required": [
                "specification",
                "datetimeField",
                "type"
            ],
            "properties": {
                "specification": {
                    "type": "string",
                    "title": "Specification",
                    "description": "Specify the source of the data",
                    "default": "",
                    "examples": [
                        "declarative*"
                    ]
                },
                "datetimeField": {
                    "type": "string",
                    "title": "Datetime",
                    "description": "Specify the datetime value for data source",
                    "default": "",
                    "examples": [
                        "@timestamp"
                    ]
                },
                "type": {
                    "type": "string",
                    "title": "Type",
                    "description": "Select the type of the source",
                    "default": [
                        "elasticsearch",
                        "sentinel"
                    ],
                    "$defs": {
                        "select": {
                            "type": "select"
                        }
                    },
                    "examples": [
                        "elasticsearch*"
                    ]
                }
            }
        },
        "Discover:prompts": {
            "type": "string",
            "title": "Discover prompts",
            "description": "Update Discover prompt configuration",
            "default": {},
            "examples": [],
            "required": [],
            "properties": {
                "dateRangePicker:datetimeStart": {
                    "type": "string",
                    "title": "Starting date time period",
                    "description": "Setup the prompt's starting date time period",
                    "default": "now-1H",
                    "examples": [
                        "now-1H"
                    ]
                },
                "dateRangePicker:datetimeEnd": {
                    "type": "string",
                    "title": "Ending date time period",
                    "description": "Setup the prompt's ending date time period",
                    "default": "now",
                    "examples": [
                        "now"
                    ]
                }
            }
        },
        "Discover:schema": {
            "type": "string",
            "title": "Discover schema name",
            "description": "Apply schema over discover values",
            "default": {},
            "properties": {
                "name": {
                    "type": "string",
                    "title": "Schema name",
                    "description": "Set up the schema name for configuration (without file extension)",
                    "default": ""
                }
            }
        },
        "Authorization": {
            "type": "string",
            "title": "Discover authorization",
            "description": "Limit access to discover configuration by tenant settings",
            "default": {},
            "examples": [],
            "required": [],
            "properties": {
                "tenants": {
                    "type": "string",
                    "title": "Tenants",
                    "description": "Specify the tenant(s) separated by comma to restrict the usage of this configuration (optional)",
                    "default": "",
                    "examples": [
                        "tenant1, tenant2"
                    ]
                }
            }
        }
    },
    "additionalProperties": false
}

Example of passing config props

Example of passing config props to the DiscoverContainer:

...

this.App.Router.addRoute({
    path: "/discover",
    exact: true,
    name: 'Discover',
    component: DiscoverContainer,
    props: {
        type: "Discover"
    }
});

...

this.App.Navigation.addItem({
        name: "Discover",
        url: "/discover",
        icon: 'cil-compass'
    });

When using DiscoverContainer as a component in your container, then the props can be passed as following:

<DiscoverContainer type="Discover" />

The static application config file remains empty:

module.exports = {
    app: {
    },
    webpackDevServer: {
        port: 3000,
        proxy: {
            '/api/elasticsearch': {
                target: "http://es-url:9200",
                pathRewrite: {'^/api/elasticsearch': ''}
            },
            '/api/asab_print': {
                target: "http://asab_print-url:8083",
                pathRewrite: {'^/api/asab_print': ''}
            },
            '/api/asab_config': {
                target: "http://asab_config-url:8082",
                pathRewrite: {'^/api/asab_config': ''}
            }
        }
    }
}

Static configuration

Discover screen does not have to be obtained only from Library. Another option is to configure it directly in the JSON file and save it in projects public folder.

Example of static configuration

In index.js, developer have to specify:

The JSON file with the configuration can be stored anywhere in the public folder, but it is strongly recommended to store it in /public/discover/ folder to distinguish it from the other publicly accessible components.

  • Config structure in public folder
- public
    - discover
        - JSON config file
    - dashboards
    - locales
    - media
    - index.html
    - manifest.json

The URL of the static config sotred in public folder can look like:

https://my-project-url/discover/Discover-config.json

Example of Discover-config.json:

[
    {
        "Config name 1": {
            "Declarative": {
                "specification": "declarative*",
                "datetimeField": "last_inform",
                "type": "elasticsearch"
            }
        }
    },
    {
        "Config name 2": {
            "Default": {
                "specification": "default*",
                "datetimeField": "@timestamp",
                "type": "elasticsearch"
            }
        }
    }
]

Example of passing config props

Passing config props to the App:

this.App.Router.addRoute({
    path: "/discover",
    exact: true,
    name: 'Discover',
    component: DiscoverContainer,
    props: {
        type: "https://my-project-url/discover/Discover-config.json"
    }
});

this.App.Navigation.addItem({
    name: "Discover",
    url: "/discover",
    icon: 'cil-compass'
});

When using DiscoverContainer as a component in your container, then the props can be passed as following:

<DiscoverContainer type="https://my-project-url/discover/Discover-config.json" />

The static application config file remains empty:

module.exports = {
    app: {
    },
    webpackDevServer: {
        port: 3000,
        proxy: {
            '/api/elasticsearch': {
                target: "http://es-url:9200",
                pathRewrite: {'^/api/elasticsearch': ''}
            },
            '/api/asab_print': {
                target: "http://asab_print-url:8083",
                pathRewrite: {'^/api/asab_print': ''}
            },
            '/api/asab_config': {
                target: "http://asab_config-url:8082",
                pathRewrite: {'^/api/asab_config': ''}
            }
        }
    }
}