Skip to content

ASAB Maestro descriptor

Descriptors are YAML files living in the Library. Each application consists of a group of descriptors /Site/<application name>/Descriptors/.

A descriptor provides detailed infomation about the service and/or technology. Descriptors serve as specific extensions of the model.

Note

Descriptors are provided by the authors of each application.

Structure of the descriptor

Example of /Site/ASAB Maestro/Descriptors/mongo.yaml:

define:
  type: rc/descriptor
  name: MongoDB document database
  url: https://github.com/mongodb/mongo

descriptor:
  image: library/mongo

  volumes:
    - "{{SLOW_STORAGE}}/{{INSTANCE_ID}}/data:/data/db"
    - "{{SITE}}/{{INSTANCE_ID}}/conf:/etc/mongo:ro"

  command: mongod --config /etc/mongo/mongod.conf --directoryperdb

  healthcheck:
    test: ["CMD-SHELL", 'echo "db.runCommand(\"ping\").ok" | mongosh 127.0.0.1:27017/rs0 --quiet']
    interval: 60s
    timeout: 10s
    retries: 5
    start_period: 30s

sherpas:
  init:
    image: library/mongo
    entrypoint: ["mongosh", "--nodb", "--file", "/script/mongo-init.js"]
    command: ["echo", "DONE"]
    volumes:
    - "{{SITE}}/{{INSTANCE_ID}}/script:/script:ro"
    - "{{SITE}}/{{INSTANCE_ID}}/conf:/etc/mongo:ro"
    depends_on: ["{{INSTANCE_ID}}"]
    environment:
      MONGO_HOSTNAMES: "{{MONGO_HOSTNAMES}}"

files:
  - "conf/mongod.conf": |
      net:
        bindIp: 0.0.0.0
        port: 27017
      replication:
        replSetName: rs0
  - "script/mongo-init.js"

Templating

Descriptors utilize Jinja2 templates that are expanded when a descriptor is applied.

Common parameters:

  • {{NODE_ID}}: Node identification / hostname of the host machine (node1).
  • {{SERVICE_ID}}: Service identification (i.e. mongo).
  • {{INSTANCE_ID}}: Instance identification (i.e. mongo-2).
  • {{INSTANCE_NO}}: Instance number (i.e, 2).
  • {{SITE}}: Directory with the site config on the host machine (i.e. /opt/site).
  • {{FAST_STORAGE}}: Directory with the fast storage on the host machine (i.e. /data/ssd).
  • {{SLOW_STORAGE}}: Directory with the slow storage on the host machine (i.e. /data/hdd).

Note

Other parameters can be specified within descriptors, in the model or provided by technologies.

Technologies

Not only that multiple Library files are composed into the final configuration. There are also techs playing their parts. Techs are part of ASAB Remote Control microservice and provide extra cluster configuration.

Some of them also introduce specific sections to the descriptors.

Learn more about Techs

Composability

Descriptors can be overridden in the deployment via specific configuration options or through model.

  • /Site/<application name>/Descriptors/__commons.yaml__ file of the Library is a common base for all descriptors of the application. It specifically contains entries for network mode, restart policy, logging and others.
  • The specific descriptor of the service (e.g. /Site/<application name>/Descriptors/nginx.yaml) is layered on the top of the content of the __commons__.yaml
  • Model can override the descriptor.

Merge algorithm

This composability is implemented through a merge algorithm. You'll find the same algorithm being used in multiple cases where chunks from various sources result into a functional site configuration.

Library layering

To get a full picture of the Library within ASAB Maestro, learn also about ASAB Library layering.

Sections

Section define

define:
  type: rc/descriptor
  name: <human-readable name>
  url: <URL with relevant info>

type must be rc/descriptor.

Items name and url provide information about the service and/or technology.

Section params

Specify parameters for templating of this and all other descriptors. Any parameter specified in this section can be used in the double courly brackets for Jinja2 templating.

define:
  type: rc/descriptor

params:
  MY_PARAMETER: "ABCDEFGH"

descriptor:
  environment: "{{MY_PARAMETER}}"

Section secrets

Similar to params, secrets can be also used as parameters for templating. However, their value is not specified in the descriptor but generated and stored in the Vault. You can customize the secret by specifying type and lenght. Default is "token" of 64 bytes.

define:
  type: rc/descriptor

secrets:
  MY_SECRET:
    type: token
    length: 32

descriptor:
  environment: "{{MY_SECRET}}"

Warning

Parts of the descriptor are used directly to prepare docker-compose.yaml. The section secrets can be specified in the docker-compose.yaml as well. However, this functionality of Docker Compose is omitted within ASAB Maestro and fully replaced by secrets section of the descriptor.

Section descriptor

The descriptor section is a template for a service section of the docker-compose.yaml.

Following transformations are done:

  • The Jinja2 variables are expanded.
  • The version from ../Versions/... is appended to image, if not present.
  • Specific techs performs custom transformations, these are typically marked by null.

Details about volumes

The service has following three storage location available for their persistent data:

  • {{SITE}}: the site directory (i.e. /opt/site/...)
  • {{SLOW_STORAGE}}: the slow storage (i.e. /data/hdd/...)
  • {{FAST_STORAGE}}: the fast storage (i.e. /data/ssd/...)

Each instance can create a sub-directory in any of above locations named by its instance_id.

Section files

This section specifies what files to be copied into the instance sub-directory of the site directory (i.e. /opt/site/...). Subsequently, this content can be made available to the instance container by relevant volumes entry.

First variant is to specify the file or directory to be copied from the ../Files directory.

This entry copies the ../Files/<service_name>/script/mongo-init.js into /opt/site/mongo-1/script/mongo-init.js, assuming the instance identification is mongo-1:

files:
  - "script/mongo-init.js"

The similar entry with trailing / copies the whole directory into /opt/site/mongo-1/conf/ directory:

files:
  - "conf/"

The content of the file can be specified in the descriptor file itself, including use of the Jinja2 parameters:

files:
  - "conf/mongod.conf": |
      net:
        bindIp: 0.0.0.0
        port: 27017
      replication:
        replSetName: rs0

How to enter files directly into the descriptor

Use pipe (|) operator that denotes multi-line string in the YAML

Files are not templated

Unlike descriptors and model, files stored in ../Files/<service_id>/ directory are not templated. That means that curly brackets with parameters are not replaced by the respective values. If you need to use templating within the file, enter the file into the decriptor directly, using the multiline string operator ("|").

Section sherpas

The sherpas are auxiliary containers that are launched together with the main instance containers. Sherpa containers are expected to finish relativelly quickly and they are not restarted. Sherpas exited sucessfuly (with exit code being 0) are promptly deleted.

Example:

sherpas:
  init:
    image: library/mongo
    entrypoint: ["mongosh", "--nodb", "--file", "/script/mongo-init.js"]
    command: ["echo", "DONE"]
    volumes:
    - "{{SITE}}/{{INSTANCE_ID}}/script:/script:ro"
    - "{{SITE}}/{{INSTANCE_ID}}/conf:/etc/mongo:ro"
    depends_on: ["{{INSTANCE_ID}}"]
    environment:
      MONGO_HOSTNAMES: "{{MONGO_HOSTNAMES}}"

This defines an init sherpa container. The container name of the sherpa would be mongo-1-init, the INSTANCE_ID parameter remains mongo-1.

The content of the sherpa is a template for a relevant part of the docker-compose.yaml.

If sherpa does not specify the image, the service image including the version is used. Alternatively, we recommend to use docker.teskalabs.com/asab/asab-governator:stable as an image for a sherpa, since this image is always present and doesn't need to be downloaded.