Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Driver app.forking

Minimal configlet:

[app#1]
type = forking
check = /usr/bin/test -f /tmp/{fqdn}.{rindex}
start = /usr/bin/touch /tmp/{fqdn}.{rindex}
stop = /usr/bin/rm -f /tmp/{fqdn}.{rindex}

Minimal setup command:

om test/svc/foo set \
	--kw="type=forking" \
	--kw="check=/usr/bin/test -f /tmp/{fqdn}.{rindex}" \
	--kw="start=/usr/bin/touch /tmp/{fqdn}.{rindex}" \
	--kw="stop=/usr/bin/rm -f /tmp/{fqdn}.{rindex}"

Supported keywords:

  • access
  • blocking_post_provision
  • blocking_post_start
  • blocking_post_stop
  • blocking_post_unprovision
  • blocking_pre_provision
  • blocking_pre_start
  • blocking_pre_stop
  • blocking_pre_unprovision
  • check
  • check_timeout
  • comment
  • configs_environment
  • cwd
  • devices_from
  • disable
  • encap
  • environment
  • group
  • info
  • info_timeout
  • limit_as
  • limit_core
  • limit_cpu
  • limit_data
  • limit_fsize
  • limit_memlock
  • limit_nofile
  • limit_nproc
  • limit_rss
  • limit_stack
  • limit_vmem
  • monitor
  • netns
  • optional
  • pg_blkio_weight
  • pg_cpu_quota
  • pg_cpu_shares
  • pg_cpus
  • pg_mem_limit
  • pg_mem_oom_control
  • pg_mem_swappiness
  • pg_mems
  • pg_vmem_limit
  • pool
  • post_provision
  • post_start
  • post_stop
  • post_unprovision
  • pre_provision
  • pre_start
  • pre_stop
  • pre_unprovision
  • provision
  • provision_requires
  • provision_timeout
  • restart
  • restart_delay
  • retcodes
  • script
  • secrets_environment
  • shared
  • size
  • standby
  • start
  • start_requires
  • start_timeout
  • stat_timeout
  • status_log
  • status_timeout
  • stop
  • stop_requires
  • stop_timeout
  • subset
  • sync_timeout
  • tags
  • timeout
  • type
  • umask
  • unprovision
  • unprovision_requires
  • unprovision_timeout
  • user

Keyword access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

Description:

The access mode of the volume.

  • rwo is Read Write Once
  • roo is Read Only Once
  • rwx is Read Write Many
  • rox is Read Only Many

rox and rwx modes are served by flex volume services.

Keyword blocking_post_provision

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors interrupt the action.

This trigger is only executed on leaders.

Keyword blocking_post_start

required:    false
scopable:    true

Description:

A command or script to execute after the resource start action.

Errors interrupt the action.

Keyword blocking_post_stop

required:    false
scopable:    true

Description:

A command or script to execute after the resource stop action.

Errors interrupt the action.

Keyword blocking_post_unprovision

required:    false
scopable:    true

Description:

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

This trigger is only executed on leaders.

Keyword blocking_pre_provision

required:    false
scopable:    true

Description:

A command or script to execute before the resource provision action.

Errors interrupt the action.

Keyword blocking_pre_start

required:    false
scopable:    true

Description:

A command or script to execute before the resource start action.

Errors interrupt the action.

Keyword blocking_pre_stop

required:    false
scopable:    true

Description:

A command or script to execute before the resource stop action.

Errors interrupt the action.

Keyword blocking_pre_unprovision

required:    false
scopable:    true

Description:

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

Keyword check

required:    false
scopable:    true

Example:

check=/usr/bin/test -f /tmp/{fqdn}.{rindex}

Description:

  • true

    Execute the script command with status argument on status action.

  • false

    Do nothing on status action.

  • <shlex expression>

    Execute this command on status action.

Keyword check_timeout

required:    false
scopable:    true
convert:     duration

Example:

check_timeout=180

Description:

Wait for <duration> before declaring the app launcher status action a failure.

Takes precedence over timeout.

If neither timeout nor check_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance status when an app launcher did not return.

Keyword comment

required:    false
scopable:    false

Description:

Comments help the users understand the role of the object and its resources.

Keyword configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment=PORT=http/port webapp/app1* {name}/* {name}-debug/settings

Description:

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn’t exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

  • <ns>/cfg/nginx a config having a user key with value user1.

  • <ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the process execution environment:

NGINX_USER=user1
key1=val1

Keyword cwd

required:    false
scopable:    true

Description:

Change the working directory to the specified location instead of the default <pathtmp>.

Keyword devices_from

required:    false
scopable:    false
convert:     list

Description:

The list of resources that contribute their exposed devices to the volume exposed devices.

If not specified, only the last disk contributes.

Keyword disable

required:    false
scopable:    true
convert:     bool

Description:

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

Keyword encap

required:    false
scopable:    false
convert:     bool

Description:

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

Keyword environment

required:    false
scopable:    true
convert:     shlex

Example:

environment=CRT=cert1/server.crt PEM=cert1/server.pem

Description:

A whitespace-separated list of <var>=<value>.

A shell expression splitter is applied, so double quotes can be around <value> only or whole <var>=<value>.

Keyword group

required:    false
scopable:    true

Description:

If the binary is owned by the root user, run it as the specified group instead of root.

Keyword info

required:    false
scopable:    true
default:     false

Description:

  • true

    Execute the script command with info argument on push resinfo action.

  • false

    Do nothing on push resinfo action.

  • <shlex expression>

    Execute this command on push resinfo action.

Stdout lines must contain only one key:value.

Invalid lines are dropped.

Keyword info_timeout

required:    false
scopable:    true
convert:     duration

Example:

info_timeout=180

Description:

Wait for <duration> before declaring the app launcher info action a failure.

Takes precedence over timeout.

If neither timeout nor info_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance info when an app launcher did not return.

Keyword limit_as

required:    false
scopable:    true
convert:     size

Description:

The limit on the total virtual memory that can be in use by a process (unit bytes) (same as limit_vmem).

When both limit_vmem and limit_as is used, the max value is chosen.

Keyword limit_core

required:    false
scopable:    true
convert:     size

Description:

The limit on the largest core dump size that can be produced (unit byte).

Keyword limit_cpu

required:    false
scopable:    true
convert:     duration

Example:

limit_cpu=30s

Description:

The limit on CPU time (duration).

Keyword limit_data

required:    false
scopable:    true
convert:     size

Description:

The limit on the data segment size of a process (unit byte).

Keyword limit_fsize

required:    false
scopable:    true
convert:     size

Description:

The limit on the largest file that can be created (unit byte).

Keyword limit_memlock

required:    false
scopable:    true
convert:     size

Description:

The limit on how much memory a process can lock with mlock(2) (unit byte, no solaris support).

Keyword limit_nofile

required:    false
scopable:    true
convert:     size

Description:

The limit on the number files a process can have open at once.

Keyword limit_nproc

required:    false
scopable:    true
convert:     size

Description:

The limit on the number of processes this user can have at one time, no solaris support.

Keyword limit_rss

required:    false
scopable:    true
convert:     size

Description:

The limit on the total physical memory that can be in use by a process (unit byte, no solaris support).

Keyword limit_stack

required:    false
scopable:    true
convert:     size

Description:

The limit on the stack size of a process (unit bytes).

Keyword limit_vmem

required:    false
scopable:    true
convert:     size

Description:

The limit on the total virtual memory that can be in use by a process (unit bytes).

Keyword monitor

required:    false
scopable:    true
convert:     bool

Description:

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

  • The resource is down.

  • The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.

  • All restart tentatives failed.

Keyword netns

required:    false
scopable:    true

Example:

netns=ip#0

Description:

Start the application process in a netns another resource is known to use.

For example a service with a cni-type ip#0 resource that doesn’t set a netns value will create a private netns named after the object id. A app#0 resource can use this netns by setting app#0.netns=ip#0.

The starter program is executed via ip netns exec.

Keyword optional

required:    false
scopable:    true
convert:     bool

Description:

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

Keyword pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight=50

Description:

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

Keyword pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota=50%@all

Description:

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

Keyword pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares=512

Description:

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

Keyword pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus=0-2

Description:

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

Keyword pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit=512m

Description:

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

Keyword pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control=1

Description:

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

  • If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
  • If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

Keyword pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness=40

Description:

Set a swappiness percentile value for the process group.

Keyword pg_mems

required:    false
scopable:    true

Example:

pg_mems=0-2

Description:

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

Keyword pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit=1g

Description:

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

Keyword pool

required:    false
scopable:    true

Description:

The name of the pool this volume was allocated from.

Keyword post_provision

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword post_start

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword post_stop

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword post_unprovision

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword pre_provision

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword pre_start

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword pre_stop

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword pre_unprovision

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword provision

required:    false
scopable:    false
default:     true
convert:     bool

Description:

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

Keyword provision_requires

required:    false
scopable:    false

Example:

provision_requires=ip#0 fs#0(down,stdby down)

Description:

A whitespace-separated list of conditions to meet to accept a ‘provision’ action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

Keyword provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout=1m30s

Description:

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

Keyword restart

required:    false
scopable:    true
default:     0
convert:     int

Description:

The daemon will try to restart a resource if:

  • The resource is down, stdby down or warn.

  • The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.

  • The node is not frozen

  • The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

Keyword restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes=0:up 1:down 3:warn 4: n/a 5:undef

Description:

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

  • up
  • down
  • warn
  • n/a
  • undef

Keyword script

required:    false
scopable:    true

Description:

Full path to the app launcher script.

This script must accept as argument 0 the action word:

  • start for start
  • stop for stop
  • status for status check
  • info for resource info

Keyword secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment=CRT=cert1/server.pem sec1/*

Description:

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn’t exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

  • <ns>/sec/cert1 a secret having a server.pem key with value mycrt.

  • <ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the process execution environment:

CRT=mycrt
key1=val1

Keyword shared

required:    false
scopable:    true
convert:     bool

Description:

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

  • When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.

  • When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don’t use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

Keyword size

required:    false
scopable:    true
convert:     size

Description:

The size used by this volume in its pool.

Keyword standby

required:    false
scopable:    true
convert:     bool

Description:

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don’t set shared resources standby, a non-clustered fs on shared disks for example.

Keyword start

required:    false
scopable:    true

Example:

start=/usr/bin/touch /tmp/{fqdn}.{rindex}

Description:

  • true

    Execute the script command with start argument on start action.

  • false

    Do nothing on start action.

  • <shlex expression>

    Execute this command on start action.

Keyword start_requires

required:    false
scopable:    false

Example:

start_requires=ip#0 fs#0(down,stdby down)

Description:

A whitespace-separated list of conditions to meet to accept a ‘start’ action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

Keyword start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout=180

Description:

Wait for <duration> before declaring the app launcher start action a failure.

Takes precedence over timeout.

If neither timeout nor start_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance start when an app launcher did not return.

Keyword stat_timeout

required:    false
scopable:    true
convert:     duration

Description:

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

Keyword status_log

required:    false
scopable:    true
default:     false
convert:     bool

Description:

If true, redirect the checker script:

  • stdout to the resource status info-log.

  • stderr to the resource status warn-log.

Keyword status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout=10s

Description:

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

  • the start_timeout Limiting the start action duration.

  • the stop_timeout Limiting the start rollback duration triggered by start errors.

  • the status_timeout Limiting the post-start instance status evaluation duration.

Keyword stop

required:    false
scopable:    true

Example:

stop=/usr/bin/rm -f /tmp/{fqdn}.{rindex}

Description:

  • true

    Execute the script command with stop argument on stop action.

  • false

    Do nothing on stop action.

  • <shlex expression>

    Execute this command on stop action.

Keyword stop_requires

required:    false
scopable:    false

Example:

stop_requires=ip#0 fs#0(down,stdby down)

Description:

A whitespace-separated list of conditions to meet to accept a ‘stop’ action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

Keyword stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout=180

Description:

Wait for <duration> before declaring the app launcher stop action a failure.

Takes precedence over timeout.

If neither timeout nor stop_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance stop when an app launcher did not return.

Keyword subset

required:    false
scopable:    true

Description:

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

This trigger is only executed on leaders.

Keyword sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout=1m30s

Description:

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

Keyword tags

required:    false
scopable:    true
convert:     set

Description:

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

  • noaction

    Skip any state changing action on the resource and imply optional=true.

  • nostatus

    Force the status n/a.

Keyword timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout=180

Description:

Wait for <duration> before declaring the app launcher action a failure.

Can be overridden by <action>_timeout.

If no timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance action when an app launcher did not return.

Keyword type

required:    false
scopable:    false

Description:

The resource driver name.

Keyword umask

required:    false
scopable:    true
convert:     umask

Example:

umask=022

Description:

The umask to set for the application process.

Keyword unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Description:

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

Keyword unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires=ip#0 fs#0(down,stdby down)

Description:

A whitespace-separated list of conditions to meet to accept a ‘unprovision’ action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

Keyword unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout=1m30s

Description:

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

Keyword user

required:    false
scopable:    true

Description:

If the binary is owned by the root user, run it as the specified user instead of root.