Introduction

What is OpenSVC

OpenSVC is an open-source software product and the name of the company developing and supporting it. In production since 2009, OpenSVC consists of:

Agent: A supervisor, clusterware, container orchestrator, and configuration manager (Apache 2.0 licensed since v3).
Collector: Optionally, aggregates agent data and manages configuration, infrastructure, and resources (Apache 2.0 licensed).

Why OpenSVC

Service Mobility: Ensures high availability and scalability.
Configuration Management: Handles thousands of servers and clusters with minimal dependencies.
Inventoring: Tracks assets, storage, networks, and services.
Orchestration: RESTful API, self-service portal, provisioning, fine-grained delegation.
Auditing: Monitors performance, alerts, and audits infrastructure and services.
Productivity Boost: Reduces infrastructure maintenance and allows for task delegation.

Requirements

OpenSVC runs on Linux servers.

Subsystem	Agent	Collector
Service mobility, clustering, container orchestration	O
Configuration management	O	O
Inventoring	O	O
Orchestration and interoperability	O	O
Continuous infrastructure, systems, and services auditing	O	O

Contribute

OpenSVC projects, including this documentation, are hosted on Github.

Install Git

sudo apt-get install git
git config --global user.name "First Last Name"
git config --global user.email "first.lastname@domain.com"

Clone the agent and book projects

cd /tmp
git clone https://github.com/opensvc/book.opensvc.com.git
git clone https://github.com/opensvc/opensvc.git

Build the agent

(cd opensvc && make om)

Install mdbook

From https://github.com/rust-lang/mdBook/releases

Make autogenerated documentation

(cd book.opensvc.com && OM=../opensvc/bin/om make)

Contribute Documentations

(cd book.opensvc.com && && mdbook serve --open)

Modify the documentation source files (.md located in the src directory and referenced in src/SUMMARY.md)

Command Line Output Coloring

When producing command line output, it is expected to follow the steps below to preserve colors, so as to provide a better experience for futures readers:

Install ansi2html

From https://github.com/ralphbean/ansi2html

generate raw html code from cli:

$ om node print devs --color=yes | aha -n

Edit the .md document and copy/paste the previous output enclosed between <pre> and </pre>:

<pre>
        <span style="font-weight: bold">centos71.opensvc.com                        </span>  <span style="font-weight: bold">Type  </span>  <span style="font-weight: bold">Size</span>  <span style="font-weight: bold">Pct of Parent</span>  
        `- <span style="color: #aa5500">vda                                      </span>  linear  15g   -              
           |- <span style="color: #aa5500">vda1                                  </span>  linear  500m  3%             
           `- <span style="color: #aa5500">vda2                                  </span>  linear  14g   96%            
              |- <span style="color: #aa5500">centos_centos71-swap               </span>  linear  1g    10%            
              `- <span style="color: #aa5500">centos_centos71-root               </span>  linear  13g   89%            
                 |- <span style="color: #aa5500">loop2                           </span>  linear  50m   0%             
                 |  |- <span style="color: #aa5500">testsvc1-lv1                 </span>  linear  20m   40%            
                 |  `- <span style="color: #aa5500">testsvc1-lv2                 </span>  linear  20m   40%            
                 |- <span style="color: #aa5500">loop1                           </span>  linear  100m  0%             
                 |  |- <span style="color: #aa5500">testsvc3-lv2                 </span>  linear  20m   20%            
                 |  |- <span style="color: #aa5500">testsvc3-lv1-real            </span>  linear  52m   52%            
                 |  |  |- <span style="color: #aa5500">testsvc3-lv1              </span>  linear  52m   100%           
                 |  |  `- <span style="color: #aa5500">testsvc3-osvc_sync_lv1    </span>  linear  52m   100%           
                 |  `- <span style="color: #aa5500">testsvc3-osvc_sync_lv1-cow   </span>  linear  8m    8%             
                 |     `- <span style="color: #aa5500">testsvc3-osvc_sync_lv1    </span>  linear  52m   650%           
                 `- <span style="color: #aa5500">loop0                           </span>  linear  100m  0%             
                    |- <span style="color: #aa5500">testsvc2-lv1                 </span>  linear  52m   52%            
                    `- <span style="color: #aa5500">testsvc2-lv2                 </span>  linear  20m   20%            
    </pre>

The result looks like:

	centos71.opensvc.com                          Type    Size  Pct of Parent  
	`- vda                                        linear  15g   -              
	   |- vda1                                    linear  500m  3%             
	   `- vda2                                    linear  14g   96%            
	      |- centos_centos71-swap                 linear  1g    10%            
	      `- centos_centos71-root                 linear  13g   89%            
	         |- loop2                             linear  50m   0%             
	         |  |- testsvc1-lv1                   linear  20m   40%            
	         |  `- testsvc1-lv2                   linear  20m   40%            
	         |- loop1                             linear  100m  0%             
	         |  |- testsvc3-lv2                   linear  20m   20%            
	         |  |- testsvc3-lv1-real              linear  52m   52%            
	         |  |  |- testsvc3-lv1                linear  52m   100%           
	         |  |  `- testsvc3-osvc_sync_lv1      linear  52m   100%           
	         |  `- testsvc3-osvc_sync_lv1-cow     linear  8m    8%             
	         |     `- testsvc3-osvc_sync_lv1      linear  52m   650%           
	         `- loop0                             linear  100m  0%             
	            |- testsvc2-lv1                   linear  52m   52%            
	            `- testsvc2-lv2                   linear  20m   20%

Decorating words

Awesome font icons can be inlined in the documentation. To make it easier and to enforce a common set of icon and color, include fragments are available in the src/inc/ directory.

Syntax	Rendering
{{#include ../inc/action}} action	action
{{#include ../inc/repo}} repo	repo
{{#include ../inc/registry}} registry	registry
{{#include ../inc/node}} node	node
{{#include ../inc/svc}} svc	svc
{{#include ../inc/res}} res	res
{{#include ../inc/tag}} tag	tag
{{#include ../inc/svcenv}} svcenv	svcenv
{{#include ../inc/env}} env	env
{{#include ../inc/pkg}} pkg	pkg
{{#include ../inc/net}} net	net
{{#include ../inc/check}} check	check
{{#include ../inc/form}} form	form
{{#include ../inc/report}} report	report
{{#include ../inc/metric}} metric	metric
{{#include ../inc/chart}} chart	chart
{{#include ../inc/group}} group	group
{{#include ../inc/priv}} priv	priv
{{#include ../inc/user}} user	user
{{#include ../inc/dns-domain}} example.com	example.com
{{#include ../inc/dns-record}} example.com. IN A 12.13.14.15	example.com. IN A 12.13.14.15
{{#include ../inc/fset}} filterset	filterset
{{#include ../inc/disk}} disk	disk
{{#include ../inc/array}} array	array
{{#include ../inc/diskgroup}} diskgroup	diskgroup
{{#include ../inc/rule}} rule	rule
{{#include ../inc/ruleset}} ruleset	ruleset
{{#include ../inc/modset}} moduleset	moduleset
{{#include ../inc/mod}} module	module
{{#include ../inc/app}} app	application
{{#include ../inc/close}} close	close
{{#include ../inc/fullscreen}} fullscreen	fullscreen
{{#include ../inc/shrink}} shrink	shrink
{{#include ../inc/link}} link	link
{{#include ../inc/kw}} keyword	keyword
{{#include ../inc/cmd}} command	command

Install

We feed packages in 3 different branches. Subscribe your servers to the appropriate repository branch:

dev: Unstable. Every candidate Pull Request causes a new package to be spawned here for OpenSVC QA purpose.
uat: Testing. OpenSVC will push there pre-release packages and packages that contain a candidate fixes for known issues that client are encouraged to validate.
prod: Stable. The recommended branch.

Debian

# Select a os version and opensvc branch
# --------------------------------------
DISTRIB=bookworm
DISTRIB=bullseye
DISTRIB=buster
BRANCH=dev
BRANCH=uat
BRANCH=prod

# Import opensvc gpg signing keys
# -------------------------------
curl -s -o- https://packages.opensvc.com/gpg.public.key.asc | \
    sudo gpg --dearmor --output /etc/apt/trusted.gpg.d/opensvc-package-pub.gpg --yes

#
# Add the opensvc repository to apt sources
# -----------------------------------------
cat - <<EOF | sudo tee /etc/apt/sources.list.d/opensvc.list 
deb https://packages.opensvc.com/apt/debian $BRANCH-opensvc-v3-$DISTRIB main
deb-src https://packages.opensvc.com/apt/debian $BRANCH-opensvc-v3-$DISTRIB main
EOF

#
# Install the opensvc server
# --------------------------
sudo apt update
sudo apt install opensvc-server

#
# Enable the systemd unit and start the server
# --------------------------------------------
sudo systemctl enable --now opensvc-server

Ubuntu

# Select a os version and opensvc branch
# --------------------------------------
DISTRIB=focal
DISTRIB=jammy
DISTRIB=noble
BRANCH=dev
BRANCH=uat
BRANCH=prod

#
# Import opensvc gpg signing keys
# -------------------------------
curl -s -o- https://packages.opensvc.com/gpg.public.key.asc | \
    sudo gpg --dearmor --output /etc/apt/trusted.gpg.d/opensvc-package-pub.gpg --yes

#
# Add the opensvc repository to apt sources
# -----------------------------------------
cat - <<EOF | sudo tee /etc/apt/sources.list.d/opensvc.list 
deb https://packages.opensvc.com/apt/ubuntu $BRANCH-opensvc-v3-$DISTRIB main
deb-src https://packages.opensvc.com/apt/ubuntu $BRANCH-opensvc-v3-$DISTRIB main
EOF

#
# Install the opensvc server
# --------------------------
sudo apt update
sudo apt install opensvc-server

#
# Enable the systemd unit and start the server
# --------------------------------------------
sudo systemctl enable --now opensvc-server

Red Hat Enterprise Linux 7

# Select a os version and opensvc branch
# --------------------------------------
DISTRIB=rhel7
BRANCH=dev
BRANCH=uat
BRANCH=prod

#
# Add the opensvc repository to apt sources
# -----------------------------------------
cat << EOF >/etc/yum.repos.d/opensvc.repo
[opensvc]
name=OpenSVC Packages RHEL \$releasever - \$basearch
baseurl=https://packages.opensvc.com/rpm/$BRANCH-opensvc-v3-$DISTRIB/\$basearch/
enabled=1
gpgcheck=0
EOF

#
# Install the opensvc server
# --------------------------
sudo yum update
sudo yum install opensvc-server

#
# Enable the systemd unit and start the server
# --------------------------------------------
sudo systemctl enable --now opensvc-server

Red Hat Enterprise Linux 8+

# Select a os version and opensvc branch
# --------------------------------------
DISTRIB=rhel8
DISTRIB=rhel9
BRANCH=dev
BRANCH=uat
BRANCH=prod

#
# Add the opensvc repository to apt sources
# -----------------------------------------
cat << EOF >/etc/yum.repos.d/opensvc.repo
[opensvc]
name=OpenSVC Packages RHEL \$releasever - \$basearch
baseurl=https://packages.opensvc.com/rpm/$BRANCH-opensvc-v3-$DISTRIB/\$basearch/
enabled=1
gpgcheck=1
gpgkey=https://packages.opensvc.com/gpg.public.key.asc
EOF

#
# Install the opensvc server
# --------------------------
sudo dnf update
sudo dnf install opensvc-server

#
# Enable the systemd unit and start the server
# --------------------------------------------
sudo systemctl enable --now opensvc-server

Agent Configuration

Concepts

The agent uses ini configuration files.

Considering a configuration like:

[env]
bar = 1
bar@n2 = 2

env is a section
bar is a option
env.bar is a keyword.
env.bar=1 is a keyword operation.
1 is the env.bar keyword value.
@n2 is a node scope for the keyword env.bar

Policies

If a keyword appears in both node.conf and cluster.conf, the value from node.conf takes precedence.
Sections only accept recognized keywords, with the exception of the [env] and [labels] sections, which are open.

More specific scoped values override less specific ones.

With the above section in a svc1 object configuration:

  # on n1:
  $ om svc1 config eval --kw env.bar
  1

  # on n2:
  $ om svc1 config eval --kw env.bar
  2

Syntax validation

A syntax check is performed before finalizing any modifications made with either the set or edit commands.

om cluster config edit

om cluster config set --kw hb#test.type=unsupported

A direct modification to the configuration file is not validated and may disrupt the cluster. In such cases, you can perform a post-hoc validation using:

# verify the syntax of cluster.conf
om cluster config validate

# verify the syntax of node.conf
om node config validate

# verify the syntax of a svc configuration
om svc1 config validate

Node Configuration

Set the Node Environment

sudo om cluster config update --set node.env=PRD

The node.env setting is used to enforce the following policies:

Only production services are allowed to start on a production node.
Only production nodes are allowed push data to a production node.

Supported node.env values:

Env	Behaves As	Description
PRD	PRD	Production
PPRD	PRD	Pre Production
REC	not PRD	Prod-like testing
INT	not PRD	Integration
DEV	not PRD	Development
TST	not PRD	Testing (Default)
TMP	not PRD	Temporary
DRP	not PRD	Disaster recovery
FOR	not PRD	Training
PRA	not PRD	Disaster recovery
PRJ	not PRD	Project
STG	not PRD	Staging

The setting is stored in /etc/opensvc/cluster.conf.

Register on a Collector

Set a Collector Url

By default, the agent does not communicate with a collector.

To enable communications with a collector, the node.dbopensvc node configuration parameter must be set. The simplest expression is:

om cluster config update --set node.dbopensvc=collector.opensvc.com

Here the protocol and path are omitted. In this case, the https protocol is selected, and the path set to a value matching the standard collector integration.

Advanced Url Formats

The following expressions are also supported:

om cluster config update --set node.dbopensvc=https://collector.opensvc.com
om cluster config update --set node.dbopensvc=https://collector.opensvc.com/feed/default/call/xmlrpc

The compliance framework uses a separate xmlrpc entrypoint. The node.dbcompliance can be set to override the default, which is deduced from the node.dbopensvc value.

om cluster config update --set node.dbcompliance=https://collector.opensvc.com/init/compliance/call/xmlrpc

The collector requires the nodes to provide an authentication token (shared secret) with each request. The token is forged by the collector and stored on the node in /etc/opensvc/node.conf. The token initialization is handled by the command:

om node register --user my.self@my.com [--app MYAPP]

If --app is not specified the collector automatically chooses one the user is responsible of.

A successful register is followed by a node discovery, so the collector has detailled information about the node and can serve contextualized compliance rulesets up front. The discovery is also scheduled daily, and can be manually replayed with:

om node push asset
om node push pkg
om node push patch
om node checks

To disable collector communications, use:

om cluster config update --unset node.dbopensvc
om cluster config update --unset node.dbcompliance

Or if the settings were added to node.conf

om node config update --unset node.dbopensvc
om node config update --unset node.dbcompliance

Extra System Configurations

Linux LVM2

OpenSVC controls volume group activation and desactivation. Old Linux distributions activate all visible volume groups at boot, some even re-activate them upon de-activation events. These mechanisms can be disabled using the following setup. It also provides another protection against unwanted volume group activation from a secondary cluster node.

This setup tells LVM2 commands to activate only the objects tagged with the hostname. Opensvc makes sure the tags are set on start and unset on stop. Opensvc also purges all tags before adding the one it needs to activate a volume group, so opensvc can satisfy a start request on a service uncleanly shut down.

/etc/lvm/lvm.conf

Add the following root-level configuration node:

tags {
    hosttags = 1
    local {}
}

And add the local tag to all local volume groups. For example:

sudo vgchange --addtag local rootvg

Finally you need to rebuild the initrd/initramfs to prevent shared vg activation at boot.

/etc/lvm/lvm_$HOSTNAME.conf

echo activation { volume_list = [\"@local\", \"@$HOSTNAME\"] } >/etc/lvm/lvm_$HOSTNAME.conf

Cluster Configuration

Upon agent installation, the node is considered part of its own 1-node cluster.

In /etc/opensvc/cluster.conf:

cluster.secret is initialized to a random value.
cluster.name is initialized to a random value.

Bootstrap a new cluster

If the node joins an existing cluster, skip this section.

Add Heartbeats

If the cluster seed node has no heartbeat setup, a unicast heartbeat with default settings will be automatically added on first join.

This default heartbeat requires every nodename to be resolved to an ip address reachable on 1215/tcp.

If this requirements are not met, you can setup one or more custom heartbeats on the seed node before joins.

For example, a custom heartbeat configuration would be:

    om cluster config update --set hb#1.type=unicast --set hb#1.port=1216

The new heartbeats are visible in the top section of the monitoring command output:

    om mon

See Also:

Heartbeats

Add Stonith methods

Stonith is optional. Skip to the next section if not concerned.

On a new cluster, the stonith configuration can be applied on the first node. The joining nodes will fetch this configuration from this joined node.

For example, a dummy stonith configuration would be

om cluster config update --set stonith#node2.cmd=/bin/true

This configuration will execute :cmd:/bin/true on the node taking over a service which was previously running on the now stalled node2.

Good, isolated fencing packages are freely available. For one, https://github.com/ClusterLabs/fence-agents

Add Arbitrators

Arbitrators are optional. Skip to the next section if not concerned.

The arbitrator configuration can be applied on any node of the cluster.

om cluster config update --set arbitrator#1.name=relay1 \
                      --set arbitrator#1.secret=10231023102310231023102310231023

This configuration will ask for the agent on node relay1 for its vote in a quorum race, if needed to get a majority.

The arbitrator#1.secret value comes from the cluster.secret value on the arbitrator relay1.

See Also:

Quorum

Join a Cluster

The joining node can choose to join any of the cluster node already joined.

On the joined node node1, generate a join token:

$ sudo om daemon auth --role join

On the joining node node2:

sudo om daemon join --token <token> --node node1

Note:

If the node was frozen before the join, it is left frozen after the join.
If the node was not frozen before the join, the join process freezes it. If the join is successful, the node is thawed. If not, the node is left frozen.

Leave a Cluster

sudo om daemon leave

See Also:

Cluster Storage Pools

Services can use volume resources to:

Abstract the disks and filesystems layout, which are hosting specificities, from the service deployment. A development cluster can for example define pools on a ceph cluster, while a production cluster can define pools on fc arrays.
Enable service redeployment while retaining the data.

In this case the translation from volumes to disks and filesystems is delegated to the storage pool drivers.

Pools are defined in the node configuration. Each pool is identified by its name (the section suffix). For example, a pool#tank section defines a pool named tank.

The default pool always exist, even if not defined in the node configuration. If not explicitely changed, the default pool driver is directory.

Volumes

A volume resource drives a volume object, automatically created upon service provisioning if not already existing.
The volume is hosted in the same namespace than its users.
If not explicitely set, the volume object name is <consumer name>-vol-<volume resource index>. For example, a svc1 service with a volume#1 resource will create a svc1-vol-1 volume object.
A volume object can be referenced by multiple services in the same namespace.
On provision, a service adds itself as a child of the volume objects mapped via volume resources. Due to this parent/child relation, stopping a volume object is delayed until all its consumers are stopped.
On unprovision, a service removes itself from the children list of the volume objects mapped via volume resources.
A consumer service instance stop also stops its node-affine volume object instances if the consumer service is the only child of the volume service.
A consumer service instance start always tries to start its node-affine volume object instances.

Volume Resources Keywords

Reference

Access Modes

roo Read Only Once
rwo Read Write Once (default)
rox Read Only from multiple instances
rwx Read Write from multiple instances

Access Mode to Volume Topology

..x access modes imply the volume is configured in a flex topology (active on all service nodes).
..o access modes imply the volume is configured in a failover topology (active on only one service node).

Volume Resource Parameter Requirements

..x + shared=true format=false requires a shared block storage (SAN array, a rados blockdev gateway, ...)
..x + shared=true format=true requires either a shared block storage (SAN array, a rados blockdev gateway, ...) plus a cluster filesystem (gfs2, ocfs, ...), or a distributed cluster filesystem (CephFS, GlusterFS, NAS, ...)

Pool Selector

A volume resource requires a size and capabilities from the pool, via its size, access, shared and format keywords.

If pool is not set explicitely to a pool name, the pool selector will return the available pool matching those criteria with the most free space.

Pool Drivers

drbd

Capabilities

rox, rwx, shared, blk, roo, rwo

Layout

A volume object from this type of pool contains:

If a vg is defined in the pool configuration,

a fs resource, with dev=<drbd devpath>
a drbd resource, layered over a logical volume of the pool vg
a lv resource

If a zpool is defined in the pool configuration,

a fs resource, with dev=<drbd devpath>
a drbd resource, layered over a zvol of the pool zpool
a zvol resource

If the pool configuration has neither vg nor zpool set,

a fs resource, with dev=<drbd devpath>
a drbd resource, layered over a logical volume
a lv resource
a vg resource
a loop resource, with image file hosted in the pool defined path or in <PATHVAR>/pool/<poolname>/

Keywords

Reference

freenas

Capabilities

roo, rwo, shared, blk, iscsi

Layout

A volume object from this type of pool contains:

a disk.disk resource named, with name=<volume fqdn>

If the consumer has format=true (default), the volume object also contains:

a fs. resource, with mnt=/srv/<volume fqdn>

Keywords

Reference

loop

Capabilities

rox, rwx, roo, rwo, blk

Layout

A volume object from this type of pool contains:

a disk.loop resource, with file=<pool head>/<volume fqdn>.img

If the consumer has format=true (default), the volume object also contains:

a fs. resource, with mnt=/srv/<volume fqdn>

Keywords

Reference

symmetrix

Capabilities

roo, rwo, shared, blk, fc

Layout

A volume object from this type of pool contains:

a disk.disk resource named, with name=<volume fqdn>

If the consumer has format=true (default), the volume object also contains:

a fs.<pool fs_type> resource, with mnt=/srv/<volume fqdn>

Keywords

Reference

vg

Capabilities

rox, rwx, roo, rwo, blk, snap

Layout

A volume object from this type of pool contains:

a disk.lv resource, with name=<volume fqdn>

If the consumer has format=true (default), the volume object also contains:

a fs. resource, with mnt=/srv/<volume fqdn>

Keywords

Reference

Capabilities

rox, rwx, roo, rwo, shared

Layout

A volume object from this type of pool contains:

a fs.directory resource, with path=<pool head>/<volume fqdn>.

Keywords

Reference

zpool

Capabilities

rox, rwx, roo, rwo, blk, snap

Layout

A volume object from this type of pool contains:

a fs.zfs resource, with name=<pool>/<volume fqdn> and mnt=/srv/<volume fqdn>.

Keywords

Reference

Virtual Pool Driver

A virtual pool allow administrators to create complex layouts based on volumes from other pools.

A typical use-case in a virtual pool allocating volumes mirrored over two other volumes allocated from arrays on two different sites.

A virtual pool volume is created from a template volume object the administrator can design at wish to meet its specific needs.

Capabilities

Capabilities are user defined.

Keywords

Reference

Pool Commands

Pool list

# om pool ls
default
freenas
mpool

Pool Status

# om pool status
name        type       caps                      head                             vols  size   used   free   
|- default  directory  rox,rwx,roo,rwo           /opt/opensvc/var/pool/directory  0     29.0g  3.57g  24.0g  
|- freenas  freenas    roo,rwo,shared,blk,iscsi  array://freenas/osvcdata         6     195g   9.37g  185g   
`- mpool    virtual    roo,rox,rwo,rwx,shared    templates/mpool                  1     -      -      -

Examples

loop pool

Pool configuration

om cluster config update \
	--set pool#loop.type=loop \
	--set pool#loop.path=/bigfs \
	--set pool#loop.mkfs_opt="-n ftype=1" \
	--set pool#loop.fs_type=xfs

[pool#loop]
type = loop
path = /bigfs
mkfs_opt = -n ftype=1
fs_type = xfs

The volume resource in the service

[volume#1]
size = 100m
pool = loop

Resulting configuration of the volume object

[disk#1]
size = 104857600
type = loop
file = /bigfs/<fqdn>.img

[fs#1]
type = xfs
dev = {disk#1.exposed_devs[0]}
mnt = /srv/<fqdn>
mkfs_opt = -n ftype=1

zfs pool

Pool configuration

om cluster config update \
	--set pool#tank.type=zpool \
	--set pool#tank.name=tank \
	--set pool#tank.mkfs_opt="-o mountpoint=legacy -o dedup=on -o compression=on"

[pool#tank]
type = zpool
name = tank
mkfs_opt = -o mountpoint=legacy -o dedup=on -o compression=on

The volume resource in the service

[volume#1]
size = 100m
pool = tank

Resulting configuration of the volume object

[fs#1]
type = zfs
dev = tank/<fqdn>
mnt = /srv/<fqdn>
mkfs_opt = -o mountpoint=legacy -o dedup=on -o compression=on

virtual pool, mirrored zpool over 2 SAN disks

Pools configuration

om cluster config update \
	--set pool#freenas1.type=array \
	--set pool#freenas1.array=freenas1 \
	--set pool#freenas1.sparse=true \
	--set pool#freenas1.diskgroup=cluster1 \
	--set pool#freenas2.type=array \
	--set pool#freenas2.array=freenas2 \
	--set pool#freenas2.sparse=true \
	--set pool#freenas2.diskgroup=cluster1 \
	--set pool#mpool.type=virtual \
	--set pool#mpool.template=templates/mpool \
	--set pool#mpool.capabilities="rox rwx roo rwo shared"

[pool#freenas1]
type = array
array = freenas1
diskgroup = cluster1
sparse = true

[pool#freenas2]
type = array
array = freenas2
diskgroup = cluster1
sparse = true

[pool#mpool]
type = virtual
template = templates/mpool
capabilities = rox rwx roo rwo shared

The volume object template referenced by the vpool

[DEFAULT]
kind = vol
nodes = *
disable = true

[disk#1]
name = {namespace}-{svcname}
type = zpool
vdev = mirror {volume#1.exposed_devs[0]} {volume#2.exposed_devs[0]}
shared = true

[fs#1]
dev = {disk#1.name}
mnt = /srv/{namespace}/{svcname}
type = zfs
shared = true

[fs#2]
dev = {disk#1.name}/data
mnt = {fs#1.mnt}/data
type = zfs
shared = true

[fs#3]
dev = {disk#1.name}/log
mnt = {fs#1.mnt}/log
type = zfs
shared = true

[volume#1]
format = false
name = {svcname}-1
pool = freenas1
size = {env.size}
shared = true

[volume#2]
format = false
name = {svcname}-2
pool = freenas2
size = {env.size}
shared = true

virtual pool, mirrored lv over 2 SAN disks

Pools configuration

om cluster config update \
	--set pool#freenas1.type=array \
	--set pool#freenas1.array=freenas1 \
	--set pool#freenas1.sparse=true \
	--set pool#freenas1.diskgroup=cluster1 \
	--set pool#freenas2.type=array \
	--set pool#freenas2.array=freenas2 \
	--set pool#freenas2.sparse=true \
	--set pool#freenas2.diskgroup=cluster1 \
	--set pool#mvg.type=virtual \
	--set pool#mvg.template=templates/mvg \
	--set pool#mvg.capabilities="rox rwx roo rwo shared"

[pool#freenas1]
type = array
array = freenas1
diskgroup = cluster1
sparse = true

[pool#freenas2]
type = array
array = freenas2
diskgroup = cluster1
sparse = true

[pool#mvg]
type = virtual
template = templates/mvg
capabilities = rox rwx roo rwo shared

The volume object template referenced by the vpool

[DEFAULT]
kind = vol
nodes = *
disable = true

[volume#1]
shared = true
size = {env.size}
name = {svcname}-1
pool = freenas
format = false

[volume#2]
shared = true
size = {env.size}
name = {svcname}-2
pool = freenas
format = false

[disk#1]
shared = true
type = vg
name = {namespace}-{svcname}
pvs = {volume#1.exposed_devs[0]} {volume#2.exposed_devs[0]}

[fs#1]
shared = true
mnt = /srv/{namespace}/{svcname}
dev = /dev/{disk#1.name}/root
type = ext4
size = 10m
create_options = -m 1
vg = {namespace}-{svcname}

[fs#2]
shared = true
mnt = {fs#1.mnt}/data
dev = /dev/{disk#1.name}/data
type = ext4
size = 60%FREE
create_options = -m 1
vg = {namespace}-{svcname}

[fs#3]
shared = true
mnt = {fs#1.mnt}/log
dev = /dev/{disk#1.name}/log
type = ext4
size = 40%FREE
create_options = -m 1
vg = {namespace}-{svcname}

virtual pool, mirrored md over 2 SAN disks

Pools configuration

om cluster config update \
	--set pool#freenas1.type=array \
	--set pool#freenas1.array=freenas1 \
	--set pool#freenas1.sparse=true \
	--set pool#freenas1.diskgroup=cluster1 \
	--set pool#freenas2.type=array \
	--set pool#freenas2.array=freenas2 \
	--set pool#freenas2.sparse=true \
	--set pool#freenas2.diskgroup=cluster1 \
	--set pool#md.type=virtual \
	--set pool#md.template=templates/md \
	--set pool#md.capabilities="rox rwx roo rwo shared"

[pool#freenas1]
type = array
array = freenas1
diskgroup = cluster1
sparse = true

[pool#freenas2]
type = array
array = freenas2
diskgroup = cluster1
sparse = true

[pool#md]
type = virtual
template = templates/md
capabilities = rox rwx roo rwo shared

The volume object template referenced by the vpool

[DEFAULT]
kind = vol
disable = true
nodes = *

[disk#1]
shared = true
devs = {volume#1.exposed_devs[0]} {volume#2.exposed_devs[0]}
type = md
level = raid1

[disk#2]
shared = true
pvs = {disk#1.exposed_devs[0]}
type = vg
name = {namespace}-{svcname}

[fs#1]
vg = {namespace}-{svcname}
mnt = /srv/{namespace}/{svcname}
dev = /dev/{disk#1.name}/root
shared = true
type = ext4
size = 10m

[fs#2]
vg = {namespace}-{svcname}
mnt = {fs#1.mnt}/data
dev = /dev/{disk#1.name}/data
shared = true
type = ext4
size = 60%FREE

[fs#3]
vg = {namespace}-{svcname}
mnt = {fs#1.mnt}/log
dev = /dev/{disk#1.name}/log
shared = true
type = ext4
size = 40%FREE

[volume#2]
shared = true
size = {env.size}
name = {svcname}-2
pool = freenas
format = false

[volume#1]
shared = true
size = {env.size}
name = {svcname}-1
pool = freenas
format = false

drbd pool

Pool configuration

om cluster config update \
  --set pool#drbdloop.type=drbd

om cluster config update \
  --set pool#drbdvg.type=drbd \
  --set pool#drbdvg.vg=centos

[pool#drbdloop]
type = drbd

[pool#drbdvg]
type = drbd
vg = centos

Example postgres service using a volume from a pool.

[DEFAULT]
nodes = *
orchestrate = ha

[volume#1]
shared = true
size = 200m
name = {name}

[container#1]
type = oci
image = postgres
volume_mounts = {name}/data:/var/lib/postgresql/data
secrets_environment = POSTGRES_PASSWORD=pg/password
rm = true
shared = true

Cluster API

The cluster API can be accessed remotely through any cluster node agent listener.

The cluster API URL servername can resolve as:

A single floating IP address, usually handled by the system/svc/vip failover service
Multiple floating IP addresses
All of the cluster nodes IP addresses
Some of the cluster nodes IP addresses

The listener supports the following authentication methods:

basic

The username is given by the client in every request header.

The system/usr/<username> object must exist on the cluster and provide the grants.
x509

The username is the cn of the certificate.

The system/usr/<username> object must exist on the cluster and provide the grants.
JWT

The username and grants are token claims.

The system/usr/<username> object does not need to exist.

Added in v3 agent.

Create Users

Example:

#
# Create a cluster admin user
# ---------------------------
om system/usr/root create --kw grant=root

#
# Create a namespace ns1 admin user
# with read permission on ns2
# ---------------------------------
om system/usr/usr1 create --kw grant="admin:ns1 guest:ns2"

Testing the API

A demonstration agent exposes the API manifest at https://relay3.opensvc.com/public/ui/

$ TOKEN=$(sudo om daemon auth token --subject usr1 --duration 10m)
$ curl -o- -k -s -H "Authorization: Bearer $TOKEN" https://localhost:1215/whoami
{"auth":"jwt","grant":{"guest":["ns2"], "admin": ["ns1"]},"name":"usr1","namespace":"system","raw_grant":"admin:ns1 guest:ns2"}

Configure the listener

A cluster-level self-signed certificate authority is automatically configured upon agent installation.

The listener needs a TLS certificate to allow remote connections. This certificate is also automatically generated.

The following steps are only necessary to resilver the CA or switch to an external PKI.

With external PKI

export CLUSTERNAME=$(om cluster config get --kw cluster.name)

Store the Certificate Authority certificate chain in a secret.

om system/sec/ca-external create
om system/sec/ca-external key add --name certificate_chain --from ~/ca_crt_chain.pem

Create the Certificate for the TLS listener as a secret.

om system/sec/cert-$CLUSTERNAME create
om system/sec/cert-$CLUSTERNAME certificate create

Make the external CA sign this certificate and load the resulting certificate key.

om system/sec/cert-$CLUSTERNAME create --kw cn=vip.$CLUSTERNAME.mycorp
om system/sec/cert-$CLUSTERNAME key decode --name certificate_signing_request >~/$CLUSTERNAME.csr

signing procedure

om system/sec/cert-clu key add --name certificate --from ~/$CLUSTERNAME_crt.pem
om system/sec/cert-clu key add --name certificate_chain --from ~/$CLUSTERNAME_crt_chain.pem

Declare this Certificate Authority for the TLS listener.

om cluster config update --set cluster.ca=system/sec/ca-external

If available, declare the Certificate Revokation List location, so the listener can refuse revoked certificates before their expiration.

om cluster config update --set cluster.crl=http://crl.mycorp

With internal PKI

At first opensvc daemon startup,

A autosigned CA certificate is created as system/sec/ca
A listener certificate is created as system/sec/cert

Recreate Users certificate

om system/usr/root certificate create
om system/usr/usr1 certificate create


om system/usr/usr1 key decode --name certificate_chain
om system/usr/usr1 key decode --name certificate
om system/usr/usr1 key decode --name private_key

See Also:

RBAC

Cluster API Client

The ox program uses only the agent API and mirrors the om commandset, which makes it suitable for managing one or more clusters from a tiers linux box.

Configure remotes

The remotes configuration is described in YAML format in the ~/.config/opensvc/contexts.yaml file.

Example:

users:
  john:
    password: xxx
  mary:
    password: xxx
clusters:
  dreamy-leopard:
    server: https://dreamy-leopard.example.com:1215
    insecure: true
  bold-rat:
    server: https://bold-rat:1215
contexts:
  john@dreamy-leopard:
    user: john
    cluster: dreamy-leopard
  mary@bold-rat:
    user: mary
    cluster: bold-rat

Terminal UI

At this point, executing ox with no argument launches the Terminal User Interface, and offers a context selector dialog.

The h keypress displays a help page.

Commandline UI

# Set a context
# -------------
$ export OSVC_CONTEXT=john@dreamy-leopard


# Manage like om
# --------------
$ ox cluster get --kw cluster.name
dreamy-leopard 

$ ox node ls
NAME                  AGENT STATE
dreamy-leopard-node-1 3.0.0 idle
dreamy-leopard-node-2 3.0.0 idle
dreamy-leopard-node-3 3.0.0 idle

$ ox svc ls
OBJECT AVAIL OVERALL 
svc2   down  down    
svc1   down  down

Cluster Backend Networks

These networks are only required for services private ip auto-allocation. If configured, the cluster DNS exposes the allocated ip addresses as predictible names, and the cluster Ingress Gateways or portmapping can expose the services to clients outside the cluster.

OpenSVC relies on CNI for this subsystem. Any CNI plugin can be used but some plugins can have dependencies like etcd or consul, which OpenSVC does not require for himself. The bridge plugin, having no such dependencies, is simpler to setup.

Install CNI

From package

Some distributions ship CNI packages.

On Red Hat or CentOS 7, for example, CNI is served by the EPEL repositories:

# to activate epel repositories:
# yum install -y epel-release

yum install -y containernetworking-cni

Then tell OpenSVC where to find the CNI plugins and network configurations:

om cluster config update --set cni.plugins=/usr/libexec/cni \
                         --set cni.config=/var/lib/opensvc/cni/net.d

From upstream

cd /tmp
wget https://github.com/containernetworking/cni/releases/download/v0.6.0/cni-amd64-v0.6.0.tgz
wget https://github.com/containernetworking/plugins/releases/download/v0.6.0/cni-plugins-amd64-v0.6.0.tgz
sudo mkdir -p /opt/cni/bin
cd  /opt/cni/bin
sudo tar xvf /tmp/cni-amd64-v0.6.0.tgz
sudo tar xvf /tmp/cni-plugins-amd64-v0.6.0.tgz
sudo mkdir -p /opt/cni/net.d

Here the plugins and network configurations directories are aligned with the OpenSVC defaults.

Configure networks

Networks are declared in the OpenSVC node or cluster configuration.

The agent create the CNI configuration files as needed.

Local Bridge

A local bridge network is always present and named default.

To create another network of this type, named local1, available on every cluster node:

$ om cluster config update --set network#local1.type=bridge \
                           --set network#local1.network=10.10.10.0/24

To create another network of this type, named local1, available on the current cluster node only:

$ om node config update --set network#local1.type=bridge \
                        --set network#local1.network=10.10.10.0/24

Routed Bridge

This network type split the subnet into per-node segments. Trafic is routed from node-to-node via static routes to each segment, and ipip tunnels are created if necessary.

The simple bridge CNI plugin is used for IPAM and plumbing in network namespaces, and OpenSVC is responsible for node-to-node routing and tunneling.

To create a network of this type, named backend1, spanned on every cluster node:

$ om cluster config update --set network#backend1.type=routed_bridge \
                           --set network#backend1.network=10.11.0.0/16 \
                           --set network#backend1.ips_per_node=1024

In this example, the network is split like:

node 1 : 10.11.0.0/22
node 2 : 10.11.4.0/22
node 3 : 10.11.8.0/22
...

Tunnel endpoints addresses are guessed using a lookup of the nodenames. Different addresses can be setup if necessary, using:

$ om cluster config update --set network#backend1.addr@node1=1.2.3.4 \
                           --set network#backend1.addr@node2=1.2.3.5 \
                           --set network#backend1.addr@node3=1.2.4.4

Some hosting providers, like OVH, don't support static network routes from node to node, even if they have an ip address in a common subnet. For this situation, you can force OpenSVC to always use tunnels for this backend network::

$ om cluster config update --set network#backend1.tunnel=always

The default tunnel mode is ipip if the network is ipv4, or ip6ip6 if the network is ipv6. The tunnel_mode keyword of the routed_bridge driver also accepts gre. The GRE tunnels can transport both ipv4 and ipv6 and may work in some hosting situations where ipip does not work (OVH).

Use in service configurations

Here is a typical ip resource configuration, using the "weave" CNI network configured above.

[ip#0]
type = cni
network = backend1
netns = container#0
expose = 80/tcp

The container pointed by netns can be a docker or lxc container. netns can also be left empty, causing the weave ip address to be assigned to the service cgroup.

The expose keyword is optional. If set, a SRV record is served by the cluster DNS (in this example _http._tcp.<svcname>.<namespace>.svc.<clustername>). If expose is set to portmapping expression, for example 80:8001/tcp, the portmap CNI plugin is will configure the portmapping and expose the 80/tcp backend server on the 8001 port of the node public ip addresses.

Useful commands

# om net ls
NAME      TYPE           NETWORK        SIZE   USED  FREE   
backend1  routed_bridge  fdfe::/112     65536  0     65536  
backend2  routed_bridge  fdff::/112     65536  0     65536  
backend3  routed_bridge  10.100.0.0/22  1024   2     1022   
lo        lo             127.0.0.1/32   1      0     1      
default   bridge         10.22.0.0/16   65536  0     65536

List the IP addresses allocated in networks associated with their respective requester object and resource:

# om net ip ls
OBJECT               NODE    RID   IP          NET_NAME  NET_TYPE       
testigw/svc/haproxy  dev2n1  ip#1  10.100.0.2  backend3  routed_bridge  
testigw/svc/haproxy  dev2n2  ip#1  10.100.1.2  backend3  routed_bridge  
...

Cluster Domain Name Server

The OpenSVC agent daemon can act as a remote backend for PowerDNS, serving dynamic records for services deployed within the cluster. This functionality is particularly useful when services are assigned IP addresses on private backends with internal IPAM.

If enabled, the agent configures the container's resolver (nameserver and search) to use the internal name server when starting a container.

This feature is not enabled by default.

Records

A record: <hostname>.<svcname>.<namespace>.svc.<clustername> for each resource that includes ipaddr and hostname in the info map in its states.
Round-Robin A Record: <svcname>.<namespace>.svc.<clustername> where each resource that includes ipaddr in the info map in its states is included in the round-robin.
Round-Robin SRV Record: _<service>._<protocol>.<svcname>.<namespace>.svc.<clustername> where each resource with an expose keyword matching <port>/<service> is included in the round-robin.

Note:

A service created without a specific namespace defaults to the root namespace.

Implementation

A farmed (flex) service.
Each instance runs a authoritative PowerDNS server, a PowerDNS recursor and a recursor cache janitoring daemon.
Each component runs as a privileged docker instance to have r/w access to shared unix domain sockets.
The DNS server and recursor share the node network namespace.
The PowerDNS server uses the dns thread of the OpenSVC daemon as a remote backend. Communications go through the /var/lib/opensvc/dns/pdns.sock unix domain socket.

Docker images

ghcr.io/opensvc/pdns_server
ghcr.io/opensvc/pdns_recursor
ghcr.io/opensvc/pdns_janitor

Configure

Preliminary steps

Make sure the cluster configuration :kw:cluster.name is set to a meaningful, unique site-wide, value. It can be a fqdn like cluster1.my.org, or just a basename like cluster1.
Choose at least 2 cluster nodes that will act as DNS backends.
Choose a free port for the DNS to listen on (default is 5300).
Identify the ip addresses you want the DNS to listen on (public or private). In the following examples, 192.168.100.11 and 192.168.100.14.
Make sure these ip addresses are resolved to the node name as declared in the :kw:cluster.nodes keyword (edit /etc/hosts if necessary).
OpenSVC agent installed, minimum version 2.1-1651
Make sure docker or podman is installed and running on selected dns nodes.
Make sure CNI is installed
Make sure you have access to pull from docker.io on selected dns nodes (you can pre-pull or save/load the images if not).

Declare DNS backends

om cluster config update --set cluster.dns+=192.168.100.11 --set cluster.dns+=192.168.100.14

Deploy the DNS service

om system/cfg/dns create
om system/cfg/dns key add --name server --from https://raw.githubusercontent.com/opensvc/opensvc_templates/main/dns/pdns.conf.template
om system/cfg/dns key add --name recursor --from https://raw.githubusercontent.com/opensvc/opensvc_templates/main/dns/recursor.conf.template
om system/cfg/dns key add --name configure --from https://raw.githubusercontent.com/opensvc/opensvc_templates/main/dns/configure
om system/svc/dns deploy --config https://raw.githubusercontent.com/opensvc/opensvc_templates/main/dns/dns.conf

Note:

Make sure allow-from in the recursor key of system/cfg/dns contains all the cluster backend networks allowed to request the DNS (the default is 127.0.0.1/32,10.0.0.0/8,fd00::/112,::1).

Configure the nodes resolver

On every node, execute:

# create the resolved configlet directory if it doesn't exist yet
$ mkdir -p /etc/systemd/resolved.conf.d

# install a configlet routing all requests to the cluster domain to the cluster nameservers
$ cat - <<EOF >/etc/systemd/resolved.conf.d/opensvc.conf
[Resolve]
Domains=$(om cluster config get --kw cluster.name)
DNS=$(om cluster config get --kw cluster.dns)
EOF

# activate the new configuration
$ systemctl restart systemd-resolved.service

Verify

Verify the backend

Dump the records served by opensvc to the PowerDNS server

om daemon dns dump

Test the unix socket served by opensvc for the PowerDNS server

echo '{"method": "list", "parameters": {"zonename": "cluster1."}}' | sudo socat - unix://var/lib/opensvc/dns/pdns.sock | jq

Verify the DNS server

Dump the zone contents asking the PowerDNS server

dig +noall +answer cluster1. AXFR @192.168.100.11 -p 5300

Verify the DNS recursor

dig +short cluster1. SOA @192.168.100.11

Administration

Add forwarding for the reverse zones

Either switch to --forward-zones-file or add new elements to forward-zones in the recursor key of system/cfg/dns.

Ingress Gateway

Services configured to obtain an IP address from a backend network are not naturally accessible to clients outside the cluster.

To expose them, the user or a cluster administrator can deploy a ingress gateway configured with a public IP address.

HAProxy is our recommended program to route layer 4 and layer 7 communications from the frontend to the backend servers.

Behaviour

The backend composition is kept up to date by HAProxy the resolvers mechanism.

To declare the cluster dns in the HAProxy configuration:

resolvers clusterdns
    parse-resolv-conf
    accepted_payload_size 8192

As the HAProxy server runs in a container resource started by OpenSVC, the /etc/resolv.conf file contains the cluster nameservers IP address. The parse-resolv-conf tells HAProxy to read the nameservers from there.

This resolvers configuration can be referenced in every backend definition like:

backend svc1
    option httpchk GET /health
    server-template svc1_ 1 svc1.ns1.svc.${CLUSTERNAME}:8080 resolvers clusterdns check init-addr none

Configurations

Intra-Cluster Load-Balancing: Run only one HAproxy on the cluster, in a failover topology svc.
Extra-Cluster Load-Balancing: Every node runs a HAProxy exposing the same servers. The upstream load-balancer picks one.

Intra-Cluster Load-Balancing Configuration

Listen on port 443, with a self-signed certificate.

Deploy a haproxy service using the basic example from the igw_haproxy template page on github.

# Create a self signed key and certificate
sudo om testigw/sec/haproxy create
sudo om testigw/sec/haproxy certificate create

# Create a haproxy configuration as a cfg key
sudo om testigw/cfg/haproxy create
sudo om testigw/cfg/haproxy key add --name haproxy.cfg --from https://raw.githubusercontent.com/opensvc/opensvc_templates/main/igw_haproxy/basic-cfg-haproxy.cfg

# Deploy the Ingress Gateway svc
# * change the network to a cluster spaning network if you have one setup
# * make sure requests from this network are allowed by the nameservers
sudo om testigw/svc/haproxy deploy --config https://raw.githubusercontent.com/opensvc/opensvc_templates/main/igw_haproxy/basic-svc.conf --kw ip#1.network=default

A ip#1 failover-capable public IP address should be added and started for this service to be useful to extra-cluster clients, but it can be tested from a cluster node already.

# Store the haproxy IP address allocated on start
eval IP=$(sudo om testigw/svc/haproxy resource ls -o json --rid ip --node $HOSTNAME| jq .items[].data.status.info.ipaddr)

# Test, faking a DNS resolution of svc1.opensvc.com to the haproxy ip address
curl -o- -k --resolve svc1.opensvc.com:443:$IP https://svc1.opensvc.com

# Deploy a test webserver to populate the svc1.opensvc.com backend:
# * change the network to a cluster spaning network if you have one setup
# * make sure requests from this network are allowed by the nameservers
sudo om testigw/svc/svc1 deploy --config https://raw.githubusercontent.com/opensvc/opensvc_templates/main/igw_haproxy/nginx.conf --kw ip#1.network=default --wait

# Retest until available
curl -o- -k --resolve svc1.opensvc.com:443:$IP https://svc1.opensvc.com

Automated Certificate Management Environment

The igw_haproxy template page on github also documents the deployment of a HAProxy cluster ingress gateway service implementing ACME.

Upgrade

The agent supports upgrading with zero service down-time.

Upgrading does not require a node reboot.

Ubuntu

sudo apt update
sudo apt install opensvc-server

Deploy Apps

Applications are composed of one or more objects (services, configs, secrets, volumes, service accounts). These objects can be deployed and operated individually or as a group.

Naming

A fully qualified object name is formatted as <namespace>/<kind>/<name>.

Namespace

Namespaces allow users to create objects with the same name in different naming spaces.

Namespace names must conform to RFC952:

only alphanum characters or dash
start with an alpha
end with an alphanum

Kind

svc A service, with a mix of ip, app, container, volume, disk, fs and task resources.
vol A data volume from a pool, with a mix of volume, disk and fs resources.
cfg A configuration map, storing unencrypted key/value pairs for use by other kinded objects.
sec A secret, storing encrypted key/value pairs for use by other kinded objects.
ccfg The special kind for the cluster configuration object.

Name

Names must conform to RFC952, with a tolerance for dots.

A name must be unique in its namespace and kind.

Create, Deploy

The following actions only modify files in /etc/opensvc. No operating system configuration file is modified, so they are safe to experiment with.

The agent support object creation via two commands:

create The object is created but not provisioned nor started.
deploy The object is created, provisioned and started.

Both actions support the same arguments. The following examples use only create commands.

From Scratch, non Interactive

Create a new object with minimal configuration. No resources are described.

om <path> create

Resources and default keywords can be set right from the create command, using --kw <keyword>=<value> options

om <path> create
	--kw container#0.type=oci \
	--kw orchestrate=ha \
	--kw nodes={clusternodes}

From Another Object

om <dst path> create --config=<src path>

From Manifest, Single Object

The manifest must be json formatted, structured like om <path> print config --format=json.

om <path> create --config=<manifest uri>

This method can also be used to clone objects

om <src path> print config --format json | \
	om <dst path> create --config=- [--interactive] [--provision]

From Manifest, Multiple Objects

The manifest must be json formatted, structured like om <selector> print config --format=json.

In this case, the <dst path> can not be specified, but the destination namespace where to create the objects can. The new objects will adopt the names set in the manifest.

om svc create --namespace=newns --config=<manifest uri>

om 'test/svc/*' print config --format=json | \
	om svc create --namespace=testclone --config=-

From Existing Local Configuration File

Experienced users may find it easier to start from a copy of the conf file of an existing similar object.

om <path> create --config <path to config file> [--interactive] [--provision]

The configuration file can be remote, referenced by URI.

From Collector Template

Templates can be served by the collector.

om <path> create --template <id|name> [--interactive] [--provision]

Update

Configuration files are stored in /etc/opensvc/.

/etc/opensvc/<name>.conf

Root objects configuration file:
/etc/opensvc/namespaces/<namespace>/<kind>/<name>.conf

Namespaced objects configuration file:

Do not edit these files directly. Use one of the following method instead.

Interactive

om <path> config edit

The configuration file syntax is checked upon editor exit. The new configuration is installed if the syntax is found correct, or saved in a temporary location if not. Two options are then possible:

Discard the erroneous configuration:
```
 om <path> config edit --discard
```
Re-edit the erroneous configuration:
```
 om <path> config edit --recover
```

Non-Interactive Resource Addition

om <path> config update --set fs#2.type=ext4 --set fs#2.mnt=/srv/{fqdn}

The resource identifier (rid) must not be specified. The resource type must be specified (rtype). A free rid will be allocated.

Non-Interactive Resource Modification

om <path> config update --set fs#2.type=ext4 --set fs#2.mnt=/srv/{fqdn}

The resource identifier must be specified.

Non-Interactive Resource Deletion

om <path> config update --delete fs#2

This command does not stop the resource before removing its definition. If desired, this can be done with

om <path> stop --rid fs#2

Purge

om <path> purge

This command asks the cluster to orchestrate a stop, unprovision and delete. Non-leader instances are sequenced first.

Purging a service does not purge its referenced volumes.

Purging a volume actually removes all volume data.

Delete

om <path> delete

This command does not stop nor unprovision the object, so it can leave unreferenced mounts, containers and processes on the nodes.

This command should be used by administrators only.

Operate Apps

Design Apps

Internals

Installed Items

Directories

/etc/opensvc

The cluster, node and objects configuration files.
/var/lib/opensvc

The state files. Deleting or creating files in this directory can have undesired side-effects.
/var/tmp/opensvc

Temporary files. Deleting or creating files in this directory can have undesired side-effects.

Executable files

/usr/bin/om

This executable, installed by the opensvc-server package, implements:
- The Cluster Resource Manager
- The Cluster Monitor and API daemon
- The local management commandline interface
/usr/bin/ox

This executable, installed by the opensvc-client package, implements:
- The remote management commandline interface

Configuration files

The agent configuration is the result of the merge of two ini configuration files:

/etc/opensvc/cluster.conf

This file is replicated on all cluster nodes.
/etc/opensvc/node.conf

This file is not replicated.

Heartbeats

Heartbeats serve the following purposes:

Exchange data between cluster nodes.
Detect stale nodes.
Execute the quorum race when a peer becomes stale.

OpenSVC supports multiple parallel running heartbeats. Exercising different code paths and infrastructure data paths (network and storage switches and site interconnects) helps limit split-brain situations.

Configuration

Heartbeats are declared in /etc/opensvc/cluster.conf, each in a dedicated section named [hb#<n>]. A heartbeat definition should work on all nodes, using scoped keywords if necessary, as the definitions are served by the joined node to the joining nodes.

Reconfiguration

Any command that changes the timestamp of the following configuration files triggers a reconfiguration of heartbeats:

/etc/opensvc/node.conf
/etc/opensvc/cluster.conf

Actions Taken During Reconfiguration:

Any updated parameters are applied to the heartbeats.
Heartbeats removed from the configuration are stopped.
Heartbeats newly defined in the configuration are started.

Set a Heartbeat Timeout

To set a timeout for the hb#1 heartbeat, use this command:

om cluster config update --set hb#1.timeout=20

Drop a Heartbeat

To delete the hb#1 heartbeat from the configuration:

om cluster config update --delete hb#1

Monitoring

Each heartbeat runs two threads: tx and rx.

The om mon command display the heartbeats status, statistics, and each peer state.


Threads                                n1        n2        n3        
 ...
 hb                                  |                                           
  hb#1.rx          running unicast   | /         O         O             
  hb#1.tx          running unicast   | /         O         O             
  hb#2.rx          running relay     | /         O         O             
  hb#2.tx          running relay     | /         O         O             
 ...

The agent daemon automatically restarts heartbeat threads if they exit unexpectedly.

Heartbeat Thread Pair

Tx (Transmit)

The Tx thread handles the transmission of the node data:

Regularly transmit data or send it as soon as changes occur.
Data is encrypted.

Rx (Receive)

The Rx thread manages data reception and integration into cluster data:

Regularly read data from disk or receive it in response to transmissions (unicast/multicast).
Update peer data in the cluster.
Timeout if no heartbeat is received within the configured <hb#n>.timeout. The default timeout is 15 seconds.

Actions Performed by Rx:

On receive data:
- Merge updated peer data to maintain accurate cluster data.
- Publish the received events on the local event bus.
On receive timeout:
- Publish a HbStale event
- Purge stale peer data if:
  - No Maintenance Advertised: Immediately purge stale peer data.
  - Maintenance Advertised: Wait for the node.maintenance grace_period before purging.

See Also:

Cluster Data

hb.unicast

The hb.unicast driver sends and receives using TCP unicast packets.

Basic Configuration

[hb#1]
type = unicast

Behavior with Basic Configuration

The Rx thread listens on 0.0.0.0:10000
The Tx thread sends to <nodename>:10000

Advanced Configuration

A more precise definition allows specifying network interfaces, addresses, and ports for each node:

[hb#1]
type = unicast
intf@node1 = eth0
intf@node2 = eth2
addr@node1 = 1.2.3.4
addr@node2 = 1.2.3.5
port@node1 = 10001
port@node2 = 10002
timeout = 15s

Note the driver accepts to use the same port for every node:

port = 10001

Proper configuration of the hb.unicast driver ensures reliable communication between cluster nodes by leveraging TCP unicast.

hb.multicast

The hb.multicast driver sends and receives using TCP multicast packets.

Basic Configuration

[hb#2]
type = multicast

Behavior with Basic Configuration

The Rx thread listens on all interfaces on port 10000
The Tx thread sends to 224.3.29.71:10000

Advanced Configuration

A more precise definition allows specifying network interfaces, addresses, and ports for each node:

[hb#2]
type = multicast
intf@node1 = eth0
intf@node2 = eth2
addr = 224.3.29.71
port = 10001
timeout = 15

The addr and port keywords are not scopable.

hb.disk

This driver reads and writes on a dedicated disk, using O_DIRECT|O_SYNC|O_DSYNC open flags on a block device on Linux.

Configuration

[hb#2]
type = disk
dev = /dev/mapper/3123412312412414214
timeout = 15

Behavior

The Rx thread loops over peer nodes and for each reads its heartbeat data at its reserved slot device offset
The Tx thread writes to its reserved slot offset on the device

On-disk format

When the tx and rx threads are started or reconfigured, they parse a metadata segment at the head of the device and prepare a <nodename>:<slot index> hash.

The metadata zone maximum size is 4MB.

A node metadata slot size is 4k, and contains the cluster node name.

Limits:

1000 nodes (metadata zone size/slot meta data size)
nodenames are limited to 4k characters (slot meta data size)
A -nodes cluster requires a (<n>+1)*4MB device
The heartbeat data (which is gziped) must not exceed 4MB (slot size). A 10 services cluster usually produces ~3k messages.

If a the local nodename is not found in any slot, the thread allocates one.

hb.relay

This driver reads and writes on a remote opensvc agent memory.

Configuration

[hb#2]
type = relay
relay = relay3.opensvc.com
timeout = 15
username = relay
password = system/sec/relay

Note the v3 relay configuration no longer supports the secret keyword. The authentication creadentials are specified using the username and password keywords. The password value is the path of a sec object containing a password key.

Behaviour

The relay listener <address>:<port> must be reachable from all cluster nodes in normal operations.

A relay should be located in a site hosting no other node of the cluster, so this heartbeat can prevent a split when the sites hosting cluster nodes are disconnected, but can still reach the relay's site.

The same relay can be used as heartbeat in different clusters. The relay host can also be used as an arbitrator.

The rx thread loops over peer nodes and for each requests its heartbeat data from the relay
The tx thread sends to the relay

OpenSVC v3 clusters must use a OpenSVC v3 relay.

Quorum

When a peer is flagged as stale by all heartbeats, the daemon assumes the cluster is in a split-brain situation, as it cannot determine whether the stale peer has failed or is isolated.

OpenSVC minimizes the likelihood of a split-brain scenario by leveraging multiple independent heartbeats.

Enabling Quorum Enforcement

Users who prefer to have a cluster segment shut down in such situations can enable quorum by setting cluster.quorum to true:

om cluster config update --set cluster.quorum=true

By default, the system allows split nodes to take over services, which may result in services running on multiple isolated segments. To revert to the default behavior, use:

om cluster config update --unset cluster.quorum

To check the current quorum configuration:

om cluster config get --kw cluster.quorum

Quorum Behavior

If the cluster is configured for quorum and a split-brain situation occurs, a node will shut down if the number of reachable nodes (including itself) plus arbitrators is less than half of the total cluster and arbitrator nodes.

Frozen nodes do no evaluate quorum. They will not shut down on split-brain.

Frozen nodes still vote for peer nodes quorum evaluation.

Example Arbitrator Requirements

To survive a interconnect outage:

In a 2-node cluster, a single node requires 1 arbitrator vote to survive the split.
In a 3-node cluster, a single node requires 2 arbitrator votes.
In a 4-node cluster, a single node requires 3 arbitrator votes.
In a 5-node cluster, a single node requires 3 arbitrator votes.

To survive a interconnect outage, plus all peers outage in the same availability zone:

In a 2-node cluster, a single node requires 1 arbitrator vote to survive the split.
In a 3-node cluster, a single node requires 2 arbitrator votes.
In a 4-node cluster, a single node requires 3 arbitrator votes.
In a 5-node cluster, a single node requires 4 arbitrator votes.

Configuring Arbitrators

Any OpenSVC agent can act as an arbitrator, and multiple arbitrators can be configured. For example, to configure an arbitrator:

Use a https server as an arbitrator

[arbitrator#a1]
uri = https://dev2n1:1215/metrics
#insecure = true

Use a tcp server as an arbitrator

[arbitrator#a2]
uri = dev2n2:22

Testing Arbitrators

Alive test of an arbitrator:

    $ om node ping --node a1

The om mon output show all arbitrator alive state from the point of view of every node.

    $ om mon
    ...
    Arbitrators                       n1   n2
     a1                warn         | X    X          
     a2                warn         | X    X          
     a3                             | O    O          
    ...

Best Practices

Configure minus 1 arbitrators
Host all arbitrators on the same 3rd site
Use one of the arbitrators as a relay for the relay heartbeat driver
Disable quorum or freeze all nodes when doing a relayout of the cluster

Example: odd-nodes cluster

    +-------------------------------------------+                                
    |  site3                                    |                                
    |                                           |                                
    |  +-------------+   +-------------+        |
    |  |             |   |             |        |
    |  | arbitrator1 |   | arbitrator2 |        |
    |  |             |   |             |        |
    |  +-------------+   +-------------+        |
    |                                           |                                
    +-------------------------------------------+                                
    
    +-------------------------------------------+     +------------------------------+
    | site1                                     |     |   site2                      |
    |                                           |     |                              |
    |    +--------------------------------------|-----|-----------------------+      |
    |    | cluster                              |     |                       |      |
    |    |                                      |     |                       |      |
    |    |    +-----------+    +-----------+    |     |   +-----------+       |      |
    |    |    |           |    |           |    |     |   |           |       |      |
    |    |    |   node1   |    |   node2   |    |     |   |   node3   |       |      |
    |    |    |           |    |           |    |     |   |           |       |      |
    |    |    +-----------+    +-----------+    |     |   +-----------+       |      |
    |    |                                      |     |                       |      |
    |    +--------------------------------------|-----|-----------------------+      |
    |                                           |     |                              |
    +-------------------------------------------+     +------------------------------+

Total: 5 votes
Majority: 3 votes

Site1 Isolated

node1 standpoint:

live nodes: 2 (node1, node2)
arbitrators votes: 0
votes: 2

=> node does not have quorum, commits suicide

node2 standpoint

live nodes: 2 (node1, node2)
arbitrators votes: 0
votes: 2

=> node does not have quorum, commits suicide

node3 standpoint

live nodes: 1 (node3)
arbitrators votes: 2
votes: 3

=> node has quorum, does not commit suicide

Site2 Isolated

node1 standpoint

live nodes: 2 (node1, node2)
arbitrators votes: 2
votes: 4

=> node has quorum, does not commit suicide

node2 standpoint

live nodes: 2 (node1, node2)
arbitrators votes: 2
votes: 4

=> node has quorum, does not commit suicide

node3 standpoint

live nodes: 1 (node3)
arbitrators votes: 0
votes: 1

=> node does not have quorum, commits suicide

Node2 Dies

node1 standpoint

live nodes: 2 (node1, node3)
arbitrators votes: 2
votes: 4

=> node has quorum, does not commit suicide

node3 standpoint

live nodes: 2 (node1, node3)
arbitrators votes: 2
votes: 4

=> node has quorum, does not commit suicide

Node2 and Node3 Die

node1 standpoint

live nodes: 1 (node1)
arbitrators votes: 2
votes: 3

=> node has quorum, does not commit suicide

Example: even-nodes cluster

    +-------------------------------------------+                                
    |  site3                                    |                                
    |                                           |                                
    |  +-------------+                          |
    |  |             |                          |
    |  | arbitrator1 |                          |
    |  |             |                          |
    |  +-------------+                          |
    |                                           |                                
    +-------------------------------------------+                                
    
    +--------------------------+     +------------------------------+
    | site1                    |     |   site2                      |
    |                          |     |                              |
    |    +---------------------|-----|-----------------------+      |
    |    | cluster             |     |                       |      |
    |    |                     |     |                       |      |
    |    |    +-----------+    |     |   +-----------+       |      |
    |    |    |           |    |     |   |           |       |      |
    |    |    |   node1   |    |     |   |   node2   |       |      |
    |    |    |           |    |     |   |           |       |      |
    |    |    +-----------+    |     |   +-----------+       |      |
    |    |                     |     |                       |      |
    |    +---------------------|-----|-----------------------+      |
    |                          |     |                              |
    +--------------------------+     +------------------------------+

Total: 3 votes
Majority: 2 votes

Site1 Isolated

node1 standpoint

live nodes: 1 (node1)
arbitrators votes: 0
votes: 1

=> node does not have quorum, commits suicide

node2 standpoint

live nodes: 1 (node2)
arbitrators votes: 1
votes: 2

=> node has quorum, does not commit suicide

Node1 dies

node2 standpoint

live nodes: 1 (node2)
arbitrators votes: 1
votes: 2

=> node has quorum, does not commit suicide

Scheduler

The OpenSVC agent includes a scheduler that manages jobs for both the node and each individual service.

Basic Schedule Definition

The schedule constraints are defined by allowed time ranges and minimum execution interval. An example schedule definition is 00:00-02:00@121m. In this example:

Time Range: From midnight to 2:00 AM.
Interval: 121 minutes.

Multiple schedule definitions can be specified using the syntax:

["00:00-02:00@121", "12:00-14:00@121"]

Execution is permitted if any one of the defined constraints is satisfied.

Policies

If an allowed time range is longer than the interval, multiple executions happen in the time range.

If not specified, the default interval is the duration of the time range, so there is only one execution of the job during the time range.

If not specified, the default time range is unrestricted. In this case a period must be specified.

If the definition begins with a ~, the execution is delayed randomly in the allowed time range. The probability of execution increases linearly as time progresses within the allowed time range. For instance:

At the beginning of the time range (00:00 in 00:00-02:00), the probability might be around 10%.
Near the end of the time range (01:50), the probability reaches 100%.

This behavior ensures that the execution of job reporting information to the collector is spread across all nodes throughout the entire time range, leveling the load on the central collector. This approach prevents sudden spikes in load.

Node Scheduler

$ om node schedule list -o +KEY:data.key
NODE  ACTION           LAST_RUN_AT                NEXT_RUN_AT           SCHEDULE      KEY                       
n1    pushasset        2025-01-27T05:57:06+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  asset.schedule            
n1    checks           2025-01-27T01:54:15+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  checks.schedule           
n1    compliance_auto  2025-01-27T02:00:00+01:00  0001-01-01T00:00:00Z  02:00-06:00   compliance.schedule       
n1    pushdisks        2025-01-27T04:56:30+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  disks.schedule            
n1    pushpkg          2025-01-27T18:59:54+01:00  0001-01-01T00:00:00Z  @1m           packages.schedule         
n1    pushpatch        2025-01-27T04:58:22+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  patches.schedule          
n1    sysreport        0001-01-01T00:00:00Z       0001-01-01T00:00:00Z  ~00:00-06:00  sysreport.schedule        
n1    dequeue_actions  0001-01-01T00:00:00Z       0001-01-01T00:00:00Z                dequeue_actions.schedule  
n2    pushasset        2025-01-29T00:35:49+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  asset.schedule            
n2    checks           2025-01-29T00:10:39+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  checks.schedule           
n2    compliance_auto  2025-01-29T02:00:00+01:00  0001-01-01T00:00:00Z  02:00-06:00   compliance.schedule       
n2    pushdisks        2025-01-29T05:14:15+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  disks.schedule            
n2    pushpkg          2025-01-29T05:33:22+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  packages.schedule         
n2    pushpatch        2025-01-29T00:42:55+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  patches.schedule          
n2    sysreport        2025-01-29T03:08:18+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  sysreport.schedule        
n2    dequeue_actions  0001-01-01T00:00:00Z       0001-01-01T00:00:00Z                dequeue_actions.schedule  
n3    pushasset        2025-01-29T04:50:18+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  asset.schedule            
n3    checks           2025-01-29T05:17:24+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  checks.schedule           
n3    compliance_auto  2025-01-29T02:00:00+01:00  0001-01-01T00:00:00Z  02:00-06:00   compliance.schedule       
n3    pushdisks        2025-01-29T05:10:43+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  disks.schedule            
n3    pushpkg          2025-01-29T03:07:57+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  packages.schedule         
n3    pushpatch        2025-01-29T05:36:14+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  patches.schedule          
n3    sysreport        2025-01-29T00:34:02+01:00  0001-01-01T00:00:00Z  ~00:00-06:00  sysreport.schedule        
n3    dequeue_actions  0001-01-01T00:00:00Z       0001-01-01T00:00:00Z                dequeue_actions.schedule

The scheduled jobs can be configured in /etc/opensvc/node.conf with a configlet like:

[<section>]
<parameter> = <definition>

The KEY column in the above command output is formatted as:

<section>.<parameter>

And the current definition, explicit or implicit, is visible in the SCHEDULE column. Empty means never scheduled.

The om node command action executed when the job fires is displayed in the ACTION column.

The node supports the following jobs:

Node inventoring tasks : pushasset pushpatch pushpkg pushdisks
Node performance metrics inventoring : pushstats
Node performance metrics collection : collect_stats
Node file content tracking task : sysreport
Node configuration audit and/or remediation task : compliance_auto
Health checking task : checks
Scheduled node reboot task : auto_reboot
Scheduled root password rotation task : auto_rotate_root_pw
Execution of node actions queued by the collector : dequeue_actions
SAN switches inventoring tasks : pushbrocade
Storage arrays inventoring tasks : pushcentera pushdcs pushemcvnx pusheva pushfreenas pushhds pushhp3par pushibmds pushibmsvc pushnecism pushnetapp pushsym pushvioserver
Backup servers saves index inventoring tasks : pushnsr

Service Scheduler

$ om tflex schedule list
OBJECT  NODE    ACTION           KEY               LAST_RUN_AT                NEXT_RUN_AT                SCHEDULE    
tflex   dev2n1  status           status_schedule   2025-01-30T11:54:55+01:00  2025-01-30T12:04:55+01:00  @10m        
tflex   dev2n1  compliance_auto  comp_schedule     2025-01-27T00:09:18+01:00  0001-01-01T00:00:00Z       ~00:00-06:00
tflex   dev2n1  run              task#1.schedule   2025-01-28T16:27:16+01:00  2025-01-30T16:27:16+01:00  @2d         
tflex   dev2n1  run              task#2.schedule   2025-01-29T16:27:08+01:00  2025-01-30T16:27:08+01:00  @1d         
tflex   dev2n1  run              task#3.schedule   2025-01-29T16:27:08+01:00  2025-01-30T16:27:08+01:00  @1d         
tflex   dev2n1  push_resinfo     resinfo_schedule  2025-01-27T18:56:47+01:00  0001-01-01T00:00:00Z       @60m

The scheduled jobs can be configured in the service configurations with a configlet like:

[<section>]
<parameter> = <definition>

The KEY column in the above command output is formatted as:

<section>.<parameter>

And the current definition, explicit or implicit, is visible in the SCHEDULE column. Empty means never scheduled.

The om <path> command action executed when the job fires is displayed in the ACTION column.

The supported jobs are:

Service configuration audit and/or remediation : compliance_auto
Service resources kvstores inventoring : push_env
Service status evaluation : status
Service data sync : sync_all

Advanced Schedule Definition

[!] <timeranges> [<days> [<weeks> [<months>]]]

!
  desc: exclusion pattern. ommiting the ! implies an inclusion

<timeranges> := <timerange>[,<timerange>]
  <timerange> := <begin>:<end>@<interval>
    <begin> <end> := <hour>:<minute>
    <interval>
      type: integer
      unit: minutes

<days> := <day>[-<day>][,<day>[-<day>]]
  <day> := <day_of_week>[:<day_of_month>]
    <day_of_week>
       * iso week day format
         type: integer between 0 and 6
       * literal format
         type: string in ("mon", "tue", "wed", "thu", "fri", "sat",
               "sun", "monday", "tuesday", "wednesday", "thursday",
               "friday", "saturday", "sunday")
    <day_of_month> := <literal> | +<nth> | -<nth> | <nth>
       <nth>
         type: integer
       <literal>
         type: string in ("first", "1st", "second", "2nd", "third",
               "3rd", "fourth", "4th", "fifth", "5th", "last")

<weeks> := <week>[-<week>][,<week>[-<week>]]
  <week>
    type: integer between 1 and 53

<months> := <monthrange>[,<monthrange>]
  <monthrange> := <month>[-<month>] | <month_filter>
    <month>
      * numeric month format
        type: integer between 1 and 12
      * literal format
        type: string in ("jan", "feb", "mar", "apr", "may", "jun",
              "jul", "aug", "sep", "oct", "nov", "dec", "january",
              "february", "march", "april", "may", "june", "july",
              "august", "september", "october", "november",
              "december")
    <month_filter> := %<modulo>[+<shift>]
      <modulo>
        type: integer
      <shift>
        type: integer

Examples

Never schedule

Either , or @0
Always schedule

*
Schedule every 60 minutes

@60
Schedule at first occasion after 9am

09:00
Schedule every hour between midnight and 6am, every day

00:00-06:00@60
Schedule once between midnight and 2am, every day

00:00-02:00
Schedule once between midnight and 2am every last day of month

00:00-02:00@121 *:last or 00:00-02:00@121 *:-1
Schedule once between midnight and 2am every last friday of month

00:00-02:00@121 fri:last or 00:00-02:00@121 fri:-1
Schedule once between midnight and 2am every week day

00:00-02:00@121 mon-fri
Schedule once between midnight and 2am every week day from january to february

00:00-02:00@121 mon-fri * jan-feb
Schedule once between midnight and 2am every odd day (1, 3, 5)

00:00-02:00@121 *:%2+1
Schedule once between midnight and 2am every monday of even weeks

00:00-02:00@121 mon %2

Rosetta Stone

Release Notes

Changelog

arbitrator

Minimal configlet:

[arbitrator]
uri = http://www.opensvc.com

Minimal setup command:

om node set --kw="uri=http://www.opensvc.com"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Set to true to disable the arbitrator SSL certificate verification on the https uri.

This should only be enabled for testing.

uri

required:    true
scopable:    false

Example:

uri = http://www.opensvc.com

The arbitrator uri used by cluster node to ask for a vote when the cluster is split.

When the uri scheme is http or https, the vote checker is based on a GET request, else it is based on a TCP connect.

For backward compatibility, when the port is not specified in a TCP connect uri, the 1214 port is implied.

Arbitrators are tried in sequence, each reachable arbitrator gives a vote.

In case of a real split, all arbitrators are expected to be unreachable from the lost segment. At least one of them is expected to be reachable from the surviving segment.

Arbitrators of a cluster must thus be located close enough to each other, so a subset of arbitrators can't be reachable from a split cluster segment, while another subset of arbitrators is reachable from the other split cluster segment.

But not close enough so they can all fail together. Usually, this can be interpreted as: same site, not same rack and power lines.

Arbitrators are verified every 60s to alert admins of the arbitrator failures.

array.centera

Minimal configlet:

[array#1]
type = centera
java_bin = /opt/java/bin/java
jcass_dir = /opt/centera/LIB
password = system/sec/array1
server = centera1
username = root

Minimal setup command:

om node set \
	--kw="type=centera" \
	--kw="java_bin=/opt/java/bin/java" \
	--kw="jcass_dir=/opt/centera/LIB" \
	--kw="password=system/sec/array1" \
	--kw="server=centera1" \
	--kw="username=root"

java_bin

required:    true
scopable:    false

Example:

java_bin = /opt/java/bin/java

The path to the java executable to use to run the Centera management program.

jcass_dir

required:    true
scopable:    false

Example:

jcass_dir = /opt/centera/LIB

The path of the directory hosting the JCASScript.jar.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

server

required:    true
scopable:    false

Example:

server = centera1

The storage server to connect.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.dorado

Minimal configlet:

[array#1]
type = dorado
api = https://array.opensvc.com/api/v1.0
password = system/sec/array1
username = root

Minimal setup command:

om node set \
	--kw="type=dorado" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

name

required:    false
scopable:    false

Example:

name = a09

The name of the array. If not provided, fallback to the section name suffix.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

timeout

required:    false
scopable:    false
default:     120s
convert:     duration

Example:

timeout = 10s

The api request timeout.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.emcvnx

Minimal configlet:

[array#1]
type = emcvnx
spa = array1-a
spb = array1-b

Minimal setup command:

om node set \
	--kw="type=emcvnx" \
	--kw="spa=array1-a" \
	--kw="spb=array1-b"

method

required:    false
scopable:    false
candidates:  secfile, credentials
default:     secfile

Example:

method = secfile

The authentication method to use.

password

required:    false
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, if configured, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

scope

required:    false
scopable:    false
default:     0

Example:

scope = 1

The VNC scope to work in.

spa

required:    true
scopable:    false

Example:

spa = array1-a

The name of the Service Processor A.

spb

required:    true
scopable:    false

Example:

spb = array1-b

The name of the Service Processor B.

username

required:    false
scopable:    false

Example:

username = root

The username to use to log in, if configured.

array.eva

Minimal configlet:

[array#1]
type = eva
manager = evamanager.mycorp
password = system/sec/array1
username = root

Minimal setup command:

om node set \
	--kw="type=eva" \
	--kw="manager=evamanager.mycorp" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

bin

required:    false
scopable:    false

Example:

bin = /opt/sssu/bin/sssu

The EVA manager executable to use.

manager

required:    true
scopable:    false

Example:

manager = evamanager.mycorp

The EVA manager to connect.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.freenas

Minimal configlet:

[array#1]
type = freenas
api = https://array.opensvc.com/api/v1.0
password = system/sec/array1
username = root

Minimal setup command:

om node set \
	--kw="type=freenas" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

schedule

required:    false
scopable:    false

Schedule parameter for the pusharray node action.

See usr/share/doc/schedule for the schedule syntax.

timeout

required:    false
scopable:    false
default:     120s
convert:     duration

Example:

timeout = 10s

The api request timeout.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.hds

Minimal configlet:

[array#1]
type = hds
password = system/sec/array1
url = https://hdsmanager/
username = root

Minimal setup command:

om node set \
	--kw="type=hds" \
	--kw="password=system/sec/array1" \
	--kw="url=https://hdsmanager/" \
	--kw="username=root"

bin

required:    false
scopable:    false

Example:

bin = /opt/hds/bin/HiCommandCLI

The HDS manager executable to use.

jre_path

required:    false
scopable:    false

Example:

jre_path = /opt/java

The path hosting the java installation to use to execute the HiCommandCLI.

name

required:    false
scopable:    false

Example:

name = HUSVM.1234

The name of the array. If not provided, fallback to the section name suffix.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

url

required:    true
scopable:    false

Example:

url = https://hdsmanager/

The url passed to HiCommandCli, pointing the manager in charge of the array.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.hoc

Minimal configlet:

[array#1]
type = hoc
api = https://array.opensvc.com/api/v1.0
model = VSP G350
password = system/sec/array1
username = root

Minimal setup command:

om node set \
	--kw="type=hoc" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="model=VSP G350" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

delay

required:    false
scopable:    false
default:     10s
convert:     duration

The delay between request attempts on retryable errors.

http_proxy

required:    false
scopable:    false

Example:

http_proxy = http://proxy.mycorp:3158

The proxy server to use for http requests to the api.

https_proxy

required:    false
scopable:    false

Example:

https_proxy = https://proxy.mycorp:3158

The proxy server to use for https requests to the api.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Example:

insecure = true

Disable secure socket verification.

model

required:    true
scopable:    false
candidates:  VSP G370, VSP G700, VSP G900, VSP F370, VSP F700, VSP F900, VSP G350, VSP F350, VSP G800, VSP F800, VSP G400, VSP G600, VSP F400, VSP F600, VSP G200, VSP G1000, VSP G1500, VSP F1500, Virtual Storage Platform, HUS VM

Example:

model = VSP G350

The array model.

name

required:    false
scopable:    false

Example:

name = a09

The name of the array. If not provided, fallback to the section name suffix.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

retry

required:    false
scopable:    false
default:     30
convert:     int

The number of request attempts on retryable errors.

schedule

required:    false
scopable:    false

Schedule parameter for the pusharray node action.

See usr/share/doc/schedule for the schedule syntax.

timeout

required:    false
scopable:    false
default:     120s
convert:     duration

Example:

timeout = 10s

The api request timeout.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

wwid_prefix

required:    false
scopable:    false

Hitachi APIs do not report the disks NAA wwids, but it can be forged from a array-specifix prefix concatenated with the LDev id. This keyword allow the cluster admin to define this prefix. Do not include the NAA Type digit prefix (define 62400000ec12ac73541d instead of 362400000ec12ac73541d).

array.hp3par

Minimal configlet:

[array#1]
type = hp3par

Minimal setup command:

om node set --kw="type=hp3par"

cli

required:    false
scopable:    false
default:     3parcli

Example:

cli = /path/to/pwf

The path of the executable hp3par CLI.

key

required:    false
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

manager

required:    false
scopable:    false
default:     The name of the array

Example:

manager = mymanager.mycorp

The array manager host name.

method

required:    false
scopable:    false
candidates:  proxy, cli, ssh
default:     ssh

Example:

method = ssh

The connection method to use.

pwf

required:    false
scopable:    false

Example:

pwf = /path/to/pwf

The path to the 3par password file to use to log in.

username

required:    false
scopable:    false

Example:

username = root

The username to use to log in, if configured.

array.ibmds

Minimal configlet:

[array#1]
type = ibmds
hmc1 = hmc1.mycorp
hmc2 = hmc2.mycorp
username = root

Minimal setup command:

om node set \
	--kw="type=ibmds" \
	--kw="hmc1=hmc1.mycorp" \
	--kw="hmc2=hmc2.mycorp" \
	--kw="username=root"

hmc1

required:    true
scopable:    false

Example:

hmc1 = hmc1.mycorp

The host name of the primary HMC.

hmc2

required:    true
scopable:    false

Example:

hmc2 = hmc2.mycorp

The host name of the secondary HMC.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.ibmsvc

Minimal configlet:

[array#1]
type = ibmsvc
key = /path/to/key
username = root

Minimal setup command:

om node set \
	--kw="type=ibmsvc" \
	--kw="key=/path/to/key" \
	--kw="username=root"

key

required:    true
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.netapp

Minimal configlet:

[array#1]
type = netapp
key = /path/to/key
server = centera1
username = root

Minimal setup command:

om node set \
	--kw="type=netapp" \
	--kw="key=/path/to/key" \
	--kw="server=centera1" \
	--kw="username=root"

key

required:    true
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

server

required:    true
scopable:    false

Example:

server = centera1

The storage server to connect.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.nexenta

Minimal configlet:

[array#1]
type = nexenta
password = system/sec/array1
username = root

Minimal setup command:

om node set \
	--kw="type=nexenta" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

port

required:    false
scopable:    false
default:     2000
convert:     int

Example:

port = 2000

The nexenta administration listener port.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.pure

Minimal configlet:

[array#1]
type = pure
api = https://array.opensvc.com/api/v1.0
client_id = bd2c75d0-f0d5-11ee-a362-8b0f2d1b83d7
issuer = opensvc
key_id = df80ae3a-f0d5-11ee-94c9-b7c8d2f57c4f
secret = system/sec/array1
username = opensvc

Minimal setup command:

om node set \
	--kw="type=pure" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="client_id=bd2c75d0-f0d5-11ee-a362-8b0f2d1b83d7" \
	--kw="issuer=opensvc" \
	--kw="key_id=df80ae3a-f0d5-11ee-94c9-b7c8d2f57c4f" \
	--kw="secret=system/sec/array1" \
	--kw="username=opensvc"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

client_id

required:    true
scopable:    false

Example:

client_id = bd2c75d0-f0d5-11ee-a362-8b0f2d1b83d7

The client id to use as the aud key in the payload of the login jwt.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Example:

insecure = true

Disable secure socket verification.

issuer

required:    true
scopable:    false

Example:

issuer = opensvc

The issuer to use as the iss key in the payload of the login jwt token.

key_id

required:    true
scopable:    false

Example:

key_id = df80ae3a-f0d5-11ee-94c9-b7c8d2f57c4f

The key id to use as the kid key in the header of the login jwt.

schedule

required:    false
scopable:    false

Schedule parameter for the pusharray node action.

See usr/share/doc/schedule for the schedule syntax.

secret

required:    true
scopable:    false

Example:

secret = system/sec/array1

The secret to use to store the information required to create the login jwt, expressed as a reference to a secret. The secret must be in the system namespace and must have the following keys: private_key.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    true
scopable:    false

Example:

username = opensvc

The username to use as the sub key in the payload of the login jwt.

array.symmetrix

Minimal configlet:

[array#1]
type = symmetrix

Minimal setup command:

om node set --kw="type=symmetrix"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

name

required:    false
scopable:    false

Example:

name = 00012345

The name of the array. If not provided, fallback to the section name suffix.

password

required:    false
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, if configured, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

schedule

required:    false
scopable:    false

Schedule parameter for the pusharray node action.

See usr/share/doc/schedule for the schedule syntax.

symcli_connect

required:    false
scopable:    false

Example:

symcli_connect = MY_SYMAPI_SERVER

Set the SYMCLI_CONNECT environment variable to this value.

If not set, the SCSI communication channels are used.

The value set must be declared in the /var/symapi/config/netcnfg file.

symcli_path

required:    false
scopable:    false
default:     /usr/symcli

Example:

symcli_path = /opt/symcli

Force use of a symcli programs installation, pointing the path of its head directory.

For the case multiple symcli versions are installed and the default selector does not select the version preferred for the array.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    false
scopable:    false

Example:

username = root

The username to use to log in, if configured.

array.truenas

Minimal configlet:

[array#1]
type = truenas

Minimal setup command:

om node set --kw="type=truenas"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false

Schedule parameter for the pusharray node action.

See usr/share/doc/schedule for the schedule syntax.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

array.vioserver

Minimal configlet:

[array#1]
type = vioserver
key = /path/to/key
username = root

Minimal setup command:

om node set \
	--kw="type=vioserver" \
	--kw="key=/path/to/key" \
	--kw="username=root"

key

required:    true
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.xtremio

Minimal configlet:

[array#1]
type = xtremio
api = https://array.opensvc.com/api/v1.0
password = system/sec/array1
username = root

Minimal setup command:

om node set \
	--kw="type=xtremio" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

name

required:    false
scopable:    false

Example:

name = array1

The name of the array. If not provided, fallback to the section name suffix.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

asset

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushasset node action.

See usr/share/doc/schedule for the schedule syntax.

checks

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushchecks node action.

See usr/share/doc/schedule for the schedule syntax.

cluster

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

ca

required:    false
scopable:    false
default:     `system/sec/ca`

convert:     list

A whitespace-separated list of sec paths.

The listener accepts a x509 client certificate if it is trusted by any CA certificate found in these sec objects.

cert

required:    false
scopable:    false
default:     `system/sec/cert`

The path of the secret hosting the certificate that the listener use for its TLS socket.

dns

required:    false
scopable:    true
convert:     list

The list of nodes to set as nameserver in the resolv.conf of the containers the CRM starts.

If set, the search will also be set to:

1/ <name>.<namespace>.svc.<clustername> 2/ <namespace>.svc.<clustername> 3/ <clustername>.

drpnodes

required:    false
scopable:    false
convert:     list

This list is fetched from the join command payload received from the joined node.

The service configuration {clusterdrpnodes} is resolved to this keyword value.

envs

required:    false
scopable:    false
default:     CERT DEV DRP FOR INT PRA PRD PRJ PPRD QUAL REC STG TMP TST UAT
convert:     list

TODO

id

required:    false
scopable:    true
default:     An autogenerated random UUID.

This unique identifier is auto-generated on install and should never be change by the cluster administrators.

It is changed when the node joins a cluster, so the remote cluster id replaces the joiners' cluster id.

name

required:    false
scopable:    false
default:     A random generated clustername.

The cluster name is used,

as the zone name in the cluster dns records
in the {fqdn} configuration reference
in the AES secret encryption metadata

The cluster name should be unique site-wide. Missing cluster name will be automatically created with random value during daemon startup.

It is always lowercased, so better to set it to a lowercase value to avoid confusion.

The cluster name is provided to joining nodes, so they can replace their own.

nodes

required:    false
scopable:    false
convert:     list

This list of node names contains only the local node name on install.

When the node joins a cluster, the joined node provides the new list, with the new node added. The joiner then replace its nodes list with the one received.

When a node receives a join request, it adds the new node to its cluster nodes list, then provide the new list to the joiner.

quorum

required:    false
scopable:    false
default:     false
convert:     bool

If true, when the cluster is split a vote happens on each cluster node.

Each reachable node and each reachable arbitrator give their vote. If the votes is less than half the total number of nodes plus arbitrators, the node trigger a node fencing method defined by node.split_action (crash, reboot or disabled).

secret

required:    false
scopable:    true
default:     A random string autogenerated on first use

The cluster shared secret used to encrypt and decrypt heartbeat payloads and sec values, with AES256

This secret is auto-generated on install, then merged from the joined nodes when joining a cluster.

The cluster name should be unique site-wide and be set right before starting to add sec keys.

vip

required:    false
scopable:    true

Example:

vip = 192.168.99.12/24@eth0

The cluster virtual ip.

If configured, the daemon creates a system/svc/vip HA failover service to manage this ip.

cni

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

config

required:    false
scopable:    false
default:     /opt/cni/net.d

Example:

config = /var/lib/opensvc/cni/net.d

The directory hosting the CNI network configuration files.

plugins

required:    false
scopable:    false
default:     /opt/cni/bin

Example:

plugins = /var/lib/opensvc/cni/bin

The directory hosting the CNI plugins.

compliance

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

auto_update

required:    false
scopable:    false
default:     false
convert:     bool

If set to true, execute om node updatecomp upon every scheduler-executed om node compliance check.

These updates keep the compliance modules in sync with the reference repository.

Warning: the module repository security is critical. Attackers could insert malicious code in served modules.

schedule

required:    false
scopable:    false
default:     02:00-06:00

Schedule parameter for the compliance auto node action, which check all attached modules and fix only those flagged autofix.

See usr/share/doc/schedule for the schedule syntax.

dequeue_actions

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false

Schedule parameter for the dequeue actions node action.

See usr/share/doc/schedule for the schedule syntax.

disks

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushdisks node action.

See usr/share/doc/schedule for the schedule syntax.

hb.disk

Minimal configlet:

[hb#1]
type = disk
dev = /dev/mapper/36589cfc000000e03957c51dabab8373a

Minimal setup command:

om node set \
	--kw="type=disk" \
	--kw="dev=/dev/mapper/36589cfc000000e03957c51dabab8373a"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

dev

required:    true
scopable:    true

Example:

dev = /dev/mapper/36589cfc000000e03957c51dabab8373a

The device to write the heartbeats to and read from.

It must be,

Dedicated to the daemon use.
Sized 1MB for metadata + 1MB/node.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

The actual interval is not fixed, because the daemon tries to send the message as soon as it has something to notify. A minimum interval protects the node from saturating the network and cpu with the daemon synchronization workload.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

hb.multicast

Minimal configlet:

[hb#1]
type = multicast

Minimal setup command:

om node set --kw="type=multicast"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

intf

required:    false
scopable:    true
default:     The natural interface for `<addr>`

Example:

intf = eth0

The interface to bind.

port

required:    false
scopable:    true
default:     10000
convert:     int

The port for each node to send to or listen on.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

hb.relay

Minimal configlet:

[hb#1]
type = relay
relay = relaynode1

Minimal setup command:

om node set \
	--kw="type=relay" \
	--kw="relay=relaynode1"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Set to true to disable the relay SSL certificate verification.

This should only be enabled for testing.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

password

required:    false
scopable:    false
default:     system/sec/relay

The name of a sec object containing a password key, which value is used as password for log in the relay api.

relay

required:    true
scopable:    false

Example:

relay = relaynode1

The relay resolvable node name.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

username

required:    false
scopable:    false
default:     relay

The username for login the relay api.

hb.unicast

Minimal configlet:

[hb#1]
type = unicast

Minimal setup command:

om node set --kw="type=unicast"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

intf

required:    false
scopable:    true
default:     The natural interface for `<addr>`

Example:

intf = eth0

The interface to bind.

nodes

required:    false
scopable:    true
default:     All nodes.

convert:     list

The nodes participating to the heartbeat.

This keyword can be used to setup a partial redundancy like:

       n1   n2   n3   n4
hb#1   O    O    O    O
hb#2   O    O          
hb#1             O    O

Which can be relevant if n[12] are in the same bladecenter b1, and n[34] are in the same bladecenter b2. The bladecenter having an internal network completely hardware independent of the network used to go outside of the bladecenters.

port

required:    false
scopable:    true
default:     10000
convert:     int

The port for each node to send to or listen on.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

hook

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

command

required:    false
scopable:    false
convert:     shlex

The command to execute on selected events.

The program is fed the json-formatted event data through stdin.

events

required:    false
scopable:    false
convert:     list

The list of events to execute the hook command on.

The special value all is also supported.

listener

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

crl

required:    false
scopable:    false
default:     /var/lib/opensvc/certs/ca_crl

Example:

crl = https://crl.opensvc.com

The URL serving the certificate revocation list.

The default points to the path of the cluster CA CRL in {var}/certs/ca_crl.

dns_sock_gid

required:    false
scopable:    false
default:     953

The gid owning the unix socket serving the remote backend to the pdns authoritative server.

dns_sock_uid

required:    false
scopable:    false
default:     953

The uid owning the unix socket serving the remote backend to the pdns authoritative server.

openid_well_known

required:    false
scopable:    false

Example:

openid_well_known = https://keycloak.opensvc.com/auth/realms/clusters/.well-known/openid-configuration

The URL serving the well-known configuration of an openid provider.

If set, the http listener will try to validate the Bearer token provided in the requests headers.

If the token is valid,

the user name is fetched from the preferred_username claim (fallback on name)
the user grant list is obtained by joining the multiple grant claims.

port

required:    false
scopable:    true
default:     1215
convert:     int

The port the daemon tls listener must listen on.

In pull action mode, the collector post request to notify there are actions to unqueue. The opensvc daemon executes the dequeue actions node action upon receive.

The listener.port value is sent to the collector on pushasset.

network.bridge

Minimal configlet:

[network#1]
type = bridge

Minimal setup command:

om node set --kw="type=bridge"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

network

required:    false
scopable:    false

The cluster backend network.

The routed_bridge driver fragments this network into ips_per_nodes blocks subnets.

type

required:    false
scopable:    false
candidates:  bridge, routed_bridge
default:     bridge

The type of network.

network.lo

Minimal configlet:

[network#1]
type = lo

Minimal setup command:

om node set --kw="type=lo"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

type

required:    false
scopable:    false
candidates:  bridge, routed_bridge
default:     bridge

The type of network.

network.routed_bridge

Minimal configlet:

[network#1]
type = routed_bridge

Minimal setup command:

om node set --kw="type=routed_bridge"

addr

required:    false
scopable:    true
default:     Detect using a name resolution of `<nodename>`.

Beware, if the nodename resolves to 127.0.1.1 or 127.0.0.1 the ipip tunnel can not work.

The ip address used as local endpoint for the ipip tunnel configured by the network setup command to access the backend subnet of peer nodes not reachable on the same subnet.

gateway

required:    false
scopable:    true

The gateway to use to reach the network segment of the node specified as scope.

ips_per_node

required:    false
scopable:    false
default:     1024
convert:     int

The number of allocatable ips per node on the network.

Converted to the closest power of two.

network

required:    false
scopable:    false

The cluster backend network.

The routed_bridge driver fragments this network into ips_per_nodes blocks subnets.

subnet

required:    false
scopable:    true

The cidr subnet handled by this node.

This parameter must be scoped for each node.

Usually, the subnets are allocated automatically upon initial network setup, each node being attributed a subnet based on its index in the cluster.nodes list.

tables

required:    false
scopable:    false
default:     main
convert:     list

Example:

tables = main custom1 custom2

The list of routing tables to add the backend network routes to.

The list of available tables is in /etc/iproute2/rt_tables.

tunnel

required:    false
scopable:    false
candidates:  auto, always, never
default:     auto

Create and route traffic through tunnels to peer nodes policy.

auto

Tunnel if the peer is not in the same subnet
always

Tunnel even if the peer seems to be in the same subnet. Some hosting providers require this as traffic goes through routers even between adjacent nodes.

tunnel_mode

required:    false
scopable:    false
candidates:  gre, ipip, ip6ip6
default:     ipip

The ip tunnel mode. gre can tunnel mcast ip and ipv6 at the price of a 24B header, ipip can only tunnel ipv4 but with a 20B header. Note, some OVH servers combinations don't support ipip but work with gre.

type

required:    false
scopable:    false
candidates:  bridge, routed_bridge
default:     bridge

The type of network.

node

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

allowed_networks

required:    false
scopable:    false
default:     10.0.0.0/8 172.16.0.0/24 192.168.0.0/16
convert:     list

The list of cidr blocks the agents allows creation of backend network into.

Should be restricted to match your site constraints.

asset_env

required:    false
scopable:    false

Example:

asset_env = Production

An asset information to push to the collector on pushasset, overriding the currently stored value.

bios_version

required:    false
scopable:    false

Example:

bios_version = 1.025

Override for the corresponding pushasset discovery probe.

branch

required:    false
scopable:    false

Example:

branch = 1.9

Set the targeted opensvc agent branch.

The downloaded upgrades will honor that branch.

If not set, the repopkg imposes the target branch via the current link.

It is recommended to set branch when repopkg points to a repository you are not responsible for.

connect_to

required:    false
scopable:    false

Example:

connect_to = 1.2.3.4

An asset information pushed to the collector on pushasset.

If not set, the collector picks one of the node ip addresses inventoried on pushasset too.

On GCE instances, defaults to the instance ip address.

cpu_cores

required:    false
scopable:    false
convert:     int

Example:

cpu_cores = 2

Override for the corresponding pushasset discovery probe.

cpu_dies

required:    false
scopable:    false
convert:     int

Example:

cpu_dies = 1

Override for the corresponding pushasset discovery probe.

cpu_freq

required:    false
scopable:    false

Example:

cpu_freq = 3.2 Ghz

Override for the corresponding pushasset discovery probe.

cpu_model

required:    false
scopable:    false

Example:

cpu_model = Alpha EV5

Override for the corresponding pushasset discovery probe.

cpu_threads

required:    false
scopable:    false
convert:     int

Example:

cpu_threads = 4

Override for the corresponding pushasset discovery probe.

dbcompliance

required:    false
scopable:    false
default:     Same protocol, server and port as `dbopensvc`, but with a different path.

Example:

dbcompliance = https://collector.opensvc.com

Set the uri of the collector's main rpc server.

The path part of the uri can be left unspecified.

dbinsecure

required:    false
scopable:    false
convert:     bool

Set to true to disable the collector x509 certificate verification.

This should only be used for testing.

dblog

required:    false
scopable:    false
default:     true
convert:     bool

If true and dbopensvc is set, the objects action logs are reported to the collector.

Set to false to disable log reporting to the collector, even if dbopensvc is set.

dbopensvc

required:    false
scopable:    false

Example:

dbopensvc = https://collector.opensvc.com

Set the uri of the collector's feed rpc server.

The path part of the uri can be left unspecified.

If dbopensvc is not set, the agent does not try to communicate with a collector.

enclosure

required:    false
scopable:    false

Example:

enclosure = 1

Override for the corresponding pushasset discovery probe.

env

required:    false
scopable:    false
default:     TST

A code like PRD, DEV, etc... the agent can use to enforce data protection policies:

A non-PRD object instance can not be started on a PRD node
A PRD object instance can be started on a non-PRD node (typically in a DRP situation)

loc_addr

required:    false
scopable:    false

Example:

loc_addr = 7 rue blanche

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_building

required:    false
scopable:    false

Example:

loc_building = Crystal

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_city

required:    false
scopable:    false

Example:

loc_city = Paris

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_country

required:    false
scopable:    false

Example:

loc_country = fr

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_floor

required:    false
scopable:    false

Example:

loc_floor = 21

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_rack

required:    false
scopable:    false

Example:

loc_rack = R42

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_room

required:    false
scopable:    false

Example:

loc_room = 102

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_zip

required:    false
scopable:    false

Example:

loc_zip = 75017

An asset information to push to the collector on pushasset, overriding the currently stored value.

maintenance_grace_period

required:    false
scopable:    false
default:     60
convert:     duration

A duration expression, like 1m30s, defining how long the daemon keeps remote node data while it is known to be in maintenance.

The maintenance state is announced to peers at the beginning of a daemon stop and daemon restart, but not on daemon shutdown.

As long as the remote node data is kept, the local daemon won't takeover the instances running on the node in maintenance.

This parameter should be adjusted to span the daemon restart time.

manufacturer

required:    false
scopable:    false

Example:

manufacturer = Digital

Override for the corresponding pushasset discovery probe.

max_parallel

required:    false
scopable:    false
default:     10
convert:     int

Allow a maximum of max_parallel CRM commands to run simultaneously.

Applies to both:

om <selector> <action> commands.
commands executed by the daemon for orchestrations

mem_banks

required:    false
scopable:    false
convert:     int

Example:

mem_banks = 4

Override for the corresponding pushasset discovery probe.

mem_bytes

required:    false
scopable:    false
convert:     size

Example:

mem_bytes = 256mb

Override for the corresponding pushasset discovery probe.

mem_slots

required:    false
scopable:    false
convert:     int

Example:

mem_slots = 4

Override for the corresponding pushasset discovery probe.

min_avail_mem

required:    false
scopable:    false
default:     2%
convert:     size

The minimum required available memory to allow orchestration.

min_avail_swap

required:    false
scopable:    false
default:     10%
convert:     size

The minimum required available swap to allow orchestration.

model

required:    false
scopable:    false

Example:

model = ds20e

Override for the corresponding pushasset discovery probe.

oci

required:    false
scopable:    false

The default micro-container driver.

If not set, prefer podman if installed, else fallback to docker.

os_arch

required:    false
scopable:    false

Example:

os_arch = 5.1234

Override for the corresponding pushasset discovery probe.

os_kernel

required:    false
scopable:    false

Example:

os_kernel = 5.1234

Override for the corresponding pushasset discovery probe.

os_release

required:    false
scopable:    false

Example:

os_release = 5

Override for the corresponding pushasset discovery probe.

os_vendor

required:    false
scopable:    false

Example:

os_vendor = Digital

Override for the corresponding pushasset discovery probe.

prkey

required:    false
scopable:    false
default:     Autogenerated on first use.

The scsi3 persistent reservation key used by the pr resources.

ready_period

required:    false
scopable:    false
default:     5s
convert:     duration

A duration expression, like 10s, defining how long the daemon waits before starting a service instance in ready state.

A peer node can preempt the start during this period.

Usually set to allow at least a couple of heartbeats to be received.

rejoin_grace_period

required:    false
scopable:    false
default:     90s
convert:     duration

A duration expression, like 1m30s, defining how long a starting daemon waits in rejoin state.

The daemon normally exits the rejoin state when it has received a heartbeat from all its peer nodes.

During this phase, the orchestration is not allowed, to give a chance to place the services optimally when multiple daemon were restarted at the same time.

But if a peer stays down, the other daemons have to stop waiting at some point to let the service start, even if not on their natural placement leader.

This should be adjusted to:

2s + <longest reboot duration>

The worse case of multiple nodes reboot is when the longest reboot node is rebooted near the end of the reboot of the second longest rebooting node.

|==========>
    n1 reboot
                |--------------------|
                n1 rejoin_grace_period
                |================>
                n1 in rejoin state
                                  |=====================
                                  n1 in idle state
              |==================>
              n2 reboot
                                  |--------------------|
                                  n2 rejoin_grace_period
                                  |=====================
                                  n2 in idle state

As a consequence, to minimize the rejoin_grace_period, prefer fast boot nodes.

repo

required:    false
scopable:    false

Example:

repo = http://opensvc.repo.corp

Set the uri of the opensvc agent package repository and compliance modules gzipped tarball repository.

This parameter is used by the om node updatepkg and om node updatecomp commands.

Expected repository structure:

repocomp

required:    false
scopable:    false

Example:

repocomp = http://compliance.repo.corp

Set the uri of the opensvc compliance modules repository.

A gzipped tarball is expected to be found there by the om node updatecomp command.

Expected repository structure:

ROOT
+- compliance-100.tar.gz
+- compliance-101.tar.gz
`- current -> compliance-101.tar.gz

repopkg

required:    false
scopable:    false

Example:

repopkg = http://repo.opensvc.com

Set the uri of the opensvc agent package repository.

This parameter is used by the om node updatepkg command.

Expected repository structure:

ruser

required:    false
scopable:    false
default:     root

Example:

ruser = root opensvc@node1

Set the remote user to use to login to a remote node with ssh and rsync.

The remote user must have the privileges to run as root the following commands on the remote node:

om
rsync

The default ruser is root for all nodes.

ruser accepts a list of user[@node]. If @node is omitted, user is considered the new default user.

sec_zone

required:    false
scopable:    false

Example:

sec_zone = dmz1

An asset information to push to the collector on pushasset, overriding the currently stored value.

secure_fetch

required:    false
scopable:    false
default:     true
convert:     bool

If set to false, disable ssl authentication checks on all uri fetches.

serial

required:    false
scopable:    false

Example:

serial = abcdef0123456

Override for the corresponding pushasset discovery probe.

sp_version

required:    false
scopable:    false

Example:

sp_version = 1.026

Override for the corresponding pushasset discovery probe.

split_action

required:    false
scopable:    true
candidates:  crash, reboot, disabled
default:     crash

The node suicide method to use when a cluster split occurs and the node does not have the quorum.

This opting-out is meant to avoid double-start situations when the cluster is split.

Possible values are:

crash

Default.
reboot

May be preferred when the node power-on is not easy. No remote access via IPMI or equivalent for example.
disabled

May be used for test or training only (it does nothing).

sshkey

required:    false
scopable:    false
default:     opensvc

The basename of the ssh public key served by the GET /node/name/:nodename/ssh/key. For example, the opensvc default value serves ~/.ssh/opensvc.pub.

team_integ

required:    false
scopable:    false

Example:

team_integ = TINT

An asset information to push to the collector on pushasset, overriding the currently stored value.

team_support

required:    false
scopable:    false

Example:

team_support = TSUP

An asset information to push to the collector on pushasset, overriding the currently stored value.

tz

required:    false
scopable:    false

Example:

tz = +0200

Override for the corresponding pushasset discovery probe.

uuid

required:    false
scopable:    false

The authentication token provided by the collector on om node register.

packages

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushpkg node action.

See usr/share/doc/schedule for the schedule syntax.

patches

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushpatch node action.

See usr/share/doc/schedule for the schedule syntax.

pool.directory

Minimal configlet:

[pool#1]
type = directory

Minimal setup command:

om node set --kw="type=directory"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

path

required:    false
scopable:    false
default:     {var}/pool/directory

The fullpath of the directory hosting the pool volumes directories or loop files.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.dorado

Minimal configlet:

[pool#1]
type = dorado
array = 
diskgroup =

Minimal setup command:

om node set \
	--kw="type=dorado" \
	--kw="array=" \
	--kw="diskgroup="

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

compression

required:    false
scopable:    false
default:     false
convert:     bool

Activate compression on created luns.

dedup

required:    false
scopable:    false
default:     false
convert:     bool

Activate data deduplcation on created luns.

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

hypermetrodomain

required:    false
scopable:    false

Example:

hypermetrodomain = HyperMetroDomain_000

Create LUN as HyperMetro replicated pairs, using this domain.

pool.drbd

Minimal configlet:

[pool#1]
type = drbd

Minimal setup command:

om node set --kw="type=drbd"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

path

required:    false
scopable:    false

The fullpath of the directory hosting the pool volumes loop files.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

vg

required:    false
scopable:    false

The name of the volume group to allocate the pool volumes logical volumes into.

zpool

required:    false
scopable:    false

The name of the zpool to allocate the pool volumes zvol into.

pool.freenas

Minimal configlet:

[pool#1]
type = freenas
array = 
diskgroup =

Minimal setup command:

om node set \
	--kw="type=freenas" \
	--kw="array=" \
	--kw="diskgroup="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

blocksize

required:    false
scopable:    false
default:     512
convert:     size

Allow initiators to xcopy without authenticating to foreign targets.

compression

required:    false
scopable:    false
candidates:  inherit, none, lz4, gzip-1, gzip-2, gzip-3, gzip-4, gzip-5, gzip-6, gzip-7, gzip-8, gzip-9, zle, lzjb
default:     inherit

Compression level.

dedup

required:    false
scopable:    false
default:     off

Activate data deduplication on created dataset and zvol. Example values: on, off, verify

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

insecure_tpc

required:    false
scopable:    false
default:     false
convert:     bool

Allow initiators to xcopy without authenticating to foreign targets.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

sparse

required:    false
scopable:    false
default:     false
convert:     bool

Create zvol in sparse mode.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.hoc

Minimal configlet:

[pool#1]
type = hoc
array = 
diskgroup =

Minimal setup command:

om node set \
	--kw="type=hoc" \
	--kw="array=" \
	--kw="diskgroup="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

compression

required:    false
scopable:    false
default:     false
convert:     bool

Activate compression on created luns.

dedup

required:    false
scopable:    false
default:     false
convert:     bool

Activate data deduplcation on created luns.

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

label_prefix

required:    false
scopable:    false

The prefix to add to the label assigned to the created disks.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

pool_id

required:    false
scopable:    false

The Hitachi Ops Center storage machine pool name. Volumes are created in this storage pool.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

volume_id_range_from

required:    false
scopable:    false

The start of the range of ldev ids to allocate from.

volume_id_range_to

required:    false
scopable:    false

The end of the range of ldev ids to allocate from.

vsm_id

required:    false
scopable:    false

The name of the virtual storage machine id to allocate volume into.

pool.loop

Minimal configlet:

[pool#1]
type = loop

Minimal setup command:

om node set --kw="type=loop"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

path

required:    false
scopable:    false
default:     {var}/pool/loop

The path to create the pool loop files in.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.pure

Minimal configlet:

[pool#1]
type = pure
array = 
diskgroup =

Minimal setup command:

om node set \
	--kw="type=pure" \
	--kw="array=" \
	--kw="diskgroup="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

delete_now

required:    false
scopable:    false
default:     true
convert:     bool

If set to false the pure volumes are not immediately deleted on unprovision, so a following provision action could fail.

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

label_prefix

required:    false
scopable:    false

The prefix to add to the label assigned to the created disks.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

pod

required:    false
scopable:    false

The pod to create volume into.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

volumegroup

required:    false
scopable:    false

The volumegroup to create volume disks into.

pool.share

Minimal configlet:

[pool#1]
type = share

Minimal setup command:

om node set --kw="type=share"

path

required:    false
scopable:    false
default:     {var}/pool/share

The fullpath of the shared directory hosting the pool volumes directories or loop files.

pool.shm

Minimal configlet:

[pool#1]
type = shm

Minimal setup command:

om node set --kw="type=shm"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.symmetrix

Minimal configlet:

[pool#1]
type = symmetrix
array = 
srp =

Minimal setup command:

om node set \
	--kw="type=symmetrix" \
	--kw="array=" \
	--kw="srp="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

rdfg

required:    false
scopable:    false

Replication Group to use for SRDF.

slo

required:    false
scopable:    false

The name of the Service Level Agreement of the selected Storage Group.

srdf

required:    false
scopable:    false
default:     false
convert:     bool

Use SRDF replication.

srp

required:    true
scopable:    false

The name of the array resource pool to allocate volumes from.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.vg

Minimal configlet:

[pool#1]
type = vg
name =

Minimal setup command:

om node set \
	--kw="type=vg" \
	--kw="name="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

name

required:    true
scopable:    false

The name of the volume group to allocate the pool volumes logical volumes into.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.virtual

Minimal configlet:

[pool#1]
type = virtual

Minimal setup command:

om node set --kw="type=virtual"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

capabilities

required:    false
scopable:    false
default:     roo rwo rox rwx
convert:     list

The capabilities exposed by the virtual pool.

Supported capabilities:

shared
roo
rox
rwo
rwx
blk

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

optional_volume_env

required:    false
scopable:    false
convert:     list

Example:

optional_volume_env = container#1.name:container_name env.foo:foo

The list of the vol consumer service config keywords which values are mapped as env keys in the allocated volume service.

If the keyword is not set at the source, the default value in the template env section applies.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

template

required:    false
scopable:    false

Example:

template = templates/vol/mpool-over-loop

The path of a vol to use as a template for new volumes.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

volume_env

required:    false
scopable:    false
convert:     list

Example:

volume_env = container#1.name:container_name env.foo:foo

The list of the vol consumer service config keywords which values are mapped as env keys in the allocated volume service.

If the keyword is not set at the source, an error is raised.

pool.zpool

Minimal configlet:

[pool#1]
type = zpool
name =

Minimal setup command:

om node set \
	--kw="type=zpool" \
	--kw="name="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

name

required:    true
scopable:    false

The name of the zpool to allocate the pool volumes zvol or datasets into.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

switch.brocade

Minimal configlet:

[switch#1]
type = brocade
username = admin

Minimal setup command:

om node set \
	--kw="type=brocade" \
	--kw="username=admin"

key

required:    false
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in the switch.

method

required:    false
scopable:    false
candidates:  telnet, ssh
default:     ssh

Example:

method = ssh

The method to use to connect to the switch.

ssh Use key to provide a ssh key, or use the sshpass program.
telnet Set username and password with this method.

name

required:    false
scopable:    false

Example:

name = sansw1.my.corp

The name connect to the switch (dns name or ip address).

If not set, fallback to the section name suffix.

password

required:    false
scopable:    false

Example:

password = mysec/password

The password to use to log in, expressed as a sec name (not path).

The secret must be in the system namespace and must have a password key.

Either username or key must be specified.

username

required:    true
scopable:    false

Example:

username = admin

The username to use to log in the switch.

syslog

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

facility

required:    false
scopable:    false
default:     daemon

The syslog facility to log to.

host

required:    false
scopable:    false
default:     `localhost` if port is set.

The syslog server host to send logs to.

If neither host nor port are specified and if /dev/log exists, the messages are posted to /dev/log.

level

required:    false
scopable:    false
candidates:  critical, error, warning, info, debug
default:     info

The minimum message criticity to feed to syslog.

Setting to critical actually disables the syslog logging, as the agent does not emit messages at this level.

port

required:    false
scopable:    false
default:     514

The syslog server port to send logs to.

If neither host nor port are specified and if /dev/log exists, the messages are posted to /dev/log.

sysreport

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the sysreport node action, which collects into an archive all files and command outputs defined in /etc/opensvc/sysreport and sends that archive to the collector.

The collector stores the unpacked files in a per-node git repository.

See usr/share/doc/schedule for the schedule syntax.

DEFAULT

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

id

required:    false
scopable:    false
default:     A autogenerated random uuid

A RFC 4122 random uuid generated by the agent.

arbitrator

Minimal configlet:

[arbitrator]
uri = http://www.opensvc.com

Minimal setup command:

om test/ccfg/foo set --kw="uri=http://www.opensvc.com"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Set to true to disable the arbitrator SSL certificate verification on the https uri.

This should only be enabled for testing.

uri

required:    true
scopable:    false

Example:

uri = http://www.opensvc.com

The arbitrator uri used by cluster node to ask for a vote when the cluster is split.

When the uri scheme is http or https, the vote checker is based on a GET request, else it is based on a TCP connect.

For backward compatibility, when the port is not specified in a TCP connect uri, the 1214 port is implied.

Arbitrators are tried in sequence, each reachable arbitrator gives a vote.

In case of a real split, all arbitrators are expected to be unreachable from the lost segment. At least one of them is expected to be reachable from the surviving segment.

But not close enough so they can all fail together. Usually, this can be interpreted as: same site, not same rack and power lines.

Arbitrators are verified every 60s to alert admins of the arbitrator failures.

array.centera

Minimal configlet:

[array#1]
type = centera
java_bin = /opt/java/bin/java
jcass_dir = /opt/centera/LIB
password = system/sec/array1
server = centera1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=centera" \
	--kw="java_bin=/opt/java/bin/java" \
	--kw="jcass_dir=/opt/centera/LIB" \
	--kw="password=system/sec/array1" \
	--kw="server=centera1" \
	--kw="username=root"

java_bin

required:    true
scopable:    false

Example:

java_bin = /opt/java/bin/java

The path to the java executable to use to run the Centera management program.

jcass_dir

required:    true
scopable:    false

Example:

jcass_dir = /opt/centera/LIB

The path of the directory hosting the JCASScript.jar.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

server

required:    true
scopable:    false

Example:

server = centera1

The storage server to connect.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.dorado

Minimal configlet:

[array#1]
type = dorado
api = https://array.opensvc.com/api/v1.0
password = system/sec/array1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=dorado" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

name

required:    false
scopable:    false

Example:

name = a09

The name of the array. If not provided, fallback to the section name suffix.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

timeout

required:    false
scopable:    false
default:     120s
convert:     duration

Example:

timeout = 10s

The api request timeout.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.emcvnx

Minimal configlet:

[array#1]
type = emcvnx
spa = array1-a
spb = array1-b

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=emcvnx" \
	--kw="spa=array1-a" \
	--kw="spb=array1-b"

method

required:    false
scopable:    false
candidates:  secfile, credentials
default:     secfile

Example:

method = secfile

The authentication method to use.

password

required:    false
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, if configured, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

scope

required:    false
scopable:    false
default:     0

Example:

scope = 1

The VNC scope to work in.

spa

required:    true
scopable:    false

Example:

spa = array1-a

The name of the Service Processor A.

spb

required:    true
scopable:    false

Example:

spb = array1-b

The name of the Service Processor B.

username

required:    false
scopable:    false

Example:

username = root

The username to use to log in, if configured.

array.eva

Minimal configlet:

[array#1]
type = eva
manager = evamanager.mycorp
password = system/sec/array1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=eva" \
	--kw="manager=evamanager.mycorp" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

bin

required:    false
scopable:    false

Example:

bin = /opt/sssu/bin/sssu

The EVA manager executable to use.

manager

required:    true
scopable:    false

Example:

manager = evamanager.mycorp

The EVA manager to connect.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.freenas

Minimal configlet:

[array#1]
type = freenas
api = https://array.opensvc.com/api/v1.0
password = system/sec/array1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=freenas" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

timeout

required:    false
scopable:    false
default:     120s
convert:     duration

Example:

timeout = 10s

The api request timeout.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.hds

Minimal configlet:

[array#1]
type = hds
password = system/sec/array1
url = https://hdsmanager/
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=hds" \
	--kw="password=system/sec/array1" \
	--kw="url=https://hdsmanager/" \
	--kw="username=root"

bin

required:    false
scopable:    false

Example:

bin = /opt/hds/bin/HiCommandCLI

The HDS manager executable to use.

jre_path

required:    false
scopable:    false

Example:

jre_path = /opt/java

The path hosting the java installation to use to execute the HiCommandCLI.

name

required:    false
scopable:    false

Example:

name = HUSVM.1234

The name of the array. If not provided, fallback to the section name suffix.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

url

required:    true
scopable:    false

Example:

url = https://hdsmanager/

The url passed to HiCommandCli, pointing the manager in charge of the array.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.hoc

Minimal configlet:

[array#1]
type = hoc
api = https://array.opensvc.com/api/v1.0
model = VSP G350
password = system/sec/array1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=hoc" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="model=VSP G350" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

delay

required:    false
scopable:    false
default:     10s
convert:     duration

The delay between request attempts on retryable errors.

http_proxy

required:    false
scopable:    false

Example:

http_proxy = http://proxy.mycorp:3158

The proxy server to use for http requests to the api.

https_proxy

required:    false
scopable:    false

Example:

https_proxy = https://proxy.mycorp:3158

The proxy server to use for https requests to the api.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Example:

insecure = true

Disable secure socket verification.

model

required:    true
scopable:    false
candidates:  VSP G370, VSP G700, VSP G900, VSP F370, VSP F700, VSP F900, VSP G350, VSP F350, VSP G800, VSP F800, VSP G400, VSP G600, VSP F400, VSP F600, VSP G200, VSP G1000, VSP G1500, VSP F1500, Virtual Storage Platform, HUS VM

Example:

model = VSP G350

The array model.

name

required:    false
scopable:    false

Example:

name = a09

The name of the array. If not provided, fallback to the section name suffix.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

retry

required:    false
scopable:    false
default:     30
convert:     int

The number of request attempts on retryable errors.

timeout

required:    false
scopable:    false
default:     120s
convert:     duration

Example:

timeout = 10s

The api request timeout.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

wwid_prefix

required:    false
scopable:    false

array.hp3par

Minimal configlet:

[array#1]
type = hp3par

Minimal setup command:

om test/ccfg/foo set --kw="type=hp3par"

cli

required:    false
scopable:    false
default:     3parcli

Example:

cli = /path/to/pwf

The path of the executable hp3par CLI.

key

required:    false
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

manager

required:    false
scopable:    false
default:     The name of the array

Example:

manager = mymanager.mycorp

The array manager host name.

method

required:    false
scopable:    false
candidates:  proxy, cli, ssh
default:     ssh

Example:

method = ssh

The connection method to use.

pwf

required:    false
scopable:    false

Example:

pwf = /path/to/pwf

The path to the 3par password file to use to log in.

username

required:    false
scopable:    false

Example:

username = root

The username to use to log in, if configured.

array.ibmds

Minimal configlet:

[array#1]
type = ibmds
hmc1 = hmc1.mycorp
hmc2 = hmc2.mycorp
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=ibmds" \
	--kw="hmc1=hmc1.mycorp" \
	--kw="hmc2=hmc2.mycorp" \
	--kw="username=root"

hmc1

required:    true
scopable:    false

Example:

hmc1 = hmc1.mycorp

The host name of the primary HMC.

hmc2

required:    true
scopable:    false

Example:

hmc2 = hmc2.mycorp

The host name of the secondary HMC.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.ibmsvc

Minimal configlet:

[array#1]
type = ibmsvc
key = /path/to/key
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=ibmsvc" \
	--kw="key=/path/to/key" \
	--kw="username=root"

key

required:    true
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.netapp

Minimal configlet:

[array#1]
type = netapp
key = /path/to/key
server = centera1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=netapp" \
	--kw="key=/path/to/key" \
	--kw="server=centera1" \
	--kw="username=root"

key

required:    true
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

server

required:    true
scopable:    false

Example:

server = centera1

The storage server to connect.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.nexenta

Minimal configlet:

[array#1]
type = nexenta
password = system/sec/array1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=nexenta" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

port

required:    false
scopable:    false
default:     2000
convert:     int

Example:

port = 2000

The nexenta administration listener port.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.pure

Minimal configlet:

[array#1]
type = pure
api = https://array.opensvc.com/api/v1.0
client_id = bd2c75d0-f0d5-11ee-a362-8b0f2d1b83d7
issuer = opensvc
key_id = df80ae3a-f0d5-11ee-94c9-b7c8d2f57c4f
secret = system/sec/array1
username = opensvc

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=pure" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="client_id=bd2c75d0-f0d5-11ee-a362-8b0f2d1b83d7" \
	--kw="issuer=opensvc" \
	--kw="key_id=df80ae3a-f0d5-11ee-94c9-b7c8d2f57c4f" \
	--kw="secret=system/sec/array1" \
	--kw="username=opensvc"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

client_id

required:    true
scopable:    false

Example:

client_id = bd2c75d0-f0d5-11ee-a362-8b0f2d1b83d7

The client id to use as the aud key in the payload of the login jwt.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Example:

insecure = true

Disable secure socket verification.

issuer

required:    true
scopable:    false

Example:

issuer = opensvc

The issuer to use as the iss key in the payload of the login jwt token.

key_id

required:    true
scopable:    false

Example:

key_id = df80ae3a-f0d5-11ee-94c9-b7c8d2f57c4f

The key id to use as the kid key in the header of the login jwt.

secret

required:    true
scopable:    false

Example:

secret = system/sec/array1

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    true
scopable:    false

Example:

username = opensvc

The username to use as the sub key in the payload of the login jwt.

array.symmetrix

Minimal configlet:

[array#1]
type = symmetrix

Minimal setup command:

om test/ccfg/foo set --kw="type=symmetrix"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

name

required:    false
scopable:    false

Example:

name = 00012345

The name of the array. If not provided, fallback to the section name suffix.

password

required:    false
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, if configured, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

symcli_connect

required:    false
scopable:    false

Example:

symcli_connect = MY_SYMAPI_SERVER

Set the SYMCLI_CONNECT environment variable to this value.

If not set, the SCSI communication channels are used.

The value set must be declared in the /var/symapi/config/netcnfg file.

symcli_path

required:    false
scopable:    false
default:     /usr/symcli

Example:

symcli_path = /opt/symcli

Force use of a symcli programs installation, pointing the path of its head directory.

For the case multiple symcli versions are installed and the default selector does not select the version preferred for the array.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

username

required:    false
scopable:    false

Example:

username = root

The username to use to log in, if configured.

array.truenas

Minimal configlet:

[array#1]
type = truenas

Minimal setup command:

om test/ccfg/foo set --kw="type=truenas"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

type

required:    true
scopable:    false
candidates:  freenas, hds, eva, nexenta, vioserver, centera, symmetrix, emcvnx, netapp, hp3par, ibmds, ibmsvc, xtremio, dorado, hoc

The storage array driver name.

array.vioserver

Minimal configlet:

[array#1]
type = vioserver
key = /path/to/key
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=vioserver" \
	--kw="key=/path/to/key" \
	--kw="username=root"

key

required:    true
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

array.xtremio

Minimal configlet:

[array#1]
type = xtremio
api = https://array.opensvc.com/api/v1.0
password = system/sec/array1
username = root

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=xtremio" \
	--kw="api=https://array.opensvc.com/api/v1.0" \
	--kw="password=system/sec/array1" \
	--kw="username=root"

api

required:    true
scopable:    false

Example:

api = https://array.opensvc.com/api/v1.0

The array rest api url.

password

required:    true
scopable:    false

Example:

password = system/sec/array1

The password to use to log in, expressed as a sec path.

The sec must be in the system namespace and must have a password key.

username

required:    true
scopable:    false

Example:

username = root

The username to use to log in.

asset

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushasset node action.

See usr/share/doc/schedule for the schedule syntax.

checks

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushchecks node action.

See usr/share/doc/schedule for the schedule syntax.

cluster

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

ca

required:    false
scopable:    false
default:     `system/sec/ca`

convert:     list

A whitespace-separated list of sec paths.

The listener accepts a x509 client certificate if it is trusted by any CA certificate found in these sec objects.

cert

required:    false
scopable:    false
default:     `system/sec/cert`

The path of the secret hosting the certificate that the listener use for its TLS socket.

dns

required:    false
scopable:    true
convert:     list

The list of nodes to set as nameserver in the resolv.conf of the containers the CRM starts.

If set, the search will also be set to:

1/ <name>.<namespace>.svc.<clustername> 2/ <namespace>.svc.<clustername> 3/ <clustername>.

drpnodes

required:    false
scopable:    false
convert:     list

This list is fetched from the join command payload received from the joined node.

The service configuration {clusterdrpnodes} is resolved to this keyword value.

envs

required:    false
scopable:    false
default:     CERT DEV DRP FOR INT PRA PRD PRJ PPRD QUAL REC STG TMP TST UAT
convert:     list

TODO

id

required:    false
scopable:    true
default:     An autogenerated random UUID.

This unique identifier is auto-generated on install and should never be change by the cluster administrators.

It is changed when the node joins a cluster, so the remote cluster id replaces the joiners' cluster id.

name

required:    false
scopable:    false
default:     A random generated clustername.

The cluster name is used,

as the zone name in the cluster dns records
in the {fqdn} configuration reference
in the AES secret encryption metadata

The cluster name should be unique site-wide. Missing cluster name will be automatically created with random value during daemon startup.

It is always lowercased, so better to set it to a lowercase value to avoid confusion.

The cluster name is provided to joining nodes, so they can replace their own.

nodes

required:    false
scopable:    false
convert:     list

This list of node names contains only the local node name on install.

When the node joins a cluster, the joined node provides the new list, with the new node added. The joiner then replace its nodes list with the one received.

When a node receives a join request, it adds the new node to its cluster nodes list, then provide the new list to the joiner.

quorum

required:    false
scopable:    false
default:     false
convert:     bool

If true, when the cluster is split a vote happens on each cluster node.

secret

required:    false
scopable:    true
default:     A random string autogenerated on first use

The cluster shared secret used to encrypt and decrypt heartbeat payloads and sec values, with AES256

This secret is auto-generated on install, then merged from the joined nodes when joining a cluster.

The cluster name should be unique site-wide and be set right before starting to add sec keys.

vip

required:    false
scopable:    true

Example:

vip = 192.168.99.12/24@eth0

The cluster virtual ip.

If configured, the daemon creates a system/svc/vip HA failover service to manage this ip.

cni

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

config

required:    false
scopable:    false
default:     /opt/cni/net.d

Example:

config = /var/lib/opensvc/cni/net.d

The directory hosting the CNI network configuration files.

plugins

required:    false
scopable:    false
default:     /opt/cni/bin

Example:

plugins = /var/lib/opensvc/cni/bin

The directory hosting the CNI plugins.

compliance

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

auto_update

required:    false
scopable:    false
default:     false
convert:     bool

If set to true, execute om node updatecomp upon every scheduler-executed om node compliance check.

These updates keep the compliance modules in sync with the reference repository.

Warning: the module repository security is critical. Attackers could insert malicious code in served modules.

schedule

required:    false
scopable:    false
default:     02:00-06:00

Schedule parameter for the compliance auto node action, which check all attached modules and fix only those flagged autofix.

See usr/share/doc/schedule for the schedule syntax.

dequeue_actions

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false

Schedule parameter for the dequeue actions node action.

See usr/share/doc/schedule for the schedule syntax.

disks

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushdisks node action.

See usr/share/doc/schedule for the schedule syntax.

hb.disk

Minimal configlet:

[hb#1]
type = disk
dev = /dev/mapper/36589cfc000000e03957c51dabab8373a

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=disk" \
	--kw="dev=/dev/mapper/36589cfc000000e03957c51dabab8373a"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

dev

required:    true
scopable:    true

Example:

dev = /dev/mapper/36589cfc000000e03957c51dabab8373a

The device to write the heartbeats to and read from.

It must be,

Dedicated to the daemon use.
Sized 1MB for metadata + 1MB/node.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

hb.multicast

Minimal configlet:

[hb#1]
type = multicast

Minimal setup command:

om test/ccfg/foo set --kw="type=multicast"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

intf

required:    false
scopable:    true
default:     The natural interface for `<addr>`

Example:

intf = eth0

The interface to bind.

port

required:    false
scopable:    true
default:     10000
convert:     int

The port for each node to send to or listen on.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

hb.relay

Minimal configlet:

[hb#1]
type = relay
relay = relaynode1

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=relay" \
	--kw="relay=relaynode1"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

insecure

required:    false
scopable:    false
default:     false
convert:     bool

Set to true to disable the relay SSL certificate verification.

This should only be enabled for testing.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

password

required:    false
scopable:    false
default:     system/sec/relay

The name of a sec object containing a password key, which value is used as password for log in the relay api.

relay

required:    true
scopable:    false

Example:

relay = relaynode1

The relay resolvable node name.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

username

required:    false
scopable:    false
default:     relay

The username for login the relay api.

hb.unicast

Minimal configlet:

[hb#1]
type = unicast

Minimal setup command:

om test/ccfg/foo set --kw="type=unicast"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

interval

required:    false
scopable:    true
default:     5s
convert:     duration

The maximum interval between 2 heartbeat payload sends.

intf

required:    false
scopable:    true
default:     The natural interface for `<addr>`

Example:

intf = eth0

The interface to bind.

nodes

required:    false
scopable:    true
default:     All nodes.

convert:     list

The nodes participating to the heartbeat.

This keyword can be used to setup a partial redundancy like:

       n1   n2   n3   n4
hb#1   O    O    O    O
hb#2   O    O          
hb#1             O    O

port

required:    false
scopable:    true
default:     10000
convert:     int

The port for each node to send to or listen on.

timeout

required:    false
scopable:    true
default:     15s
convert:     duration

The delay since the last received heartbeat from a node before considering this node is gone.

type

required:    true
scopable:    false
candidates:  unicast, multicast, disk, relay

The heartbeat driver name.

hook

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

command

required:    false
scopable:    false
convert:     shlex

The command to execute on selected events.

The program is fed the json-formatted event data through stdin.

events

required:    false
scopable:    false
convert:     list

The list of events to execute the hook command on.

The special value all is also supported.

listener

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

crl

required:    false
scopable:    false
default:     /var/lib/opensvc/certs/ca_crl

Example:

crl = https://crl.opensvc.com

The URL serving the certificate revocation list.

The default points to the path of the cluster CA CRL in {var}/certs/ca_crl.

dns_sock_gid

required:    false
scopable:    false
default:     953

The gid owning the unix socket serving the remote backend to the pdns authoritative server.

dns_sock_uid

required:    false
scopable:    false
default:     953

The uid owning the unix socket serving the remote backend to the pdns authoritative server.

openid_well_known

required:    false
scopable:    false

Example:

openid_well_known = https://keycloak.opensvc.com/auth/realms/clusters/.well-known/openid-configuration

The URL serving the well-known configuration of an openid provider.

If set, the http listener will try to validate the Bearer token provided in the requests headers.

If the token is valid,

the user name is fetched from the preferred_username claim (fallback on name)
the user grant list is obtained by joining the multiple grant claims.

port

required:    false
scopable:    true
default:     1215
convert:     int

The port the daemon tls listener must listen on.

In pull action mode, the collector post request to notify there are actions to unqueue. The opensvc daemon executes the dequeue actions node action upon receive.

The listener.port value is sent to the collector on pushasset.

network.bridge

Minimal configlet:

[network#1]
type = bridge

Minimal setup command:

om test/ccfg/foo set --kw="type=bridge"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

network

required:    false
scopable:    false

The cluster backend network.

The routed_bridge driver fragments this network into ips_per_nodes blocks subnets.

type

required:    false
scopable:    false
candidates:  bridge, routed_bridge
default:     bridge

The type of network.

network.lo

Minimal configlet:

[network#1]
type = lo

Minimal setup command:

om test/ccfg/foo set --kw="type=lo"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

type

required:    false
scopable:    false
candidates:  bridge, routed_bridge
default:     bridge

The type of network.

network.routed_bridge

Minimal configlet:

[network#1]
type = routed_bridge

Minimal setup command:

om test/ccfg/foo set --kw="type=routed_bridge"

addr

required:    false
scopable:    true
default:     Detect using a name resolution of `<nodename>`.

Beware, if the nodename resolves to 127.0.1.1 or 127.0.0.1 the ipip tunnel can not work.

The ip address used as local endpoint for the ipip tunnel configured by the network setup command to access the backend subnet of peer nodes not reachable on the same subnet.

gateway

required:    false
scopable:    true

The gateway to use to reach the network segment of the node specified as scope.

ips_per_node

required:    false
scopable:    false
default:     1024
convert:     int

The number of allocatable ips per node on the network.

Converted to the closest power of two.

network

required:    false
scopable:    false

The cluster backend network.

The routed_bridge driver fragments this network into ips_per_nodes blocks subnets.

subnet

required:    false
scopable:    true

The cidr subnet handled by this node.

This parameter must be scoped for each node.

Usually, the subnets are allocated automatically upon initial network setup, each node being attributed a subnet based on its index in the cluster.nodes list.

tables

required:    false
scopable:    false
default:     main
convert:     list

Example:

tables = main custom1 custom2

The list of routing tables to add the backend network routes to.

The list of available tables is in /etc/iproute2/rt_tables.

tunnel

required:    false
scopable:    false
candidates:  auto, always, never
default:     auto

Create and route traffic through tunnels to peer nodes policy.

auto

Tunnel if the peer is not in the same subnet
always

Tunnel even if the peer seems to be in the same subnet. Some hosting providers require this as traffic goes through routers even between adjacent nodes.

tunnel_mode

required:    false
scopable:    false
candidates:  gre, ipip, ip6ip6
default:     ipip

type

required:    false
scopable:    false
candidates:  bridge, routed_bridge
default:     bridge

The type of network.

node

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

allowed_networks

required:    false
scopable:    false
default:     10.0.0.0/8 172.16.0.0/24 192.168.0.0/16
convert:     list

The list of cidr blocks the agents allows creation of backend network into.

Should be restricted to match your site constraints.

asset_env

required:    false
scopable:    false

Example:

asset_env = Production

An asset information to push to the collector on pushasset, overriding the currently stored value.

branch

required:    false
scopable:    false

Example:

branch = 1.9

Set the targeted opensvc agent branch.

The downloaded upgrades will honor that branch.

If not set, the repopkg imposes the target branch via the current link.

It is recommended to set branch when repopkg points to a repository you are not responsible for.

dbcompliance

required:    false
scopable:    false
default:     Same protocol, server and port as `dbopensvc`, but with a different path.

Example:

dbcompliance = https://collector.opensvc.com

Set the uri of the collector's main rpc server.

The path part of the uri can be left unspecified.

dbinsecure

required:    false
scopable:    false
convert:     bool

Set to true to disable the collector x509 certificate verification.

This should only be used for testing.

dblog

required:    false
scopable:    false
default:     true
convert:     bool

If true and dbopensvc is set, the objects action logs are reported to the collector.

Set to false to disable log reporting to the collector, even if dbopensvc is set.

dbopensvc

required:    false
scopable:    false

Example:

dbopensvc = https://collector.opensvc.com

Set the uri of the collector's feed rpc server.

The path part of the uri can be left unspecified.

If dbopensvc is not set, the agent does not try to communicate with a collector.

env

required:    false
scopable:    false
default:     TST

A code like PRD, DEV, etc... the agent can use to enforce data protection policies:

A non-PRD object instance can not be started on a PRD node
A PRD object instance can be started on a non-PRD node (typically in a DRP situation)

loc_addr

required:    false
scopable:    false

Example:

loc_addr = 7 rue blanche

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_building

required:    false
scopable:    false

Example:

loc_building = Crystal

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_city

required:    false
scopable:    false

Example:

loc_city = Paris

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_country

required:    false
scopable:    false

Example:

loc_country = fr

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_floor

required:    false
scopable:    false

Example:

loc_floor = 21

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_rack

required:    false
scopable:    false

Example:

loc_rack = R42

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_room

required:    false
scopable:    false

Example:

loc_room = 102

An asset information to push to the collector on pushasset, overriding the currently stored value.

loc_zip

required:    false
scopable:    false

Example:

loc_zip = 75017

An asset information to push to the collector on pushasset, overriding the currently stored value.

maintenance_grace_period

required:    false
scopable:    false
default:     60
convert:     duration

A duration expression, like 1m30s, defining how long the daemon keeps remote node data while it is known to be in maintenance.

The maintenance state is announced to peers at the beginning of a daemon stop and daemon restart, but not on daemon shutdown.

As long as the remote node data is kept, the local daemon won't takeover the instances running on the node in maintenance.

This parameter should be adjusted to span the daemon restart time.

max_parallel

required:    false
scopable:    false
default:     10
convert:     int

Allow a maximum of max_parallel CRM commands to run simultaneously.

Applies to both:

om <selector> <action> commands.
commands executed by the daemon for orchestrations

min_avail_mem

required:    false
scopable:    false
default:     2%
convert:     size

The minimum required available memory to allow orchestration.

min_avail_swap

required:    false
scopable:    false
default:     10%
convert:     size

The minimum required available swap to allow orchestration.

ready_period

required:    false
scopable:    false
default:     5s
convert:     duration

A duration expression, like 10s, defining how long the daemon waits before starting a service instance in ready state.

A peer node can preempt the start during this period.

Usually set to allow at least a couple of heartbeats to be received.

rejoin_grace_period

required:    false
scopable:    false
default:     90s
convert:     duration

A duration expression, like 1m30s, defining how long a starting daemon waits in rejoin state.

The daemon normally exits the rejoin state when it has received a heartbeat from all its peer nodes.

During this phase, the orchestration is not allowed, to give a chance to place the services optimally when multiple daemon were restarted at the same time.

But if a peer stays down, the other daemons have to stop waiting at some point to let the service start, even if not on their natural placement leader.

This should be adjusted to:

2s + <longest reboot duration>

The worse case of multiple nodes reboot is when the longest reboot node is rebooted near the end of the reboot of the second longest rebooting node.

|==========>
    n1 reboot
                |--------------------|
                n1 rejoin_grace_period
                |================>
                n1 in rejoin state
                                  |=====================
                                  n1 in idle state
              |==================>
              n2 reboot
                                  |--------------------|
                                  n2 rejoin_grace_period
                                  |=====================
                                  n2 in idle state

As a consequence, to minimize the rejoin_grace_period, prefer fast boot nodes.

repo

required:    false
scopable:    false

Example:

repo = http://opensvc.repo.corp

Set the uri of the opensvc agent package repository and compliance modules gzipped tarball repository.

This parameter is used by the om node updatepkg and om node updatecomp commands.

Expected repository structure:

repocomp

required:    false
scopable:    false

Example:

repocomp = http://compliance.repo.corp

Set the uri of the opensvc compliance modules repository.

A gzipped tarball is expected to be found there by the om node updatecomp command.

Expected repository structure:

ROOT
+- compliance-100.tar.gz
+- compliance-101.tar.gz
`- current -> compliance-101.tar.gz

repopkg

required:    false
scopable:    false

Example:

repopkg = http://repo.opensvc.com

Set the uri of the opensvc agent package repository.

This parameter is used by the om node updatepkg command.

Expected repository structure:

ruser

required:    false
scopable:    false
default:     root

Example:

ruser = root opensvc@node1

Set the remote user to use to login to a remote node with ssh and rsync.

The remote user must have the privileges to run as root the following commands on the remote node:

om
rsync

The default ruser is root for all nodes.

ruser accepts a list of user[@node]. If @node is omitted, user is considered the new default user.

sec_zone

required:    false
scopable:    false

Example:

sec_zone = dmz1

An asset information to push to the collector on pushasset, overriding the currently stored value.

secure_fetch

required:    false
scopable:    false
default:     true
convert:     bool

If set to false, disable ssl authentication checks on all uri fetches.

split_action

required:    false
scopable:    true
candidates:  crash, reboot, disabled
default:     crash

The node suicide method to use when a cluster split occurs and the node does not have the quorum.

This opting-out is meant to avoid double-start situations when the cluster is split.

Possible values are:

crash

Default.
reboot

May be preferred when the node power-on is not easy. No remote access via IPMI or equivalent for example.
disabled

May be used for test or training only (it does nothing).

sshkey

required:    false
scopable:    false
default:     opensvc

The basename of the ssh public key served by the GET /node/name/:nodename/ssh/key. For example, the opensvc default value serves ~/.ssh/opensvc.pub.

team_integ

required:    false
scopable:    false

Example:

team_integ = TINT

An asset information to push to the collector on pushasset, overriding the currently stored value.

team_support

required:    false
scopable:    false

Example:

team_support = TSUP

An asset information to push to the collector on pushasset, overriding the currently stored value.

packages

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushpkg node action.

See usr/share/doc/schedule for the schedule syntax.

patches

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the pushpatch node action.

See usr/share/doc/schedule for the schedule syntax.

pool.directory

Minimal configlet:

[pool#1]
type = directory

Minimal setup command:

om test/ccfg/foo set --kw="type=directory"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

path

required:    false
scopable:    false
default:     {var}/pool/directory

The fullpath of the directory hosting the pool volumes directories or loop files.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.dorado

Minimal configlet:

[pool#1]
type = dorado
array = 
diskgroup =

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=dorado" \
	--kw="array=" \
	--kw="diskgroup="

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

compression

required:    false
scopable:    false
default:     false
convert:     bool

Activate compression on created luns.

dedup

required:    false
scopable:    false
default:     false
convert:     bool

Activate data deduplcation on created luns.

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

hypermetrodomain

required:    false
scopable:    false

Example:

hypermetrodomain = HyperMetroDomain_000

Create LUN as HyperMetro replicated pairs, using this domain.

pool.drbd

Minimal configlet:

[pool#1]
type = drbd

Minimal setup command:

om test/ccfg/foo set --kw="type=drbd"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

path

required:    false
scopable:    false

The fullpath of the directory hosting the pool volumes loop files.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

vg

required:    false
scopable:    false

The name of the volume group to allocate the pool volumes logical volumes into.

zpool

required:    false
scopable:    false

The name of the zpool to allocate the pool volumes zvol into.

pool.freenas

Minimal configlet:

[pool#1]
type = freenas
array = 
diskgroup =

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=freenas" \
	--kw="array=" \
	--kw="diskgroup="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

blocksize

required:    false
scopable:    false
default:     512
convert:     size

Allow initiators to xcopy without authenticating to foreign targets.

compression

required:    false
scopable:    false
candidates:  inherit, none, lz4, gzip-1, gzip-2, gzip-3, gzip-4, gzip-5, gzip-6, gzip-7, gzip-8, gzip-9, zle, lzjb
default:     inherit

Compression level.

dedup

required:    false
scopable:    false
default:     off

Activate data deduplication on created dataset and zvol. Example values: on, off, verify

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

insecure_tpc

required:    false
scopable:    false
default:     false
convert:     bool

Allow initiators to xcopy without authenticating to foreign targets.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

sparse

required:    false
scopable:    false
default:     false
convert:     bool

Create zvol in sparse mode.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.hoc

Minimal configlet:

[pool#1]
type = hoc
array = 
diskgroup =

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=hoc" \
	--kw="array=" \
	--kw="diskgroup="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

compression

required:    false
scopable:    false
default:     false
convert:     bool

Activate compression on created luns.

dedup

required:    false
scopable:    false
default:     false
convert:     bool

Activate data deduplcation on created luns.

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

label_prefix

required:    false
scopable:    false

The prefix to add to the label assigned to the created disks.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

pool_id

required:    false
scopable:    false

The Hitachi Ops Center storage machine pool name. Volumes are created in this storage pool.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

volume_id_range_from

required:    false
scopable:    false

The start of the range of ldev ids to allocate from.

volume_id_range_to

required:    false
scopable:    false

The end of the range of ldev ids to allocate from.

vsm_id

required:    false
scopable:    false

The name of the virtual storage machine id to allocate volume into.

pool.loop

Minimal configlet:

[pool#1]
type = loop

Minimal setup command:

om test/ccfg/foo set --kw="type=loop"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

path

required:    false
scopable:    false
default:     {var}/pool/loop

The path to create the pool loop files in.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.pure

Minimal configlet:

[pool#1]
type = pure
array = 
diskgroup =

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=pure" \
	--kw="array=" \
	--kw="diskgroup="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

delete_now

required:    false
scopable:    false
default:     true
convert:     bool

If set to false the pure volumes are not immediately deleted on unprovision, so a following provision action could fail.

diskgroup

required:    true
scopable:    false

The name of the array disk group to allocate volumes from.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

label_prefix

required:    false
scopable:    false

The prefix to add to the label assigned to the created disks.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

pod

required:    false
scopable:    false

The pod to create volume into.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

volumegroup

required:    false
scopable:    false

The volumegroup to create volume disks into.

pool.share

Minimal configlet:

[pool#1]
type = share

Minimal setup command:

om test/ccfg/foo set --kw="type=share"

path

required:    false
scopable:    false
default:     {var}/pool/share

The fullpath of the shared directory hosting the pool volumes directories or loop files.

pool.shm

Minimal configlet:

[pool#1]
type = shm

Minimal setup command:

om test/ccfg/foo set --kw="type=shm"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.symmetrix

Minimal configlet:

[pool#1]
type = symmetrix
array = 
srp =

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=symmetrix" \
	--kw="array=" \
	--kw="srp="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

array

required:    true
scopable:    true

The name of the array, known as array#<name> in the node or cluster configuration.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

rdfg

required:    false
scopable:    false

Replication Group to use for SRDF.

slo

required:    false
scopable:    false

The name of the Service Level Agreement of the selected Storage Group.

srdf

required:    false
scopable:    false
default:     false
convert:     bool

Use SRDF replication.

srp

required:    true
scopable:    false

The name of the array resource pool to allocate volumes from.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.vg

Minimal configlet:

[pool#1]
type = vg
name =

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=vg" \
	--kw="name="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

fs_type

required:    false
scopable:    false
default:     xfs

The filesystem to format the pool devices with.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

name

required:    true
scopable:    false

The name of the volume group to allocate the pool volumes logical volumes into.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

pool.virtual

Minimal configlet:

[pool#1]
type = virtual

Minimal setup command:

om test/ccfg/foo set --kw="type=virtual"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

capabilities

required:    false
scopable:    false
default:     roo rwo rox rwx
convert:     list

The capabilities exposed by the virtual pool.

Supported capabilities:

shared
roo
rox
rwo
rwx
blk

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

optional_volume_env

required:    false
scopable:    false
convert:     list

Example:

optional_volume_env = container#1.name:container_name env.foo:foo

The list of the vol consumer service config keywords which values are mapped as env keys in the allocated volume service.

If the keyword is not set at the source, the default value in the template env section applies.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

template

required:    false
scopable:    false

Example:

template = templates/vol/mpool-over-loop

The path of a vol to use as a template for new volumes.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

volume_env

required:    false
scopable:    false
convert:     list

Example:

volume_env = container#1.name:container_name env.foo:foo

The list of the vol consumer service config keywords which values are mapped as env keys in the allocated volume service.

If the keyword is not set at the source, an error is raised.

pool.zpool

Minimal configlet:

[pool#1]
type = zpool
name =

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=zpool" \
	--kw="name="

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

mkblk_opt

required:    false
scopable:    false

The zvol, lv, and other block device creation command options to use to prepare the pool devices.

mkfs_opt

required:    false
scopable:    false

Example:

mkfs_opt = -O largefile

The mkfs command options to use to format the pool devices.

mnt_opt

required:    false
scopable:    true

The mount options of the fs created over the pool devices.

name

required:    true
scopable:    false

The name of the zpool to allocate the pool volumes zvol or datasets into.

status_schedule

required:    false
scopable:    false

The value to set to the status_schedule keyword of the vol objects allocated from the pool.

See usr/share/doc/schedule for the schedule syntax.

type

required:    false
scopable:    false
candidates:  directory, loop, vg, zpool, freenas, share, shm, symmetrix, virtual, dorado, hoc, drbd, pure
default:     directory

The pool type.

switch.brocade

Minimal configlet:

[switch#1]
type = brocade
username = admin

Minimal setup command:

om test/ccfg/foo set \
	--kw="type=brocade" \
	--kw="username=admin"

key

required:    false
scopable:    false

Example:

key = /path/to/key

The path to the private key to use to log in the switch.

method

required:    false
scopable:    false
candidates:  telnet, ssh
default:     ssh

Example:

method = ssh

The method to use to connect to the switch.

ssh Use key to provide a ssh key, or use the sshpass program.
telnet Set username and password with this method.

name

required:    false
scopable:    false

Example:

name = sansw1.my.corp

The name connect to the switch (dns name or ip address).

If not set, fallback to the section name suffix.

password

required:    false
scopable:    false

Example:

password = mysec/password

The password to use to log in, expressed as a sec name (not path).

The secret must be in the system namespace and must have a password key.

Either username or key must be specified.

username

required:    true
scopable:    false

Example:

username = admin

The username to use to log in the switch.

syslog

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

facility

required:    false
scopable:    false
default:     daemon

The syslog facility to log to.

host

required:    false
scopable:    false
default:     `localhost` if port is set.

The syslog server host to send logs to.

If neither host nor port are specified and if /dev/log exists, the messages are posted to /dev/log.

level

required:    false
scopable:    false
candidates:  critical, error, warning, info, debug
default:     info

The minimum message criticity to feed to syslog.

Setting to critical actually disables the syslog logging, as the agent does not emit messages at this level.

port

required:    false
scopable:    false
default:     514

The syslog server port to send logs to.

If neither host nor port are specified and if /dev/log exists, the messages are posted to /dev/log.

sysreport

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

schedule

required:    false
scopable:    false
default:     ~00:00-06:00

Schedule parameter for the sysreport node action, which collects into an archive all files and command outputs defined in /etc/opensvc/sysreport and sends that archive to the collector.

The collector stores the unpacked files in a per-node git repository.

See usr/share/doc/schedule for the schedule syntax.

DEFAULT

app

required:    false
scopable:    false
default:     default

A user-defined code linking to:

who is responsible for this service.
who is billable.

This code thus provides a most useful object grouping and filtering key.

Short and simple codes, like ERP, are easier to work with.

children

required:    false
scopable:    false
convert:     list-lowercase

The list of services or instances expressed as <path>[@<nodename>] that must be down or stdby up to allow this service to be stopped by the daemon.

The list is whitespace-separated.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

comp_schedule

required:    false
scopable:    true
default:     ~00:00-06:00

The instance compliance run schedule.

See usr/share/doc/schedule for the schedule syntax.

create_pg

required:    false
scopable:    true
default:     true
convert:     bool

Use process grouping when possible.

If turned on, the agent will create a container group for:

the object
each resource group (ie, the subset:drivergroup tuple)
each resource

A container group allows capping the memory, swap and cpu usage. These cappings can be defined using the pg_* keywords in the DEFAULT, the subset or the resource section.

disable

required:    false
scopable:    true
convert:     bool

Disables the object instance, which has the following effects:

The instance status and the status of all its resource is n/a.
Stop and start actions have no effect, and not produce error.
Disabled resources are not enabled when DEFAULT.disable=false.

drpnodes

required:    false
scopable:    true
convert:     other-nodes

Example:

drpnodes = n1 n2

A node selector expression specifying the list of cluster nodes hosting object instances when all primary nodes are unavailable, like in a DRP situation.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

Labels can be used to define a list of nodes by an arbitrary property. For example cn=fr cn=kr would be evaluated as n1 n2 n3 if n1 and n2 have the cn=fr label and n3 has the cn=kr label.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

The drpnodes can be data synchronization targets for sync resources.

encapnodes

required:    false
scopable:    false
convert:     other-nodes

Example:

encapnodes = n1 n2

A node selector expression specifying the list of cluster nodes hosting object encapsulated instances.

An object with container resources can have resources managed by OpensSVC agents deployed in these containers. These encapsulated agents form an encapsulated cluster, usually a single node cluster for a failover service.

For example a test/svc/s1 failover service, with a container#0 resource managing a e1 lxc host, can define encapnodes = e1. A app#1 resource with encap = true is then managed by the OpenSVC agent in e1.

env

required:    false
scopable:    false
default:     The same as the node `env`.

A code like PRD, DEV, etc... the agent can use to enforce data protection policies:

A non-PRD object instance can not be started on a PRD node
A PRD object instance can be started on a non-PRD node (typically in a DRP situation)

The default value is read from the node env keyword.

flex_max

required:    false
scopable:    false
depends:     topology=flex
default:     The number of elements in `nodes`.

convert:     int

The maximum number of up instances of this object in the cluster. Above this number the aggregated object status is degraded to warn.

The 0 value is interpreted as unlimited.

flex_min

required:    false
scopable:    false
depends:     topology=flex
default:     1
convert:     int

The minimum number of up instances of this object in the cluster. Below this number the aggregated object status is degraded to warn.

flex_primary

required:    false
scopable:    true
depends:     topology=flex
default:     The first node of `nodes`.

convert:     list-lowercase

The node in charge of syncing the other nodes in a flex object.

flex_target

required:    false
scopable:    false
depends:     topology=flex
default:     The value of `flex_min`.

convert:     int

The optimal number of up instances of the object in the cluster. The value must be between flex_min and flex_max.

If orchestrate=ha, the daemon is free to take action to reach the flex_target.

hard_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

hard_affinity = svc1 svc2

A whitespace separated list of object paths.

These objects must be started on the local node to allow the local monitor to start an instance of the service.

hard_anti_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

hard_anti_affinity = svc1 svc2

A whitespace separated list of object paths.

These object must not be started on the local node to allow the local monitor to start an instance of the object.

id

required:    false
scopable:    false
default:     A random generated UUID.

A rfc4122 random uuid generated by the agent.

monitor_action

required:    false
scopable:    true
candidates:  crash, freezestop, none, reboot, switch, no-op
default:     none
convert:     list

Example:

monitor_action = reboot

The action to trigger when a monitored resource is no longer in the "up" or "standby up" state, and all restart attempts for the resource have failed.

The reboot and crash monitor actions do not attempt to cleanly stop any processes. On Linux, they utilize system-level sysrq triggers.

This behavior is designed to ensure that the host stops writing to shared disks as quickly as possible, minimizing the risk of data corruption. This is critical because a failover node is likely preparing to write to the same shared disks.

You can append a fallback monitor action to this keyword. A common example is freezestop reboot. In this case, the reboot action will be executed if the stop fails or times out.

Other monitor_actions values:

none: the default value for monitor action disabled (monitorkeyword must be also false or undefined).
freezestop: freeze and subsequently stop the monitored instance.
switch: try monitored instance stop to allow any other cluster nodes to takeover the instance.
no-op: The monitor action No Operation is called but does nothing. It may be used for demonstration. The final local expect after call will be set to evicted.

monitor_schedule

required:    false
scopable:    true
default:     @5m

The instance monitored resources status evaluation schedule.

See usr/share/doc/schedule for the schedule syntax.

nodes

required:    false
scopable:    true
default:     The lowercased hostname of the evaluating node.

convert:     nodes

Example:

nodes = n1 n*

A node selector expression specifying the list of cluster nodes hosting object instances.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

orchestrate

required:    false
scopable:    false
candidates:  no, ha, start
default:     no

Orchestrate defines how the daemon will manage the service.

no The daemon does not try to keep the service up. On boot, the service won't be started.

The daemon does not try to reach the flex_target number of up instances for flex services.
start Services with topology=failover won't failover automatically only if the target instance is the natural placement leader. Which means the service is started when its primary node reboots, if it does not run elsewhere already.

The daemon does not try to reach the flex_target number of up instances for flex services.
ha Services with topology=failover failover automatically.

The daemon tries to reach the flex_target number of up instances for flex services.

The resource restart policy is not affected by the orchestrate value.

parents

required:    false
scopable:    false
convert:     list-lowercase

The list of services or instances expressed as <path>[@<nodename>] that must be up to allow this service to be started by the daemon.

The list is whitespace-separated.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

placement

required:    false
scopable:    false
candidates:  nodes order, last start, load avg, shift, spread, score, , none
default:     nodes order

Set a service instances placement policy:

none

No placement policy. a policy for dummy, observe-only, services.
nodes order

The left-most available node is allowed to start a service instance when necessary.
last start

The preferred instances is the one started last.
load avg

The least loaded node takes precedences.
shift

Shift the nodes order ranking by the service prefix converter to an integer.
spread

A spread policy tends to perfect leveling with many services.
score

The highest scoring node takes precedence (the score is a composite indice of load, mem and swap).

pre_monitor_action

required:    false
scopable:    true

Example:

pre_monitor_action = /bin/true

A callout to execute before the monitor_action.

For example, if monitor_action = freezestop, a pre_monitor_action script may decide to crash the server if it detects a situation were freezestop can not succeed (for example, a fs can not be umounted due to an unresponsive storage array).

priority

required:    false
scopable:    false
default:     50
convert:     int

When the daemon has so many actions to submit in parallel that the node.max_parallel limit is reached, this priority is used to determine which service are served first.

The priority is an just an number used as a sort key. The smaller the number, the higher the priority.

The priority setting is dropped from a service configuration injected via the api by a user not having the prioritizer grant.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

resinfo_schedule

required:    false
scopable:    true
default:     @60m

The instance key-val table emit schedule.

See usr/share/doc/schedule for the schedule syntax.

rollback

required:    false
scopable:    true
default:     true
convert:     bool

If set to false, the default rollback on start action error behaviour is disabled, leaving the instance in its half-started state (avail warn).

The daemon then refuses to failover a service if any instance is in warn availabity state. It is highly recommended to not use rollback=false if orchestrate=ha.

run_schedule

required:    false
scopable:    true

The instance tasks run action default schedule.

See usr/share/doc/schedule for the schedule syntax.

shared

required:    false
scopable:    true
default:     true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

soft_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

soft_affinity = svc1 svc2

A whitespace separated list of services that must be started on the node to allow the monitor to start this service.

If the local node is the only candidate ignore this constraint and allow start.

soft_anti_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

soft_anti_affinity = svc1 svc2

A whitespace separated list of services that must not be started on the node to allow the monitor to start this service.

If the local node is the only candidate ignore this constraint and allow start.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_schedule

required:    false
scopable:    true
default:     @10m

The instance status evaluation schedule.

See usr/share/doc/schedule for the schedule syntax.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stonith

required:    false
scopable:    false
depends:     topology=failover
default:     false
convert:     bool

Shoot The Other Node In The Head, aka fence, using a callout.

The callout is triggered after a quorum vote won, when the surviving node is about to start a local instance of a service that was known to be started on a unreachable peer node.

The callout is meant to prevent the peer from writing to shared disks, remote databases, and from responding to clients.

The Fence Agents project is a well known bundle of callout used by many clustering tools.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync_schedule

required:    false
scopable:    true
default:     04:00-06:00

The instance sync default schedule.

See usr/share/doc/schedule for the schedule syntax.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

topology

required:    false
scopable:    false
candidates:  failover, flex
default:     failover

failover

The service is allowed to be up on one node at a time.
flex

The service can be up on flex_target nodes, where flex_target must be in the [flex_min, flex_max] range.

type

required:    false
scopable:    false

The resource driver name.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

app.forking

Minimal configlet:

[app#1]
type = forking

Minimal setup command:

om test/svc/foo set --kw="type=forking"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true

true

Execute the script command with status argument on status action.
false

Do nothing on status action.
<shlex expression>

Execute this command on status action.

check_timeout

required:    false
scopable:    true
convert:     duration

Example:

check_timeout = 180

Wait for <duration> before declaring the app launcher status action a failure.

Takes precedence over timeout.

If neither timeout nor check_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance status when an app launcher did not return.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the process execution environment:

NGINX_USER=user1
key1=val1

cwd

required:    false
scopable:    true

Change the working directory to the specified location instead of the default <pathtmp>.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = CRT=cert1/server.crt PEM=cert1/server.pem

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

group

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified group instead of root.

info

required:    false
scopable:    true
default:     false

true

Execute the script command with info argument on push resinfo action.
false

Do nothing on push resinfo action.
<shlex expression>

Execute this command on push resinfo action.

Stdout lines must contain only one key:value.

Invalid lines are dropped.

info_timeout

required:    false
scopable:    true
convert:     duration

Example:

info_timeout = 180

Wait for <duration> before declaring the app launcher info action a failure.

Takes precedence over timeout.

If neither timeout nor info_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance info when an app launcher did not return.

limit_as

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes) (same as limit_vmem).

When both limit_vmem and limit_as is used, the max value is chosen.

limit_core

required:    false
scopable:    true
convert:     size

The limit on the largest core dump size that can be produced (unit byte).

limit_cpu

required:    false
scopable:    true
convert:     duration

Example:

limit_cpu = 30s

The limit on CPU time (duration).

limit_data

required:    false
scopable:    true
convert:     size

The limit on the data segment size of a process (unit byte).

limit_fsize

required:    false
scopable:    true
convert:     size

The limit on the largest file that can be created (unit byte).

limit_memlock

required:    false
scopable:    true
convert:     size

The limit on how much memory a process can lock with mlock(2) (unit byte, no solaris support).

limit_nofile

required:    false
scopable:    true
convert:     size

The limit on the number files a process can have open at once.

limit_nproc

required:    false
scopable:    true
convert:     size

The limit on the number of processes this user can have at one time, no solaris support.

limit_rss

required:    false
scopable:    true
convert:     size

The limit on the total physical memory that can be in use by a process (unit byte, no solaris support).

limit_stack

required:    false
scopable:    true
convert:     size

The limit on the stack size of a process (unit bytes).

limit_vmem

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes).

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

script

required:    false
scopable:    true

Full path to the app launcher script.

This script must accept as argument 0 the action word:

start for start
stop for stop
status for status check
info for resource info

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the process execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start

required:    false
scopable:    true

true

Execute the script command with start argument on start action.
false

Do nothing on start action.
<shlex expression>

Execute this command on start action.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 180

Wait for <duration> before declaring the app launcher start action a failure.

Takes precedence over timeout.

If neither timeout nor start_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance start when an app launcher did not return.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_log

required:    false
scopable:    true
default:     false
convert:     bool

If true, redirect the checker script:

stdout to the resource status info-log.
stderr to the resource status warn-log.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop

required:    false
scopable:    true

true

Execute the script command with stop argument on stop action.
false

Do nothing on stop action.
<shlex expression>

Execute this command on stop action.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 180

Wait for <duration> before declaring the app launcher stop action a failure.

Takes precedence over timeout.

If neither timeout nor stop_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance stop when an app launcher did not return.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 180

Wait for <duration> before declaring the app launcher action a failure.

Can be overridden by <action>_timeout.

If no timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance action when an app launcher did not return.

type

required:    false
scopable:    false

The resource driver name.

umask

required:    false
scopable:    true
convert:     umask

Example:

umask = 022

The umask to set for the application process.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified user instead of root.

app.simple

Minimal configlet:

[app#1]
type = simple

Minimal setup command:

om test/svc/foo set --kw="type=simple"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true

true

Execute the script command with status argument on status action.
false

Do nothing on status action.
<shlex expression>

Execute this command on status action.

check_timeout

required:    false
scopable:    true
convert:     duration

Example:

check_timeout = 180

Wait for <duration> before declaring the app launcher status action a failure.

Takes precedence over timeout.

If neither timeout nor check_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance status when an app launcher did not return.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the process execution environment:

NGINX_USER=user1
key1=val1

cwd

required:    false
scopable:    true

Change the working directory to the specified location instead of the default <pathtmp>.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = CRT=cert1/server.crt PEM=cert1/server.pem

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

group

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified group instead of root.

info

required:    false
scopable:    true
default:     false

true

Execute the script command with info argument on push resinfo action.
false

Do nothing on push resinfo action.
<shlex expression>

Execute this command on push resinfo action.

Stdout lines must contain only one key:value.

Invalid lines are dropped.

info_timeout

required:    false
scopable:    true
convert:     duration

Example:

info_timeout = 180

Wait for <duration> before declaring the app launcher info action a failure.

Takes precedence over timeout.

If neither timeout nor info_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance info when an app launcher did not return.

limit_as

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes) (same as limit_vmem).

When both limit_vmem and limit_as is used, the max value is chosen.

limit_core

required:    false
scopable:    true
convert:     size

The limit on the largest core dump size that can be produced (unit byte).

limit_cpu

required:    false
scopable:    true
convert:     duration

Example:

limit_cpu = 30s

The limit on CPU time (duration).

limit_data

required:    false
scopable:    true
convert:     size

The limit on the data segment size of a process (unit byte).

limit_fsize

required:    false
scopable:    true
convert:     size

The limit on the largest file that can be created (unit byte).

limit_memlock

required:    false
scopable:    true
convert:     size

The limit on how much memory a process can lock with mlock(2) (unit byte, no solaris support).

limit_nofile

required:    false
scopable:    true
convert:     size

The limit on the number files a process can have open at once.

limit_nproc

required:    false
scopable:    true
convert:     size

The limit on the number of processes this user can have at one time, no solaris support.

limit_rss

required:    false
scopable:    true
convert:     size

The limit on the total physical memory that can be in use by a process (unit byte, no solaris support).

limit_stack

required:    false
scopable:    true
convert:     size

The limit on the stack size of a process (unit bytes).

limit_vmem

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes).

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

script

required:    false
scopable:    true

Full path to the app launcher script.

This script must accept as argument 0 the action word:

start for start
stop for stop
status for status check
info for resource info

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the process execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start

required:    false
scopable:    true

true

Execute the script command with start argument on start action.
false

Do nothing on start action.
<shlex expression>

Execute this command on start action.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_log

required:    false
scopable:    true
default:     false
convert:     bool

If true, redirect the checker script:

stdout to the resource status info-log.
stderr to the resource status warn-log.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop

required:    false
scopable:    true

true

Execute the script command with stop argument on stop action.
false

Do nothing on stop action.
<shlex expression>

Execute this command on stop action.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 180

Wait for <duration> before declaring the app launcher stop action a failure.

Takes precedence over timeout.

If neither timeout nor stop_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance stop when an app launcher did not return.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 180

Wait for <duration> before declaring the app launcher action a failure.

Can be overridden by <action>_timeout.

If no timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance action when an app launcher did not return.

type

required:    false
scopable:    false

The resource driver name.

umask

required:    false
scopable:    true
convert:     umask

Example:

umask = 022

The umask to set for the application process.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified user instead of root.

certificate.tls

Minimal configlet:

[certificate#1]
type = tls

Minimal setup command:

om test/svc/foo set --kw="type=tls"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

certificate_chain_filename

required:    false
scopable:    true

Local filesystem data source of the TLS certificate chain.

certificate_chain_inline_string

required:    false
scopable:    true

String inlined data source of the TLS certificate chain.

certificate_secret

required:    false
scopable:    true

The name of the secret object name hosting the certificate files.

The secret must have the certificate_chain and server_key keys set.

This setting makes the certificate served to envoy via the secret discovery service, which allows its live rotation.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

private_key_filename

required:    false
scopable:    true

Local filesystem data source of the TLS private key.

private_key_inline_string

required:    false
scopable:    true

String inlined filesystem data source of the TLS private key. A reference to a secret for example.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

validation_secret

required:    false
scopable:    true

The name of the secret object name hosting the certificate authority files for certificate_secret validation.

The secret must have the trusted_ca and verify_certificate_hash keys set.

This setting makes the validation data served to envoy via the secret discovery service, which allows certificates live rotation.

container.docker

Minimal configlet:

[container#1]
type = docker

Minimal setup command:

om test/svc/foo set --kw="type=docker"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

detach

required:    false
scopable:    true
default:     true
convert:     bool

Run container in background.

Set to false only for init containers, alongside start_timeout and the nostatus tag.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    false
scopable:    true
default:     ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log_outputs

required:    false
scopable:    true
default:     false
convert:     bool

Log the container run commands stdout and stderr

Set to true to enable logging of container run commands.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

rm

required:    false
scopable:    true
convert:     bool

Example:

rm = false

If rm=true, the container instance is removed when the resource is stopped. If detach=false, the driver automatically behaves as if rm=true.

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

Example:

start_timeout = 1m5s

Wait for <duration> before declaring the container action a failure.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
default:     2m30s
convert:     duration

Example:

stop_timeout = 2m

Wait for <duration> before declaring the container action a failure.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

If not set, the container will have a private userns other containers can share.

A container with userns=host will share the host's userns.

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

container.kvm

Minimal configlet:

[container#1]
type = kvm
name =

Minimal setup command:

om test/svc/foo set \
	--kw="type=kvm" \
	--kw="name="

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, the container name is used.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

The name to assign to the container.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    true
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

rcmd

required:    false
scopable:    true
convert:     shlex

Example:

rcmd = lxc-attach -e -n osvtavnprov01 --

The command to wrap another command to execute it in the container.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
default:     4m
convert:     duration

Example:

start_timeout = 1m5s

Wait for <duration> before declaring the container action a failure.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

stop_timeout = 2m30s

Wait for <duration> before declaring the container action a failure.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

virtinst

required:    false
scopable:    false
convert:     shlex

Example:

virtinst = --release focal

The arguments to pass through lxc-create to the per-template script.

container.lxc

Minimal configlet:

[container#1]
type = lxc
name =

Minimal setup command:

om test/svc/foo set \
	--kw="type=lxc" \
	--kw="name="

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

cf

required:    false
scopable:    false

Example:

cf = /srv/svc1/config

Defines a lxc configuration file in a non-standard location.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

create_configs_environment = CRT=cert1/server.crt PEM=cert1/server.pem

Set variables in the lxc-create execution environment.

A whitespace separated list of <var>=<cfg name>/<key path>.

A shell expression spliter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

create_environment

required:    false
scopable:    true
convert:     shlex

Example:

create_environment = FOO=bar BAR=baz

Set variables in the lxc-create execution environment.

A whitespace separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

create_secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

create_secrets_environment = CRT=cert1/server.crt PEM=cert1/server.pem

Set variables in the lxc-create execution environment.

A whitespace separated list of <var>=<sec name>/<key path>.

A shell expression spliter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

data_dir

required:    false
scopable:    true

Example:

data_dir = /srv/svc1/data/containers

If this keyword is set, the service configures a resource-private container data store.

This setup is required for stateful service relocalization.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, the container name is used.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

The name to assign to the container.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    true
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

rcmd

required:    false
scopable:    true
convert:     shlex

Example:

rcmd = lxc-attach -e -n osvtavnprov01 --

The command to wrap another command to execute it in the container.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

rootfs

required:    false
scopable:    false

Example:

rootfs = /srv/svc1/data/containers

Sets the root fs directory of the container.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
default:     4m
convert:     duration

Example:

start_timeout = 1m5s

Wait for <duration> before declaring the container action a failure.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

stop_timeout = 2m30s

Wait for <duration> before declaring the container action a failure.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

template

required:    false
scopable:    false

Example:

template = ubuntu

Sets the url of the template unpacked into the container root fs or the name of the template passed to lxc-create.

template_options

required:    false
scopable:    false
convert:     shlex

Example:

template_options = --release focal

The arguments to pass through lxc-create to the per-template script.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

container.oci

Minimal configlet:

[container#1]
type = oci

Minimal setup command:

om test/svc/foo set --kw="type=oci"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

detach

required:    false
scopable:    true
default:     true
convert:     bool

Run container in background.

Set to false only for init containers, alongside start_timeout and the nostatus tag.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    false
scopable:    true
default:     ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log_outputs

required:    false
scopable:    true
default:     false
convert:     bool

Log the container run commands stdout and stderr

Set to true to enable logging of container run commands.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

rm

required:    false
scopable:    true
convert:     bool

Example:

rm = false

If rm=true, the container instance is removed when the resource is stopped. If detach=false, the driver automatically behaves as if rm=true.

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

Example:

start_timeout = 1m5s

Wait for <duration> before declaring the container action a failure.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
default:     2m30s
convert:     duration

Example:

stop_timeout = 2m

Wait for <duration> before declaring the container action a failure.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

If not set, the container will have a private userns other containers can share.

A container with userns=host will share the host's userns.

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

container.podman

Minimal configlet:

[container#1]
type = podman

Minimal setup command:

om test/svc/foo set --kw="type=podman"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

detach

required:    false
scopable:    true
default:     true
convert:     bool

Run container in background.

Set to false only for init containers, alongside start_timeout and the nostatus tag.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    false
scopable:    true
default:     ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log_outputs

required:    false
scopable:    true
default:     false
convert:     bool

Log the container run commands stdout and stderr

Set to true to enable logging of container run commands.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

rm

required:    false
scopable:    true
convert:     bool

Example:

rm = false

If rm=true, the container instance is removed when the resource is stopped. If detach=false, the driver automatically behaves as if rm=true.

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

Example:

start_timeout = 1m5s

Wait for <duration> before declaring the container action a failure.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
default:     2m30s
convert:     duration

Example:

stop_timeout = 2m

Wait for <duration> before declaring the container action a failure.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

Defines the podman container run --userns value.

the 'container#...' values are converted to container:id

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

container.vbox

Minimal configlet:

[container#1]
type = vbox
name =

Minimal setup command:

om test/svc/foo set \
	--kw="type=vbox" \
	--kw="name="

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

headless

required:    false
scopable:    false
default:     false
convert:     bool

Using --type Headless in the 'VBoxManage startvm' command

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, the container name is used.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

The name to assign to the container.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    true
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

rcmd

required:    false
scopable:    true
convert:     shlex

Example:

rcmd = lxc-attach -e -n osvtavnprov01 --

The command to wrap another command to execute it in the container.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
default:     4m
convert:     duration

Example:

start_timeout = 1m5s

Wait for <duration> before declaring the container action a failure.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

stop_timeout = 2m30s

Wait for <duration> before declaring the container action a failure.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.crypt

Minimal configlet:

[disk#1]
type = crypt
dev = /dev/{fqdn}/lv1

Minimal setup command:

om test/svc/foo set \
	--kw="type=crypt" \
	--kw="dev=/dev/{fqdn}/lv1"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/{fqdn}/lv1

The fullpath of the underlying block device.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

label

required:    false
scopable:    true
default:     {fqdn}

The label to set in the cryptsetup metadata written on dev.

A label helps admin understand the role of a device.

manage_passphrase

required:    false
scopable:    true
default:     true
convert:     bool

By default, on provision the driver allocates a new random passphrase (256 printable chars), and forgets it on unprovision.

If set to false, require a passphrase to be already present in the sec object to provision, and don't remove it on unprovision.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     The basename of the underlying device, suffixed with `-crypt`.

Example:

name = {fqdn}-crypt

The basename of the exposed device.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

secret

required:    false
scopable:    true
default:     {name}

The name of the sec object hosting the crypt secrets.

The sec object must be in the same namespace than the object defining the disk.crypt resource.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.disk

Minimal configlet:

[disk#1]
type = disk

Minimal setup command:

om test/svc/foo set --kw="type=disk"

array

required:    false
scopable:    true

Example:

array = xtremio-prod1

The array to provision the disk from.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

disk_id

required:    false
scopable:    true

Example:

disk_id = 6589cfc00000097484f0728d8b2118a6

The wwn of the disk.

diskgroup

required:    false
scopable:    true

Example:

diskgroup = default

The array disk group to provision the disk from.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true

Example:

name = myfcdisk1

The name of the disk.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

Example:

pool = fcpool1

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

Example:

size = 20g

A size expression for the disk allocation.

slo

required:    false
scopable:    true

Example:

slo = Optimized

The provisioned disk service level objective.

This keyword is honored on arrays supporting this notion (ex: EMC VMAX).

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.drbd

Minimal configlet:

[disk#1]
type = drbd

Minimal setup command:

om test/svc/foo set --kw="type=drbd"

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

disk

required:    false
scopable:    true

Example:

disk = /dev/vg1/lv1

The path of the device to provision the drbd on.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_peers

required:    false
scopable:    false
default:     (nodes_count*2)-1

convert:     int

Example:

max_peers = 8

The integer value to use in create-md --max-peers <n>.

The driver ensures the value is not lesser than the number of instances.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

network

required:    false
scopable:    false

Example:

network = benet1

The name of the backend network to use for drbd traffic.

Set this keyword if some node names are resolved to NATed addresses.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

port

required:    false
scopable:    true
convert:     int

Example:

port = 1.2.3.4

The port to use to connect a peer.

Auto-allocated on provision if not already defined.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

res

required:    false
scopable:    false

Example:

res = r1

The name of the drbd resource associated with this service resource.

OpenSVC expects the resource configuration file to reside in /etc/drbd.d/<res>.res and takes care of its replication on peer nodes.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.loop

Minimal configlet:

[disk#1]
type = loop
file = /srv/{fqdn}-loop-{rindex}

Minimal setup command:

om test/svc/foo set \
	--kw="type=loop" \
	--kw="file=/srv/{fqdn}-loop-{rindex}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

file

required:    true
scopable:    true

Example:

file = /srv/{fqdn}-loop-{rindex}

The loopback device backing file full path.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true

Example:

size = 100m

The size of the loop file to provision.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.lv

Minimal configlet:

[disk#1]
type = lv
name = lv1
vg = vg1

Minimal setup command:

om test/svc/foo set \
	--kw="type=lv" \
	--kw="name=lv1" \
	--kw="vg=vg1"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_options

required:    false
scopable:    true
convert:     shlex

Example:

create_options = --contiguous y

Additional options to pass to the logical volume create command (lvcreate or vxassist, depending on the driver).

The size and name are managed natively.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = lv1

The name of the logical volume.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true

Example:

size = 10m

The size of the logical volume to provision. A size expression or <n>%{FREE|PVS|VG}.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

vg

required:    true
scopable:    true

Example:

vg = vg1

The name of the volume group hosting the logical volume.

disk.lvm

Minimal configlet:

[disk#1]
type = lvm
name = vg1

Minimal setup command:

om test/svc/foo set \
	--kw="type=lvm" \
	--kw="name=vg1"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = vg1

The name of the logical volume group.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

options

required:    false
scopable:    true
convert:     shlex

Example:

options = --zero=y

The vgcreate options to use upon vg provisioning.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pvs

required:    false
scopable:    true
convert:     list

Example:

pvs = /dev/mapper/23 /dev/mapper/24

The list of paths to the physical volumes of the volume group.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.md

Minimal configlet:

[disk#1]
type = md

Minimal setup command:

om test/svc/foo set --kw="type=md"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

chunk

required:    false
scopable:    true
convert:     size

Example:

chunk = 128k

The md chunk size to use with the mdadm create command.

The value is adjusted to the first greater or equal multiple of 4.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

devs

required:    false
scopable:    true
convert:     list

Example:

devs = /dev/mapper/23 /dev/mapper/24

The md member devices to use with the mdadm create command.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

level

required:    false
scopable:    true

Example:

level = raid1

The md raid level to use with the mdadm create command.

See man mdadm for the list of supported values.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

spares

required:    false
scopable:    true
default:     0
convert:     int

Example:

spares = 1

The md number of spare devices to use with the mdadm create command.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

uuid

required:    false
scopable:    true

Example:

uuid = dev1

The md uuid to use with the mdadm assemble command.

disk.raw

Minimal configlet:

[disk#1]
type = raw
devs = /dev/mapper/svc.d0:/dev/oracle/redo001 /dev/mapper/svc.d1

Minimal setup command:

om test/svc/foo set \
	--kw="type=raw" \
	--kw="devs=/dev/mapper/svc.d0:/dev/oracle/redo001 /dev/mapper/svc.d1"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_char_devices

required:    false
scopable:    true
default:     true
convert:     bool

Example:

create_char_devices = false

On Linux, char devices are not automatically created when devices are discovered.

If set to true (the default), the raw resource driver will create and delete them using the raw kernel driver.

devs

required:    true
scopable:    true
convert:     list

Example:

devs = /dev/mapper/svc.d0:/dev/oracle/redo001 /dev/mapper/svc.d1

A whitespace-separated list of device paths or <src>[:<dst>] device paths mappings.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should own the device. Either in numeric or symbolic form.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 600

The permissions the device should have. A string representing the octal permissions.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should own the device. Either in numeric or symbolic form.

zone

required:    false
scopable:    true

Example:

zone = zone1

The zone name the raw resource is linked to.

If set, the raw files are configured from the global reparented to the zonepath.

disk.vg

Minimal configlet:

[disk#1]
type = vg
name = vg1

Minimal setup command:

om test/svc/foo set \
	--kw="type=vg" \
	--kw="name=vg1"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = vg1

The name of the logical volume group.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

options

required:    false
scopable:    true
convert:     shlex

Example:

options = --zero=y

The vgcreate options to use upon vg provisioning.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pvs

required:    false
scopable:    true
convert:     list

Example:

pvs = /dev/mapper/23 /dev/mapper/24

The list of paths to the physical volumes of the volume group.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.zpool

Minimal configlet:

[disk#1]
type = zpool
name = tank

Minimal setup command:

om test/svc/foo set \
	--kw="type=zpool" \
	--kw="name=tank"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_options

required:    false
scopable:    true
convert:     shlex

Example:

create_options = -O dedup=on

The zone name the zpool refers to. If set, the zpool is activated in the zone context.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

multihost

required:    false
scopable:    true
convert:     tristate

Example:

multihost = yes

If true, set the zfs property multihost=on on start if not already set. This requires all nodes to be booted with a /etc/hostid installed, preferably generated by the zgenhostid command.

If false, set the zfs property multihost=off.

If left empty, the current multihost property is left unchanged.

name

required:    true
scopable:    true

Example:

name = tank

The name of the zfs pool.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

vdev

required:    false
scopable:    true
convert:     list

Example:

vdev = /dev/mapper/23 /dev/mapper/24

The vdev list, including optional parity keywords, as would be passed to zpool create.

zone

required:    false
scopable:    true

The zone name the zpool refers to. If set, the zpool is activated in the zone context.

disk.zvol

Minimal configlet:

[disk#1]
type = zvol
name = tank/zvol1

Minimal setup command:

om test/svc/foo set \
	--kw="type=zvol" \
	--kw="name=tank/zvol1"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

blocksize

required:    false
scopable:    true
convert:     size

Example:

blocksize = 256k

The blocksize of the zfs volume to create.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_options

required:    false
scopable:    true
convert:     shlex

Example:

create_options = -o dedup=on

The zfs create -V <name> extra options.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = tank/zvol1

The full name of the zfs volume in the <pool>/<name> form.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

Example:

size = 10m

The size of the zfs volume to create.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

expose.envoy

Minimal configlet:

[expose#1]
type = envoy
port =

Minimal setup command:

om test/svc/foo set \
	--kw="type=envoy" \
	--kw="port="

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

cluster_certificates

required:    false
scopable:    true
convert:     list

The TLS certificates used to communicate with cluster endpoints.

cluster_data

required:    false
scopable:    true

The envoy protocol compliant data in json format used to bootstrap the Cluster config messages.

Parts of this structure, like endpoints, are amended to reflect the actual cluster state.

cluster_private_key_filename

required:    false
scopable:    true

Local filesystem data source of the TLS private key used to communicate with cluster endpoints.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

filter_config_data

required:    false
scopable:    true

The envoy protocol compliant data in json format used to bootstrap the Listener filter config messages.

Parts of this structure, like routes, are amended by more specific keywords.

gateway

required:    false
scopable:    true

The name of the ingress gateway that should handle this expose.

lb_policy

required:    false
scopable:    true
candidates:  round robin, least_request, ring_hash, random, original_dst_lb, maglev
default:     round robin

The name of the envoy cluster load balancing policy.

listener_addr

required:    false
scopable:    true
default:     The main proxy ip address.

The public ip address to expose from.

In case the envoy server has multiple front-facing ip addresses, and the service must not be exposed on all of these addresses.

listener_certificates

required:    false
scopable:    true
convert:     list

The TLS certificates used by the listener.

listener_port

required:    false
scopable:    true
default:     The expose <port>.

convert:     int

The public port number to expose on.

The special value 0 is interpreted as a request for auto-allocation.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

port

required:    true
scopable:    true
convert:     int

The port number of the endpoint.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

protocol

required:    false
scopable:    true
candidates:  tcp, udp
default:     tcp

The protocol of the endpoint.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

sni

required:    false
scopable:    true
convert:     list

The SNI server names to match on the proxy to select this service endpoints.

The socket server must support TLS.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

vhosts

required:    false
scopable:    true
convert:     list

The list of vhost resource identifiers for this expose.

fs.9pfs

Minimal configlet:

[fs#1]
type = 9pfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=9pfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

This can help detection of nfs stale file systems. It is ignored when mnt_opt contains 'nointr'. The file system read check is: 'timeout {stat_timeout} stat -f {mnt}' The file system write check is disabled when fs_type is a network file system or when mnt_opt contains 'ro'.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.afs

Minimal configlet:

[fs#1]
type = afs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=afs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.bfs

Minimal configlet:

[fs#1]
type = bfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=bfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.bind

Minimal configlet:

[fs#1]
type = bind
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=bind" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.btrfs

Minimal configlet:

[fs#1]
type = btrfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=btrfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.cephfs

Minimal configlet:

[fs#1]
type = cephfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=cephfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.cifs

Minimal configlet:

[fs#1]
type = cifs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=cifs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.directory

Minimal configlet:

[fs#1]
type = directory
path =

Minimal setup command:

om test/svc/foo set \
	--kw="type=directory" \
	--kw="path="

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the directory. Either in numeric or symbolic form.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

path

required:    true
scopable:    true

The fullpath of the directory to create.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The permissions the directory should have. A string representing the octal permissions.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the directory. Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to. If set, the fs mount point is reparented into the zonepath rootfs.

fs.ext2

Minimal configlet:

[fs#1]
type = ext2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ext2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ext3

Minimal configlet:

[fs#1]
type = ext3
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ext3" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ext4

Minimal configlet:

[fs#1]
type = ext4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ext4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.f2fs

Minimal configlet:

[fs#1]
type = f2fs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=f2fs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.flag

Minimal configlet:

[fs#1]
type = flag

Minimal setup command:

om test/svc/foo set --kw="type=flag"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

fs.gfs

Minimal configlet:

[fs#1]
type = gfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=gfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.gfs2

Minimal configlet:

[fs#1]
type = gfs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=gfs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.glusterfs

Minimal configlet:

[fs#1]
type = glusterfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=glusterfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.gpfs

Minimal configlet:

[fs#1]
type = gpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=gpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.hfs

Minimal configlet:

[fs#1]
type = hfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=hfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.hfsplus

Minimal configlet:

[fs#1]
type = hfsplus
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=hfsplus" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.hpfs

Minimal configlet:

[fs#1]
type = hpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=hpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jffs

Minimal configlet:

[fs#1]
type = jffs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=jffs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jffs2

Minimal configlet:

[fs#1]
type = jffs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=jffs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jfs

Minimal configlet:

[fs#1]
type = jfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=jfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jfs2

Minimal configlet:

[fs#1]
type = jfs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=jfs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.lofs

Minimal configlet:

[fs#1]
type = lofs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=lofs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.logfs

Minimal configlet:

[fs#1]
type = logfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=logfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.minix

Minimal configlet:

[fs#1]
type = minix
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=minix" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.msdos

Minimal configlet:

[fs#1]
type = msdos
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=msdos" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ncpfs

Minimal configlet:

[fs#1]
type = ncpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ncpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.nfs

Minimal configlet:

[fs#1]
type = nfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=nfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.nfs4

Minimal configlet:

[fs#1]
type = nfs4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=nfs4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.nilfs

Minimal configlet:

[fs#1]
type = nilfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=nilfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.none

Minimal configlet:

[fs#1]
type = none
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=none" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ntfs

Minimal configlet:

[fs#1]
type = ntfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ntfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ocfs

Minimal configlet:

[fs#1]
type = ocfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ocfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ocfs2

Minimal configlet:

[fs#1]
type = ocfs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ocfs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.qnx4

Minimal configlet:

[fs#1]
type = qnx4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=qnx4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.reiserfs

Minimal configlet:

[fs#1]
type = reiserfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=reiserfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.reiserfs4

Minimal configlet:

[fs#1]
type = reiserfs4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=reiserfs4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.smbfs

Minimal configlet:

[fs#1]
type = smbfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=smbfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.tmpfs

Minimal configlet:

[fs#1]
type = tmpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=tmpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.tux3

Minimal configlet:

[fs#1]
type = tux3
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=tux3" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ufs

Minimal configlet:

[fs#1]
type = ufs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ufs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ufs2

Minimal configlet:

[fs#1]
type = ufs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=ufs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.umsdos

Minimal configlet:

[fs#1]
type = umsdos
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=umsdos" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.vfat

Minimal configlet:

[fs#1]
type = vfat
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=vfat" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.vxfs

Minimal configlet:

[fs#1]
type = vxfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=vxfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.xfs

Minimal configlet:

[fs#1]
type = xfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=xfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.xia

Minimal configlet:

[fs#1]
type = xia
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=xia" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.zfs

Minimal configlet:

[fs#1]
type = zfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=zfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

quota

required:    false
scopable:    true

The dataset quota property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

refquota

required:    false
scopable:    true
default:     x1

The dataset refquota property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

refreservation

required:    false
scopable:    true

The dataset refreservation property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

reservation

required:    false
scopable:    true

The dataset reservation property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

Used by default as the refquota of the provisioned dataset.

The quota, refquota, reservation and refreservation values can be expressed as a multiplier of size (example: quota=x2).

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

ip.cni

Minimal configlet:

[ip#1]
type = cni

Minimal setup command:

om test/svc/foo set --kw="type=cni"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

expose

required:    false
scopable:    true
convert:     list

Example:

expose = 443/tcp:8443 53/udp

A whitespace-separated list of <port>/<protocol>[:<host port>] describing socket services that mandate a SRV exposition.

If <host port> is set, the ip.cni driver configures the port mappings.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

netns

required:    false
scopable:    true

Example:

netns = container#0

The resource id of the container to plumb the ip into.

network

required:    false
scopable:    true
default:     default

Example:

network = mynet

The name of the CNI network to plug into.

The default network is created using the host-local bridge plugin.

nsdev

required:    false
scopable:    true

Example:

nsdev = front

The interface name in the container namespace.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

wait_dns

required:    false
scopable:    true
default:     0
convert:     duration

Example:

wait_dns = 10s

Wait for the cluster DNS records associated to the resource to appear after a resource start and before the next resource can be started.

This can be set when apps or containers require the ip or ip name to be resolvable to provision or start properly.

ip.docker

Minimal configlet:

[ip#1]
type = docker
ipdev = eth0
netns = container#0

Minimal setup command:

om test/svc/foo set \
	--kw="type=docker" \
	--kw="ipdev=eth0" \
	--kw="netns=container#0"

alias

required:    false
scopable:    true
default:     true
convert:     bool

Use network interface stacking.

Modern ip stack support multiple ip addresses per interface, so alias should be set to false when possible.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_carrier

required:    false
scopable:    true
default:     true
convert:     bool

Activate the link carrier check.

Set to false if ipdev is a backend bridge or switch.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

del_net_route

required:    false
scopable:    true
default:     false
convert:     bool

Some docker ip configurations require dropping the network route autoconfigured when installing the ip address.

In this case set del_net_route=true and network=<cidr>.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

expose

required:    false
scopable:    true
convert:     list

Example:

expose = 443/tcp:8443 53/udp

A whitespace-separated list of <port>/<protocol>[:<host port>] describing socket services that mandate a SRV exposition.

gateway

required:    false
scopable:    true

The gateway to configure in the network namespace.

ipdev

required:    true
scopable:    true

Example:

ipdev = eth0

The interface name to setup the ip address on.

This interface can be different from one node to the other, in which case the ipdev@<nodename> scoping syntax can be used.

If the value is expressed as <intf>:<n>, the stacked interface index is forced to <n> instead of the lowest free index.

If the value is expressed as <name>@<intf>, a macvtap interface named <name> is created and attached to <intf>.

ipname

required:    false
scopable:    true

Example:

ipname = 1.2.3.4

The DNS name or IP address of the ip resource.

Can be different from one node to the other, in which case the ipname@<nodename> scoping syntax can be used.

This is most useful to specify a different ip when the service starts in DRP mode, where subnets are likely to be different than those of the production datacenter.

macaddr

required:    false
scopable:    true

Example:

macaddr = ce:32:cc:ca:41:33

The hardware address to set on the interface in the network namespace.

mode

required:    false
scopable:    true
candidates:  bridge, dedicated, macvlan, ipvlan-l2, ipvlan-l3, ipvlan-l3s, ovs
default:     bridge

Example:

mode = access

The ip link mode.

If ipdev is set to a bridge interface the mode defaults to bridge, else defaults to macvlan. The ipvlan mode requires a 4.2+ Linux kernel.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

netmask

required:    false
scopable:    true

Example:

netmask = 24

The netmask to configure with ipname.

If an ip is already plumbed on the root interface, the netmask default is the netmask of this existing ip.

netmask is mandatory for interfaces dedicated to the object. Dummy interfaces are likely to be in this case.

The format is:

dotted or octal for IPv4, ex: 255.255.252.0 or 22.
octal only for IPv6, ex: 64.

netns

required:    true
scopable:    true

Example:

netns = container#0

The resource id of the container to plumb the ip into.

network

required:    false
scopable:    true

Example:

network = 10.0.0.0/16

The ip address network, in dotted notation.

Used to set the network route if del_net_route=true.

nsdev

required:    false
scopable:    true

Example:

nsdev = front

The first eth<n> available in the network namespace.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

vlan_mode

required:    false
scopable:    true
candidates:  access, native-tagged, native-untagged
depends:     .mode=ovs
default:     native-untagged

Example:

vlan_mode = access

The VLAN port mode.

vlan_tag

required:    false
scopable:    true
depends:     .mode=ovs

Example:

vlan_tag = 44

The VLAN tag the switch port will relay.

wait_dns

required:    false
scopable:    true
default:     0
convert:     duration

Example:

wait_dns = 10s

Wait for the cluster DNS records associated to the resource to appear after a resource start and before the next resource can be started.

This can be set when apps or containers require the ip or ip name to be resolvable to provision or start properly.

ip.host

Minimal configlet:

[ip#1]
type = host
ipdev = eth0

Minimal setup command:

om test/svc/foo set \
	--kw="type=host" \
	--kw="ipdev=eth0"

alias

required:    false
scopable:    true
default:     true
convert:     bool

The DNS name or IP address of the ip resource.

Can be different from one node to the other, in which case the ipname@<nodename> scoping syntax can be used.

This is most useful to specify a different ip when the service starts in DRP mode, where subnets are likely to be different than those of the production datacenter.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_carrier

required:    false
scopable:    true
default:     true
convert:     bool

The DNS name or IP address of the ip resource.

Can be different from one node to the other, in which case the ipname@<nodename> scoping syntax can be used.

This is most useful to specify a different ip when the service starts in DRP mode, where subnets are likely to be different than those of the production datacenter.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

expose

required:    false
scopable:    true
convert:     list

Example:

expose = 443/tcp:8443 53/udp

The DNS name or IP address of the ip resource.

Can be different from one node to the other, in which case the ipname@<nodename> scoping syntax can be used.

This is most useful to specify a different ip when the service starts in DRP mode, where subnets are likely to be different than those of the production datacenter.

gateway

required:    false
scopable:    true

A zone ip provisioning parameter used in the sysidcfg formatting

ipdev

required:    true
scopable:    true

Example:

ipdev = eth0

The interface name to setup the ip address on.

This interface can be different from one node to the other, in which case the ipdev@<nodename> scoping syntax can be used.

If the value is expressed as <intf>:<n>, the stacked interface index is forced to <n> instead of the lowest free index.

If the value is expressed as <name>@<intf>, a macvtap interface named <name> is created and attached to <intf>.

ipname

required:    false
scopable:    true

Example:

ipname = 1.2.3.4

The DNS name or IP address of the ip resource.

Can be different from one node to the other, in which case the ipname@<nodename> scoping syntax can be used.

This is most useful to specify a different ip when the service starts in DRP mode, where subnets are likely to be different than those of the production datacenter.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

netmask

required:    false
scopable:    true

Example:

netmask = 24

The netmask to configure with ipname.

If an ip is already plumbed on the root interface, the netmask default is the netmask of this existing ip.

netmask is mandatory for interfaces dedicated to the object. Dummy interfaces are likely to be in this case.

The format is:

dotted or octal for IPv4, ex: 255.255.252.0 or 22.
octal only for IPv6, ex: 64.

network

required:    false
scopable:    true

Example:

network = 10.0.0.0/16

The ip address network, in dotted notation.

Used to set the network route if del_net_route=true.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

wait_dns

required:    false
scopable:    true
default:     0
convert:     duration

Example:

wait_dns = 10s

Wait for the cluster DNS records associated to the resource to appear after a resource start and before the next resource can be started.

This can be set when apps or containers require the ip or ip name to be resolvable to provision or start properly.

ip.netns

Minimal configlet:

[ip#1]
type = netns
ipdev = eth0
netns = container#0

Minimal setup command:

om test/svc/foo set \
	--kw="type=netns" \
	--kw="ipdev=eth0" \
	--kw="netns=container#0"

alias

required:    false
scopable:    true
default:     true
convert:     bool

Use network interface stacking.

Modern ip stack support multiple ip addresses per interface, so alias should be set to false when possible.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_carrier

required:    false
scopable:    true
default:     true
convert:     bool

Activate the link carrier check.

Set to false if ipdev is a backend bridge or switch.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

del_net_route

required:    false
scopable:    true
default:     false
convert:     bool

Some docker ip configurations require dropping the network route autoconfigured when installing the ip address.

In this case set del_net_route=true and network=<cidr>.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

expose

required:    false
scopable:    true
convert:     list

Example:

expose = 443/tcp:8443 53/udp

A whitespace-separated list of <port>/<protocol>[:<host port>] describing socket services that mandate a SRV exposition.

gateway

required:    false
scopable:    true

The gateway to configure in the network namespace.

ipdev

required:    true
scopable:    true

Example:

ipdev = eth0

The interface name to setup the ip address on.

This interface can be different from one node to the other, in which case the ipdev@<nodename> scoping syntax can be used.

If the value is expressed as <intf>:<n>, the stacked interface index is forced to <n> instead of the lowest free index.

If the value is expressed as <name>@<intf>, a macvtap interface named <name> is created and attached to <intf>.

ipname

required:    false
scopable:    true

Example:

ipname = 1.2.3.4

The DNS name or IP address of the ip resource.

Can be different from one node to the other, in which case the ipname@<nodename> scoping syntax can be used.

This is most useful to specify a different ip when the service starts in DRP mode, where subnets are likely to be different than those of the production datacenter.

macaddr

required:    false
scopable:    true

Example:

macaddr = ce:32:cc:ca:41:33

The hardware address to set on the interface in the network namespace.

mode

required:    false
scopable:    true
candidates:  bridge, dedicated, macvlan, ipvlan-l2, ipvlan-l3, ipvlan-l3s, ovs
default:     bridge

Example:

mode = access

The ip link mode.

If ipdev is set to a bridge interface the mode defaults to bridge, else defaults to macvlan. The ipvlan mode requires a 4.2+ Linux kernel.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

netmask

required:    false
scopable:    true

Example:

netmask = 24

The netmask to configure with ipname.

If an ip is already plumbed on the root interface, the netmask default is the netmask of this existing ip.

netmask is mandatory for interfaces dedicated to the object. Dummy interfaces are likely to be in this case.

The format is:

dotted or octal for IPv4, ex: 255.255.252.0 or 22.
octal only for IPv6, ex: 64.

netns

required:    true
scopable:    true

Example:

netns = container#0

The resource id of the container to plumb the ip into.

network

required:    false
scopable:    true

Example:

network = 10.0.0.0/16

The ip address network, in dotted notation.

Used to set the network route if del_net_route=true.

nsdev

required:    false
scopable:    true

Example:

nsdev = front

The first eth<n> available in the network namespace.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

vlan_mode

required:    false
scopable:    true
candidates:  access, native-tagged, native-untagged
depends:     .mode=ovs
default:     native-untagged

Example:

vlan_mode = access

The VLAN port mode.

vlan_tag

required:    false
scopable:    true
depends:     .mode=ovs

Example:

vlan_tag = 44

The VLAN tag the switch port will relay.

wait_dns

required:    false
scopable:    true
default:     0
convert:     duration

Example:

wait_dns = 10s

Wait for the cluster DNS records associated to the resource to appear after a resource start and before the next resource can be started.

This can be set when apps or containers require the ip or ip name to be resolvable to provision or start properly.

ip.route

Minimal configlet:

[ip#1]
type = route
gateway = 1.2.3.4
netns = container#0
to = 192.168.100.0/24

Minimal setup command:

om test/svc/foo set \
	--kw="type=route" \
	--kw="gateway=1.2.3.4" \
	--kw="netns=container#0" \
	--kw="to=192.168.100.0/24"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    false
scopable:    true
default:     Any first dev with an addr in the same network than the gateway.

Example:

dev = eth1

The network link to add the route on.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

gateway

required:    true
scopable:    true

Example:

gateway = 1.2.3.4

The gateway ip address.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

netns

required:    true
scopable:    true

Example:

netns = container#0

the resource id of the container to plumb the ip into.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

to

required:    true
scopable:    true

Example:

to = 192.168.100.0/24

The route destination node.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

route.envoy

Minimal configlet:

[route#1]
type = envoy

Minimal setup command:

om test/svc/foo set --kw="type=envoy"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

hash_policies

required:    false
scopable:    true
convert:     list

The list of hash policy resource ids for the route.

Honored if lb_policy is set to ring_hash or maglev.

match_case_sensitive

required:    false
scopable:    true
default:     true
convert:     bool

Indicates that prefix/path matching should be case sensitive.

The default is true.

match_path

required:    false
scopable:    true

If specified, the route is an exact path rule meaning that the path must exactly match the :path header once the query string is removed.

Precisely one of prefix, path, regex must be set.

match_prefix

required:    false
scopable:    true

If specified, the route is a prefix rule meaning that the prefix must match the beginning of the :path header.

Precisely one of prefix, path, regex must be set.

match_regex

required:    false
scopable:    true

Example:

match_regex = /b[io]t

If specified, the route is a regular expression rule meaning that the regex must match the :path header once the query string is removed. The entire path (without the query string) must match the regex.

The rule will not match if only a subsequence of the :path header matches the regex.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

redirect_host_redirect

required:    false
scopable:    true

The host portion of the URL will be swapped with this value.

redirect_https_redirect

required:    false
scopable:    true
convert:     bool

The scheme portion of the URL will be swapped with https.

redirect_path_redirect

required:    false
scopable:    true

Indicates that the route is a redirect rule.

If there is a match, a 301 redirect response will be sent which swaps the path portion of the URL with this value.

The host_redirect keyword can also be set alongside redirect_path_redirect.

redirect_prefix_rewrite

required:    false
scopable:    true

Indicates that during redirection, the matched prefix (or path) should be swapped with this value.

This option allows redirect URLs dynamically created based on the request.

redirect_response_code

required:    false
scopable:    true

The HTTP status code to use in the redirect response.

The default response code is MOVED_PERMANENTLY (301).

redirect_strip_query

required:    false
scopable:    true
convert:     bool

Indicates that during redirection, the query portion of the URL will be removed.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

route_cluster_header

required:    false
scopable:    true

If the route is not a redirect (host_redirect and/or path_redirect are not specified), one of cluster, cluster_header, or weighted_clusters must be specified.

When cluster_header is specified, Envoy will determine the cluster to route to by reading the value of the HTTP header named by cluster_header from the request headers.

If the header is not found or the referenced cluster does not exist, Envoy will return a 404 response.

route_host_rewrite

required:    false
scopable:    true

Indicates that during forwarding, the host header will be swapped with this value.

route_prefix_rewrite

required:    false
scopable:    true

The string replacing the url path prefix if matching.

route_timeout

required:    false
scopable:    true
default:     15s
convert:     duration

Specifies the timeout for the route.

Note: This timeout includes all retries.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

share.nfs

Minimal configlet:

[share#1]
type = nfs
opts = *(ro)
path = /srv/{fqdn}/share

Minimal setup command:

om test/svc/foo set \
	--kw="type=nfs" \
	--kw="opts=*(ro)" \
	--kw="path=/srv/{fqdn}/share"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

opts

required:    true
scopable:    true

Example:

opts = *(ro)

The NFS share export options.

On Linux, as they would be set in /etc/exports

On Solaris, as they would be passed to the share command.

path

required:    true
scopable:    true

Example:

path = /srv/{fqdn}/share

The path of the directory to share.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

parallel

required:    false
scopable:    true
convert:     bool

If set to true, actions are executed in parallel on the resource of this subset.

For example, a service with:

[app#1]
start = /bin/true

[app#2]
start = /bin/true

[app#3]
start = /bin/true
subset = workers

[app#4]
start = /bin/true
subset = workers

[subset#app:workers]
parallel = true

Would have a start sequence like:

app#1 ---> app#2 -+-> app#3
                  `-> app#4

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.rsync

Minimal configlet:

[sync#1]
type = rsync

Minimal setup command:

om test/svc/foo set --kw="type=rsync"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

bwlimit

required:    false
scopable:    false

Bandwidth limit (the default unit is kb/s) applied to this rsync transfer.

Leave empty to enforce no limit.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dst

required:    false
scopable:    true

Example:

dst = /srv/{fqdn}

The destination of the sync.

Can be a whitespace-separated list of files or directories passed untamed to rsync.

Attention: The ending '/' is meaningful. Refer to the rsync man page for details.

dstfs

required:    false
scopable:    true

Example:

dstfs = /srv/{fqdn}

If set to a remote mount point, OpenSVC will verify that the specified mount point is really hosting a mounted FS.

This is a safety net to prevent overflowing the parent filesystem (/ for example).

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

options

required:    false
scopable:    true
convert:     shlex

Example:

options = --acls --xattrs --exclude foo/bar

A whitespace-separated list of params passed unchanged to rsync.

Typical usage is ACL preservation activation.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

reset_options

required:    false
scopable:    false
convert:     bool

Use options only instead of merging options to default hardcoded options.

This keyword can be used to disable --xattr or --acls for example.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

snap

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to snapshot the first snapshottable parent of the src and replicate from the snap instead of the src.

src

required:    false
scopable:    true

Example:

src = /srv/{fqdn}/

The source of the sync.

Can be a whitespace-separated list of files or directories passed untamed to rsync.

Attention: The ending '/' is meaningful. Refer to the rsync man page for details.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

target

required:    false
scopable:    true
candidates:  nodes, drpnodes
convert:     list

Which nodes should receive this data sync from the PRD node where the instance is up and running.

A shared filesystem (shared disk, replicated disk, clustered fs or networked fs) should not have a rsync target containing nodes where the fs resource can be started.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the sync action a failure.

If no timeout is set, the agent waits indefinitely for the sync action to exit.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.symsnapvx

Minimal configlet:

[sync#1]
type = symsnapvx
symid = 0000001234

Minimal setup command:

om test/svc/foo set \
	--kw="type=symsnapvx" \
	--kw="symid=0000001234"

absolute

required:    false
scopable:    true

Example:

absolute = 12:15

Use -absolute in symsnapvx commands.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

delta

required:    false
scopable:    true

Example:

delta = 00:15

Use -delta in symsnapvx commands.

devs

required:    false
scopable:    true
convert:     list

Example:

devs = 012a 012b

The whitespace-separated list of symmetrix device identifiers.

devs_from

required:    false
scopable:    true
convert:     list

Example:

devs_from = disk#0 disk#1

Use the pointed resource subdevices as the list of devices to snapshot.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true

Example:

name = prod_db1_weekly

Use -name in symsnapvx commands.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secure

required:    false
scopable:    true
convert:     bool

Use -secure in symsnapvx commands.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

symid

required:    true
scopable:    true

Example:

symid = 0000001234

The symmetrix array identifier.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.symsrdfs

Minimal configlet:

[sync#1]
type = symsrdfs
symdg = prod_db1
symid = 0000001234

Minimal setup command:

om test/svc/foo set \
	--kw="type=symsrdfs" \
	--kw="symdg=prod_db1" \
	--kw="symid=0000001234"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

rdfg

required:    false
scopable:    false
convert:     int

Example:

rdfg = 5

Id of the RDF group pairing the source and target devices.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

symdg

required:    true
scopable:    false

Example:

symdg = prod_db1

Name of the symmetrix device group where the source and target devices are grouped.

symid

required:    true
scopable:    true

Example:

symid = 0000001234

The symid of the local Symmetrix array hosting the symdg. This parameter must be scoped to specify which array each node can access.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.zfs

Minimal configlet:

[sync#1]
type = zfs
dst = pool/{fqdn}
src = pool/{fqdn}

Minimal setup command:

om test/svc/foo set \
	--kw="type=zfs" \
	--kw="dst=pool/{fqdn}" \
	--kw="src=pool/{fqdn}"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dst

required:    true
scopable:    true

Example:

dst = pool/{fqdn}

Destination dataset of the sync.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

intermediary

required:    false
scopable:    true
default:     true
convert:     bool

Synchronize the snapshots of the selected datasets that were created since the last update.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

recursive

required:    false
scopable:    true
default:     true
convert:     bool

Which nodes should receive this data sync from the PRD node where the instance is up and running.

A shared filesystem (shared disk, replicated disk, clustered fs or networked fs) should not have a rsync target containing nodes where the fs resource can be started.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

src

required:    true
scopable:    true

Example:

src = pool/{fqdn}

Source dataset of the sync.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

target

required:    false
scopable:    true
candidates:  nodes, drpnodes
convert:     list

Which nodes should receive this data sync from the PRD node where the instance is up and running.

A shared filesystem (shared disk, replicated disk, clustered fs or networked fs) should not have a rsync target containing nodes where the fs resource can be started.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the sync action a failure.

If no timeout is set, the agent waits indefinitely for the sync action to exit.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.zfssnap

Minimal configlet:

[sync#1]
type = zfssnap
dataset = svc1fs/data svc1fs/log

Minimal setup command:

om test/svc/foo set \
	--kw="type=zfssnap" \
	--kw="dataset=svc1fs/data svc1fs/log"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dataset

required:    true
scopable:    true
convert:     list

Example:

dataset = svc1fs/data svc1fs/log

A whitespace separated list of datasets to snapshot.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

keep

required:    false
scopable:    true
default:     3
convert:     int

Example:

keep = 3

The maximum number of snapshots to retain.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true

Example:

name = weekly

A name included in the snapshot name to avoid retention conflicts between multiple zfs snapshot resources.

A full snapshot name is formatted as <subvol>.<name>.snap.<datetime>.

Example: data.weekly.snap.2016-03-09.10:09:52

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

recursive

required:    false
scopable:    true
default:     true
convert:     bool

Set to true to snap recursively the datasets.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

task.docker

Minimal configlet:

[task#1]
type = docker
image = ghcr.io/opensvc/pause

Minimal setup command:

om test/svc/foo set \
	--kw="type=docker" \
	--kw="image=ghcr.io/opensvc/pause"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    true
scopable:    true

Example:

image = ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

The PID file is normally removed when the task execution ends, but if the executor dies abruptly (e.g., due to a SIGKILL), the stale PID file is not considered when computing the resource status. It is removed before the count check of the next run.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

If not set, the container will have a private userns other containers can share.

A container with userns=host will share the host's userns.

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

task.host

Minimal configlet:

[task#1]
type = host

Minimal setup command:

om test/svc/foo set --kw="type=host"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true

The shlex expression of the command to execute on run actions.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the process execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Change the working directory to the specified location instead of the default <pathtmp>.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = CRT=cert1/server.crt PEM=cert1/server.pem

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

group

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified group instead of root.

limit_as

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes) (same as limit_vmem).

When both limit_vmem and limit_as is used, the max value is chosen.

limit_core

required:    false
scopable:    true
convert:     size

The limit on the largest core dump size that can be produced (unit byte).

limit_cpu

required:    false
scopable:    true
convert:     duration

Example:

limit_cpu = 30s

The limit on CPU time (duration).

limit_data

required:    false
scopable:    true
convert:     size

The limit on the data segment size of a process (unit byte).

limit_fsize

required:    false
scopable:    true
convert:     size

The limit on the largest file that can be created (unit byte).

limit_memlock

required:    false
scopable:    true
convert:     size

The limit on how much memory a process can lock with mlock(2) (unit byte, no solaris support).

limit_nofile

required:    false
scopable:    true
convert:     size

The limit on the number files a process can have open at once.

limit_nproc

required:    false
scopable:    true
convert:     size

The limit on the number of processes this user can have at one time, no solaris support.

limit_rss

required:    false
scopable:    true
convert:     size

The limit on the total physical memory that can be in use by a process (unit byte, no solaris support).

limit_stack

required:    false
scopable:    true
convert:     size

The limit on the stack size of a process (unit bytes).

limit_vmem

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes).

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the process execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop

required:    false
scopable:    true

true

Execute the script command with stop argument on stop action.
false

Do nothing on stop action.
<shlex expression>

Execute this command on stop action.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 180

Wait for <duration> before declaring the app launcher stop action a failure.

Takes precedence over timeout.

If neither timeout nor stop_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance stop when an app launcher did not return.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

type

required:    false
scopable:    false

The resource driver name.

umask

required:    false
scopable:    true
convert:     umask

Example:

umask = 022

The umask to set for the application process.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified user instead of root.

task.oci

Minimal configlet:

[task#1]
type = oci
image = ghcr.io/opensvc/pause

Minimal setup command:

om test/svc/foo set \
	--kw="type=oci" \
	--kw="image=ghcr.io/opensvc/pause"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    true
scopable:    true

Example:

image = ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

If not set, the container will have a private userns other containers can share.

A container with userns=host will share the host's userns.

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

task.podman

Minimal configlet:

[task#1]
type = podman
image = ghcr.io/opensvc/pause

Minimal setup command:

om test/svc/foo set \
	--kw="type=podman" \
	--kw="image=ghcr.io/opensvc/pause"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    true
scopable:    true

Example:

image = ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

Defines the podman container run --userns value.

the 'container#...' values are converted to container:id

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

vhost.envoy

Minimal configlet:

[vhost#1]
type = envoy

Minimal setup command:

om test/svc/foo set --kw="type=envoy"

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

domains

required:    false
scopable:    true
default:     {name}
convert:     list

Example:

domains = {name}

The list of http domains in this expose.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

routes

required:    false
scopable:    true
convert:     list

Example:

routes = route#1 route#2

The list of route resource identifiers for this vhost.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

volume

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs

required:    false
scopable:    true
convert:     shlex

Example:

configs = conf/mycnf:/etc/mysql/my.cnf:ro conf/sysctl:/etc/sysctl.d/01-db.conf

The whitespace-separated list of <config name>/<key>:<volume relative path>:<options>.

directories

required:    false
scopable:    true
convert:     list

Example:

directories = a/b/c d /e

The whitespace-separated list of directories to create in the vol head.

dirperm

required:    false
scopable:    true
default:     700
convert:     file-mode

Example:

dirperm = 750

The permissions, in octal notation, to apply to the volume root and installed directories.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

format

required:    false
scopable:    true
default:     true
convert:     bool

If true, the pool's vol configuration producer will include a fs resource layered over the disk resource.

group

required:    false
scopable:    true

Example:

group = 1001

The group name or id that will own the volume root and installed files and directories.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     {name}-vol-{rindex}

The vol name.

An object can only reference a vol in the same namespace.

nodes

required:    false
scopable:    true
default:     {.nodes}
convert:     nodes

A node selector expression filtering the creator nodes to determine the volume nodes.

If not set, all the creator nodes will be volume nodes.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 660

The permissions, in octal notation, to apply to the installed files.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool to allocate from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

secrets

required:    false
scopable:    true
convert:     shlex

Example:

secrets = cert/pem:server.pem cert/key:server.key

The whitespace-separated list of <secret name>/<key>:<volume relative path>:<options>.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

signal

required:    false
scopable:    true

Example:

signal = hup:container#1

A <signal>:<target> whitespace-separated list, where <signal> is a signal name or number (ex. 1, hup or sighup), and target is the comma-separated list of resource ids to send the signal to (ex: container#1,container#2).

If only the signal is specified, all candidate resources will be signaled.

This keyword is typically used to reload daemons on certificate or configuration files changes.

size

required:    false
scopable:    true
convert:     size

The size to allocate in the pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    true

The type of the pool to allocate the vol from.

The selected pool will be the one matching type and capabilities and with the maximum available space.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = 1001

The user name or id that will own the volume root and installed files and directories.

DEFAULT

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

app

required:    false
scopable:    false
default:     default

A user-defined code linking to:

who is responsible for this service.
who is billable.

This code thus provides a most useful object grouping and filtering key.

Short and simple codes, like ERP, are easier to work with.

children

required:    false
scopable:    false
convert:     list-lowercase

The list of services or instances expressed as <path>[@<nodename>] that must be down or stdby up to allow this service to be stopped by the daemon.

The list is whitespace-separated.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

comp_schedule

required:    false
scopable:    true
default:     ~00:00-06:00

The instance compliance run schedule.

See usr/share/doc/schedule for the schedule syntax.

create_pg

required:    false
scopable:    true
default:     true
convert:     bool

Use process grouping when possible.

If turned on, the agent will create a container group for:

the object
each resource group (ie, the subset:drivergroup tuple)
each resource

A container group allows capping the memory, swap and cpu usage. These cappings can be defined using the pg_* keywords in the DEFAULT, the subset or the resource section.

disable

required:    false
scopable:    true
convert:     bool

Disables the object instance, which has the following effects:

The instance status and the status of all its resource is n/a.
Stop and start actions have no effect, and not produce error.
Disabled resources are not enabled when DEFAULT.disable=false.

drpnodes

required:    false
scopable:    true
convert:     other-nodes

Example:

drpnodes = n1 n2

A node selector expression specifying the list of cluster nodes hosting object instances when all primary nodes are unavailable, like in a DRP situation.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

The drpnodes can be data synchronization targets for sync resources.

env

required:    false
scopable:    false
default:     The same as the node `env`.

A code like PRD, DEV, etc... the agent can use to enforce data protection policies:

A non-PRD object instance can not be started on a PRD node
A PRD object instance can be started on a non-PRD node (typically in a DRP situation)

The default value is read from the node env keyword.

flex_max

required:    false
scopable:    false
depends:     topology=flex
default:     The number of elements in `nodes`.

convert:     int

The maximum number of up instances of this object in the cluster. Above this number the aggregated object status is degraded to warn.

The 0 value is interpreted as unlimited.

flex_min

required:    false
scopable:    false
depends:     topology=flex
default:     1
convert:     int

The minimum number of up instances of this object in the cluster. Below this number the aggregated object status is degraded to warn.

flex_primary

required:    false
scopable:    true
depends:     topology=flex
default:     The first node of `nodes`.

convert:     list-lowercase

The node in charge of syncing the other nodes in a flex object.

flex_target

required:    false
scopable:    false
depends:     topology=flex
default:     The value of `flex_min`.

convert:     int

The optimal number of up instances of the object in the cluster. The value must be between flex_min and flex_max.

If orchestrate=ha, the daemon is free to take action to reach the flex_target.

hard_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

hard_affinity = svc1 svc2

A whitespace separated list of object paths.

These objects must be started on the local node to allow the local monitor to start an instance of the service.

hard_anti_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

hard_anti_affinity = svc1 svc2

A whitespace separated list of object paths.

These object must not be started on the local node to allow the local monitor to start an instance of the object.

id

required:    false
scopable:    false
default:     A random generated UUID.

A rfc4122 random uuid generated by the agent.

monitor_action

required:    false
scopable:    true
candidates:  crash, freezestop, none, reboot, switch, no-op
default:     none
convert:     list

Example:

monitor_action = reboot

The action to trigger when a monitored resource is no longer in the "up" or "standby up" state, and all restart attempts for the resource have failed.

The reboot and crash monitor actions do not attempt to cleanly stop any processes. On Linux, they utilize system-level sysrq triggers.

You can append a fallback monitor action to this keyword. A common example is freezestop reboot. In this case, the reboot action will be executed if the stop fails or times out.

Other monitor_actions values:

none: the default value for monitor action disabled (monitorkeyword must be also false or undefined).
freezestop: freeze and subsequently stop the monitored instance.
switch: try monitored instance stop to allow any other cluster nodes to takeover the instance.
no-op: The monitor action No Operation is called but does nothing. It may be used for demonstration. The final local expect after call will be set to evicted.

monitor_schedule

required:    false
scopable:    true
default:     @5m

The instance monitored resources status evaluation schedule.

See usr/share/doc/schedule for the schedule syntax.

nodes

required:    false
scopable:    true
default:     The lowercased hostname of the evaluating node.

convert:     nodes

Example:

nodes = n1 n*

A node selector expression specifying the list of cluster nodes hosting object instances.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

parents

required:    false
scopable:    false
convert:     list-lowercase

The list of services or instances expressed as <path>[@<nodename>] that must be up to allow this service to be started by the daemon.

The list is whitespace-separated.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

pre_monitor_action

required:    false
scopable:    true

Example:

pre_monitor_action = /bin/true

A callout to execute before the monitor_action.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

resinfo_schedule

required:    false
scopable:    true
default:     @60m

The instance key-val table emit schedule.

See usr/share/doc/schedule for the schedule syntax.

rollback

required:    false
scopable:    true
default:     true
convert:     bool

If set to false, the default rollback on start action error behaviour is disabled, leaving the instance in its half-started state (avail warn).

The daemon then refuses to failover a service if any instance is in warn availabity state. It is highly recommended to not use rollback=false if orchestrate=ha.

run_schedule

required:    false
scopable:    true

The instance tasks run action default schedule.

See usr/share/doc/schedule for the schedule syntax.

shared

required:    false
scopable:    true
default:     true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

soft_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

soft_affinity = svc1 svc2

A whitespace separated list of services that must be started on the node to allow the monitor to start this service.

If the local node is the only candidate ignore this constraint and allow start.

soft_anti_affinity

required:    false
scopable:    false
convert:     list-lowercase

Example:

soft_anti_affinity = svc1 svc2

A whitespace separated list of services that must not be started on the node to allow the monitor to start this service.

If the local node is the only candidate ignore this constraint and allow start.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_schedule

required:    false
scopable:    true
default:     @10m

The instance status evaluation schedule.

See usr/share/doc/schedule for the schedule syntax.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stonith

required:    false
scopable:    false
depends:     topology=failover
default:     false
convert:     bool

Shoot The Other Node In The Head, aka fence, using a callout.

The callout is triggered after a quorum vote won, when the surviving node is about to start a local instance of a service that was known to be started on a unreachable peer node.

The callout is meant to prevent the peer from writing to shared disks, remote databases, and from responding to clients.

The Fence Agents project is a well known bundle of callout used by many clustering tools.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync_schedule

required:    false
scopable:    true
default:     04:00-06:00

The instance sync default schedule.

See usr/share/doc/schedule for the schedule syntax.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

topology

required:    false
scopable:    false
candidates:  failover, flex
default:     failover

failover

The service is allowed to be up on one node at a time.
flex

The service can be up on flex_target nodes, where flex_target must be in the [flex_min, flex_max] range.

type

required:    false
scopable:    false

The resource driver name.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.crypt

Minimal configlet:

[disk#1]
type = crypt
dev = /dev/{fqdn}/lv1

Minimal setup command:

om test/vol/foo set \
	--kw="type=crypt" \
	--kw="dev=/dev/{fqdn}/lv1"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/{fqdn}/lv1

The fullpath of the underlying block device.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

label

required:    false
scopable:    true
default:     {fqdn}

The label to set in the cryptsetup metadata written on dev.

A label helps admin understand the role of a device.

manage_passphrase

required:    false
scopable:    true
default:     true
convert:     bool

By default, on provision the driver allocates a new random passphrase (256 printable chars), and forgets it on unprovision.

If set to false, require a passphrase to be already present in the sec object to provision, and don't remove it on unprovision.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     The basename of the underlying device, suffixed with `-crypt`.

Example:

name = {fqdn}-crypt

The basename of the exposed device.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

secret

required:    false
scopable:    true
default:     {name}

The name of the sec object hosting the crypt secrets.

The sec object must be in the same namespace than the object defining the disk.crypt resource.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.disk

Minimal configlet:

[disk#1]
type = disk

Minimal setup command:

om test/vol/foo set --kw="type=disk"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

array

required:    false
scopable:    true

Example:

array = xtremio-prod1

The array to provision the disk from.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

disk_id

required:    false
scopable:    true

Example:

disk_id = 6589cfc00000097484f0728d8b2118a6

The wwn of the disk.

diskgroup

required:    false
scopable:    true

Example:

diskgroup = default

The array disk group to provision the disk from.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true

Example:

name = myfcdisk1

The name of the disk.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

Example:

pool = fcpool1

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

Example:

size = 20g

A size expression for the disk allocation.

slo

required:    false
scopable:    true

Example:

slo = Optimized

The provisioned disk service level objective.

This keyword is honored on arrays supporting this notion (ex: EMC VMAX).

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.drbd

Minimal configlet:

[disk#1]
type = drbd

Minimal setup command:

om test/vol/foo set --kw="type=drbd"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

addr

required:    false
scopable:    true
default:     The ipaddr resolved for the nodename.

Example:

addr = 1.2.3.4

The addr to use to connect a peer. Use scoping to define each non-default address.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

disk

required:    false
scopable:    true

Example:

disk = /dev/vg1/lv1

The path of the device to provision the drbd on.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_peers

required:    false
scopable:    false
default:     (nodes_count*2)-1

convert:     int

Example:

max_peers = 8

The integer value to use in create-md --max-peers <n>.

The driver ensures the value is not lesser than the number of instances.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

network

required:    false
scopable:    false

Example:

network = benet1

The name of the backend network to use for drbd traffic.

Set this keyword if some node names are resolved to NATed addresses.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

port

required:    false
scopable:    true
convert:     int

Example:

port = 1.2.3.4

The port to use to connect a peer.

Auto-allocated on provision if not already defined.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

res

required:    false
scopable:    false

Example:

res = r1

The name of the drbd resource associated with this service resource.

OpenSVC expects the resource configuration file to reside in /etc/drbd.d/<res>.res and takes care of its replication on peer nodes.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.loop

Minimal configlet:

[disk#1]
type = loop
file = /srv/{fqdn}-loop-{rindex}

Minimal setup command:

om test/vol/foo set \
	--kw="type=loop" \
	--kw="file=/srv/{fqdn}-loop-{rindex}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

file

required:    true
scopable:    true

Example:

file = /srv/{fqdn}-loop-{rindex}

The loopback device backing file full path.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true

Example:

size = 100m

The size of the loop file to provision.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.lv

Minimal configlet:

[disk#1]
type = lv
name = lv1
vg = vg1

Minimal setup command:

om test/vol/foo set \
	--kw="type=lv" \
	--kw="name=lv1" \
	--kw="vg=vg1"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_options

required:    false
scopable:    true
convert:     shlex

Example:

create_options = --contiguous y

Additional options to pass to the logical volume create command (lvcreate or vxassist, depending on the driver).

The size and name are managed natively.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = lv1

The name of the logical volume.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true

Example:

size = 10m

The size of the logical volume to provision. A size expression or <n>%{FREE|PVS|VG}.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

vg

required:    true
scopable:    true

Example:

vg = vg1

The name of the volume group hosting the logical volume.

disk.lvm

Minimal configlet:

[disk#1]
type = lvm
name = vg1

Minimal setup command:

om test/vol/foo set \
	--kw="type=lvm" \
	--kw="name=vg1"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = vg1

The name of the logical volume group.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

options

required:    false
scopable:    true
convert:     shlex

Example:

options = --zero=y

The vgcreate options to use upon vg provisioning.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pvs

required:    false
scopable:    true
convert:     list

Example:

pvs = /dev/mapper/23 /dev/mapper/24

The list of paths to the physical volumes of the volume group.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.md

Minimal configlet:

[disk#1]
type = md

Minimal setup command:

om test/vol/foo set --kw="type=md"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

chunk

required:    false
scopable:    true
convert:     size

Example:

chunk = 128k

The md chunk size to use with the mdadm create command.

The value is adjusted to the first greater or equal multiple of 4.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

devs

required:    false
scopable:    true
convert:     list

Example:

devs = /dev/mapper/23 /dev/mapper/24

The md member devices to use with the mdadm create command.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

level

required:    false
scopable:    true

Example:

level = raid1

The md raid level to use with the mdadm create command.

See man mdadm for the list of supported values.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

spares

required:    false
scopable:    true
default:     0
convert:     int

Example:

spares = 1

The md number of spare devices to use with the mdadm create command.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

uuid

required:    false
scopable:    true

Example:

uuid = dev1

The md uuid to use with the mdadm assemble command.

disk.raw

Minimal configlet:

[disk#1]
type = raw
devs = /dev/mapper/svc.d0:/dev/oracle/redo001 /dev/mapper/svc.d1

Minimal setup command:

om test/vol/foo set \
	--kw="type=raw" \
	--kw="devs=/dev/mapper/svc.d0:/dev/oracle/redo001 /dev/mapper/svc.d1"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_char_devices

required:    false
scopable:    true
default:     true
convert:     bool

Example:

create_char_devices = false

On Linux, char devices are not automatically created when devices are discovered.

If set to true (the default), the raw resource driver will create and delete them using the raw kernel driver.

devs

required:    true
scopable:    true
convert:     list

Example:

devs = /dev/mapper/svc.d0:/dev/oracle/redo001 /dev/mapper/svc.d1

A whitespace-separated list of device paths or <src>[:<dst>] device paths mappings.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should own the device. Either in numeric or symbolic form.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 600

The permissions the device should have. A string representing the octal permissions.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should own the device. Either in numeric or symbolic form.

zone

required:    false
scopable:    true

Example:

zone = zone1

The zone name the raw resource is linked to.

If set, the raw files are configured from the global reparented to the zonepath.

disk.vg

Minimal configlet:

[disk#1]
type = vg
name = vg1

Minimal setup command:

om test/vol/foo set \
	--kw="type=vg" \
	--kw="name=vg1"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = vg1

The name of the logical volume group.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

options

required:    false
scopable:    true
convert:     shlex

Example:

options = --zero=y

The vgcreate options to use upon vg provisioning.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pvs

required:    false
scopable:    true
convert:     list

Example:

pvs = /dev/mapper/23 /dev/mapper/24

The list of paths to the physical volumes of the volume group.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

disk.zpool

Minimal configlet:

[disk#1]
type = zpool
name = tank

Minimal setup command:

om test/vol/foo set \
	--kw="type=zpool" \
	--kw="name=tank"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_options

required:    false
scopable:    true
convert:     shlex

Example:

create_options = -O dedup=on

The zone name the zpool refers to. If set, the zpool is activated in the zone context.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

multihost

required:    false
scopable:    true
convert:     tristate

Example:

multihost = yes

If true, set the zfs property multihost=on on start if not already set. This requires all nodes to be booted with a /etc/hostid installed, preferably generated by the zgenhostid command.

If false, set the zfs property multihost=off.

If left empty, the current multihost property is left unchanged.

name

required:    true
scopable:    true

Example:

name = tank

The name of the zfs pool.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

vdev

required:    false
scopable:    true
convert:     list

Example:

vdev = /dev/mapper/23 /dev/mapper/24

The vdev list, including optional parity keywords, as would be passed to zpool create.

zone

required:    false
scopable:    true

The zone name the zpool refers to. If set, the zpool is activated in the zone context.

disk.zvol

Minimal configlet:

[disk#1]
type = zvol
name = tank/zvol1

Minimal setup command:

om test/vol/foo set \
	--kw="type=zvol" \
	--kw="name=tank/zvol1"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

blocksize

required:    false
scopable:    true
convert:     size

Example:

blocksize = 256k

The blocksize of the zfs volume to create.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

create_options

required:    false
scopable:    true
convert:     shlex

Example:

create_options = -o dedup=on

The zfs create -V <name> extra options.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    true
scopable:    true

Example:

name = tank/zvol1

The full name of the zfs volume in the <pool>/<name> form.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the exposed devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

Example:

size = 10m

The size of the zfs volume to create.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

fs.9pfs

Minimal configlet:

[fs#1]
type = 9pfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=9pfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.afs

Minimal configlet:

[fs#1]
type = afs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=afs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.bfs

Minimal configlet:

[fs#1]
type = bfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=bfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.bind

Minimal configlet:

[fs#1]
type = bind
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=bind" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.btrfs

Minimal configlet:

[fs#1]
type = btrfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=btrfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.cephfs

Minimal configlet:

[fs#1]
type = cephfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=cephfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.cifs

Minimal configlet:

[fs#1]
type = cifs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=cifs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.directory

Minimal configlet:

[fs#1]
type = directory
path =

Minimal setup command:

om test/vol/foo set \
	--kw="type=directory" \
	--kw="path="

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the directory. Either in numeric or symbolic form.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

path

required:    true
scopable:    true

The fullpath of the directory to create.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The permissions the directory should have. A string representing the octal permissions.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the directory. Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to. If set, the fs mount point is reparented into the zonepath rootfs.

fs.ext2

Minimal configlet:

[fs#1]
type = ext2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ext2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ext3

Minimal configlet:

[fs#1]
type = ext3
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ext3" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ext4

Minimal configlet:

[fs#1]
type = ext4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ext4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.f2fs

Minimal configlet:

[fs#1]
type = f2fs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=f2fs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.flag

Minimal configlet:

[fs#1]
type = flag

Minimal setup command:

om test/vol/foo set --kw="type=flag"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

fs.gfs

Minimal configlet:

[fs#1]
type = gfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=gfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.gfs2

Minimal configlet:

[fs#1]
type = gfs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=gfs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.glusterfs

Minimal configlet:

[fs#1]
type = glusterfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=glusterfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.gpfs

Minimal configlet:

[fs#1]
type = gpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=gpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.hfs

Minimal configlet:

[fs#1]
type = hfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=hfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.hfsplus

Minimal configlet:

[fs#1]
type = hfsplus
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=hfsplus" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.hpfs

Minimal configlet:

[fs#1]
type = hpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=hpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jffs

Minimal configlet:

[fs#1]
type = jffs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=jffs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jffs2

Minimal configlet:

[fs#1]
type = jffs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=jffs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jfs

Minimal configlet:

[fs#1]
type = jfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=jfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.jfs2

Minimal configlet:

[fs#1]
type = jfs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=jfs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.lofs

Minimal configlet:

[fs#1]
type = lofs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=lofs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.logfs

Minimal configlet:

[fs#1]
type = logfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=logfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.minix

Minimal configlet:

[fs#1]
type = minix
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=minix" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.msdos

Minimal configlet:

[fs#1]
type = msdos
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=msdos" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ncpfs

Minimal configlet:

[fs#1]
type = ncpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ncpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.nfs

Minimal configlet:

[fs#1]
type = nfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=nfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.nfs4

Minimal configlet:

[fs#1]
type = nfs4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=nfs4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.nilfs

Minimal configlet:

[fs#1]
type = nilfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=nilfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.none

Minimal configlet:

[fs#1]
type = none
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=none" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ntfs

Minimal configlet:

[fs#1]
type = ntfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ntfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ocfs

Minimal configlet:

[fs#1]
type = ocfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ocfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ocfs2

Minimal configlet:

[fs#1]
type = ocfs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ocfs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.qnx4

Minimal configlet:

[fs#1]
type = qnx4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=qnx4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.reiserfs

Minimal configlet:

[fs#1]
type = reiserfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=reiserfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.reiserfs4

Minimal configlet:

[fs#1]
type = reiserfs4
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=reiserfs4" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.smbfs

Minimal configlet:

[fs#1]
type = smbfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=smbfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.tmpfs

Minimal configlet:

[fs#1]
type = tmpfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=tmpfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.tux3

Minimal configlet:

[fs#1]
type = tux3
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=tux3" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ufs

Minimal configlet:

[fs#1]
type = ufs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ufs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.ufs2

Minimal configlet:

[fs#1]
type = ufs2
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=ufs2" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.umsdos

Minimal configlet:

[fs#1]
type = umsdos
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=umsdos" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.vfat

Minimal configlet:

[fs#1]
type = vfat
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=vfat" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.vxfs

Minimal configlet:

[fs#1]
type = vxfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=vxfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.xfs

Minimal configlet:

[fs#1]
type = xfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=xfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.xia

Minimal configlet:

[fs#1]
type = xia
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=xia" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check_read

required:    false
scopable:    true
convert:     bool

Activate file system read check during status evaluation when the file system is mounted but file system write check is disabled.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

no_preempt_abort

required:    false
scopable:    true
convert:     bool

If true, the agent will preempt the scsi3 persistent reservation with a preempt command instead of a preempt and and abort.

Some scsi target implementations do not support preempt and and abort (esx).

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

prkey

required:    false
scopable:    true

A specific scsi3 persistent reservation key for the resource.

It overrides the object-level prkey and the node-level prkey.

promote_rw

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to promote the base devices to read-write on start actions.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

scsireserv

required:    false
scopable:    false
convert:     bool

If true, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource.

Existing reservations are preempted to not block service failover.

If the start was not legitimate the data are still protected from being written concurrently from all nodes.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

fs.zfs

Minimal configlet:

[fs#1]
type = zfs
dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8
mnt = /srv/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=zfs" \
	--kw="dev=/dev/disk/by-id/nvme-eui.002538ba11b75ec8" \
	--kw="mnt=/srv/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dev

required:    true
scopable:    true

Example:

dev = /dev/disk/by-id/nvme-eui.002538ba11b75ec8

The block device file or filesystem image file hosting the filesystem to mount.

A different device can be set up on different nodes using the dev@<nodename> scoping syntax.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

group

required:    false
scopable:    true
convert:     group

Example:

group = sys

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

mkfs_opt

required:    false
scopable:    true
convert:     shlex

Options to pass to the mkfs command called by the provision action.

mnt

required:    true
scopable:    true

Example:

mnt = /srv/{fqdn}

The mount point where to mount the filesystem.

mnt_opt

required:    false
scopable:    true

The mount options, as they would be defined in the fstab.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 1777

The group that should be owner of the mnt directory.

Either in numeric or symbolic form.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

quota

required:    false
scopable:    true

The dataset quota property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

refquota

required:    false
scopable:    true
default:     x1

The dataset refquota property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

refreservation

required:    false
scopable:    true

The dataset refreservation property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

reservation

required:    false
scopable:    true

The dataset reservation property value to set on provision.

The value can be:

none
A size expression
A multiplier of the size keyword value (ex: x2).

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

Used by default as the refquota of the provisioned dataset.

The quota, refquota, reservation and refreservation values can be expressed as a multiplier of size (example: quota=x2).

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
default:     5s
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true
convert:     user

Example:

user = root

The user that should be owner of the mnt directory.

Either in numeric or symbolic form.

zone

required:    false
scopable:    true

The zone name the fs refers to.

If set, the fs mount point is reparented into the zonepath rootfs.

share.nfs

Minimal configlet:

[share#1]
type = nfs
opts = *(ro)
path = /srv/{fqdn}/share

Minimal setup command:

om test/vol/foo set \
	--kw="type=nfs" \
	--kw="opts=*(ro)" \
	--kw="path=/srv/{fqdn}/share"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

opts

required:    true
scopable:    true

Example:

opts = *(ro)

The NFS share export options.

On Linux, as they would be set in /etc/exports

On Solaris, as they would be passed to the share command.

path

required:    true
scopable:    true

Example:

path = /srv/{fqdn}/share

The path of the directory to share.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

parallel

required:    false
scopable:    true
convert:     bool

If set to true, actions are executed in parallel on the resource of this subset.

For example, a service with:

[app#1]
start = /bin/true

[app#2]
start = /bin/true

[app#3]
start = /bin/true
subset = workers

[app#4]
start = /bin/true
subset = workers

[subset#app:workers]
parallel = true

Would have a start sequence like:

app#1 ---> app#2 -+-> app#3
                  `-> app#4

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.rsync

Minimal configlet:

[sync#1]
type = rsync

Minimal setup command:

om test/vol/foo set --kw="type=rsync"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

bwlimit

required:    false
scopable:    false

Bandwidth limit (the default unit is kb/s) applied to this rsync transfer.

Leave empty to enforce no limit.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dst

required:    false
scopable:    true

Example:

dst = /srv/{fqdn}

The destination of the sync.

Can be a whitespace-separated list of files or directories passed untamed to rsync.

Attention: The ending '/' is meaningful. Refer to the rsync man page for details.

dstfs

required:    false
scopable:    true

Example:

dstfs = /srv/{fqdn}

If set to a remote mount point, OpenSVC will verify that the specified mount point is really hosting a mounted FS.

This is a safety net to prevent overflowing the parent filesystem (/ for example).

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

options

required:    false
scopable:    true
convert:     shlex

Example:

options = --acls --xattrs --exclude foo/bar

A whitespace-separated list of params passed unchanged to rsync.

Typical usage is ACL preservation activation.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

reset_options

required:    false
scopable:    false
convert:     bool

Use options only instead of merging options to default hardcoded options.

This keyword can be used to disable --xattr or --acls for example.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

snap

required:    false
scopable:    false
convert:     bool

If true, OpenSVC will try to snapshot the first snapshottable parent of the src and replicate from the snap instead of the src.

src

required:    false
scopable:    true

Example:

src = /srv/{fqdn}/

The source of the sync.

Can be a whitespace-separated list of files or directories passed untamed to rsync.

Attention: The ending '/' is meaningful. Refer to the rsync man page for details.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

target

required:    false
scopable:    true
candidates:  nodes, drpnodes
convert:     list

Which nodes should receive this data sync from the PRD node where the instance is up and running.

A shared filesystem (shared disk, replicated disk, clustered fs or networked fs) should not have a rsync target containing nodes where the fs resource can be started.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the sync action a failure.

If no timeout is set, the agent waits indefinitely for the sync action to exit.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.symsnapvx

Minimal configlet:

[sync#1]
type = symsnapvx
symid = 0000001234

Minimal setup command:

om test/vol/foo set \
	--kw="type=symsnapvx" \
	--kw="symid=0000001234"

absolute

required:    false
scopable:    true

Example:

absolute = 12:15

Use -absolute in symsnapvx commands.

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

delta

required:    false
scopable:    true

Example:

delta = 00:15

Use -delta in symsnapvx commands.

devs

required:    false
scopable:    true
convert:     list

Example:

devs = 012a 012b

The whitespace-separated list of symmetrix device identifiers.

devs_from

required:    false
scopable:    true
convert:     list

Example:

devs_from = disk#0 disk#1

Use the pointed resource subdevices as the list of devices to snapshot.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true

Example:

name = prod_db1_weekly

Use -name in symsnapvx commands.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secure

required:    false
scopable:    true
convert:     bool

Use -secure in symsnapvx commands.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

symid

required:    true
scopable:    true

Example:

symid = 0000001234

The symmetrix array identifier.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.symsrdfs

Minimal configlet:

[sync#1]
type = symsrdfs
symdg = prod_db1
symid = 0000001234

Minimal setup command:

om test/vol/foo set \
	--kw="type=symsrdfs" \
	--kw="symdg=prod_db1" \
	--kw="symid=0000001234"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

rdfg

required:    false
scopable:    false
convert:     int

Example:

rdfg = 5

Id of the RDF group pairing the source and target devices.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

symdg

required:    true
scopable:    false

Example:

symdg = prod_db1

Name of the symmetrix device group where the source and target devices are grouped.

symid

required:    true
scopable:    true

Example:

symid = 0000001234

The symid of the local Symmetrix array hosting the symdg. This parameter must be scoped to specify which array each node can access.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.zfs

Minimal configlet:

[sync#1]
type = zfs
dst = pool/{fqdn}
src = pool/{fqdn}

Minimal setup command:

om test/vol/foo set \
	--kw="type=zfs" \
	--kw="dst=pool/{fqdn}" \
	--kw="src=pool/{fqdn}"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dst

required:    true
scopable:    true

Example:

dst = pool/{fqdn}

Destination dataset of the sync.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

intermediary

required:    false
scopable:    true
default:     true
convert:     bool

Synchronize the snapshots of the selected datasets that were created since the last update.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

recursive

required:    false
scopable:    true
default:     true
convert:     bool

Which nodes should receive this data sync from the PRD node where the instance is up and running.

A shared filesystem (shared disk, replicated disk, clustered fs or networked fs) should not have a rsync target containing nodes where the fs resource can be started.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

src

required:    true
scopable:    true

Example:

src = pool/{fqdn}

Source dataset of the sync.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

target

required:    false
scopable:    true
candidates:  nodes, drpnodes
convert:     list

Which nodes should receive this data sync from the PRD node where the instance is up and running.

A shared filesystem (shared disk, replicated disk, clustered fs or networked fs) should not have a rsync target containing nodes where the fs resource can be started.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the sync action a failure.

If no timeout is set, the agent waits indefinitely for the sync action to exit.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

sync.zfssnap

Minimal configlet:

[sync#1]
type = zfssnap
dataset = svc1fs/data svc1fs/log

Minimal setup command:

om test/vol/foo set \
	--kw="type=zfssnap" \
	--kw="dataset=svc1fs/data svc1fs/log"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

dataset

required:    true
scopable:    true
convert:     list

Example:

dataset = svc1fs/data svc1fs/log

A whitespace separated list of datasets to snapshot.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

keep

required:    false
scopable:    true
default:     3
convert:     int

Example:

keep = 3

The maximum number of snapshots to retain.

max_delay

required:    false
scopable:    false
convert:     duration

The delay above which the status of the resource reports warn.

It should be set according to your application service level agreement.

The scheduler task interval should be lower than max_delay.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true

Example:

name = weekly

A name included in the snapshot name to avoid retention conflicts between multiple zfs snapshot resources.

A full snapshot name is formatted as <subvol>.<name>.snap.<datetime>.

Example: data.weekly.snap.2016-03-09.10:09:52

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

recursive

required:    false
scopable:    true
default:     true
convert:     bool

Set to true to snap recursively the datasets.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_requires

required:    false
scopable:    false

Example:

sync_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'sync update' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

task.docker

Minimal configlet:

[task#1]
type = docker
image = ghcr.io/opensvc/pause

Minimal setup command:

om test/vol/foo set \
	--kw="type=docker" \
	--kw="image=ghcr.io/opensvc/pause"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    true
scopable:    true

Example:

image = ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

If not set, the container will have a private userns other containers can share.

A container with userns=host will share the host's userns.

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

task.host

Minimal configlet:

[task#1]
type = host

Minimal setup command:

om test/vol/foo set --kw="type=host"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true

The shlex expression of the command to execute on run actions.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the process execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Change the working directory to the specified location instead of the default <pathtmp>.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = CRT=cert1/server.crt PEM=cert1/server.pem

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

group

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified group instead of root.

limit_as

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes) (same as limit_vmem).

When both limit_vmem and limit_as is used, the max value is chosen.

limit_core

required:    false
scopable:    true
convert:     size

The limit on the largest core dump size that can be produced (unit byte).

limit_cpu

required:    false
scopable:    true
convert:     duration

Example:

limit_cpu = 30s

The limit on CPU time (duration).

limit_data

required:    false
scopable:    true
convert:     size

The limit on the data segment size of a process (unit byte).

limit_fsize

required:    false
scopable:    true
convert:     size

The limit on the largest file that can be created (unit byte).

limit_memlock

required:    false
scopable:    true
convert:     size

The limit on how much memory a process can lock with mlock(2) (unit byte, no solaris support).

limit_nofile

required:    false
scopable:    true
convert:     size

The limit on the number files a process can have open at once.

limit_nproc

required:    false
scopable:    true
convert:     size

The limit on the number of processes this user can have at one time, no solaris support.

limit_rss

required:    false
scopable:    true
convert:     size

The limit on the total physical memory that can be in use by a process (unit byte, no solaris support).

limit_stack

required:    false
scopable:    true
convert:     size

The limit on the stack size of a process (unit bytes).

limit_vmem

required:    false
scopable:    true
convert:     size

The limit on the total virtual memory that can be in use by a process (unit bytes).

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the process execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop

required:    false
scopable:    true

true

Execute the script command with stop argument on stop action.
false

Do nothing on stop action.
<shlex expression>

Execute this command on stop action.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 180

Wait for <duration> before declaring the app launcher stop action a failure.

Takes precedence over timeout.

If neither timeout nor stop_timeout is set, the agent waits indefinitely for the app launcher to return.

A timeout can be coupled with optional=true to not abort a service instance stop when an app launcher did not return.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

type

required:    false
scopable:    false

The resource driver name.

umask

required:    false
scopable:    true
convert:     umask

Example:

umask = 022

The umask to set for the application process.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

If the binary is owned by the root user, run it as the specified user instead of root.

task.oci

Minimal configlet:

[task#1]
type = oci
image = ghcr.io/opensvc/pause

Minimal setup command:

om test/vol/foo set \
	--kw="type=oci" \
	--kw="image=ghcr.io/opensvc/pause"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    true
scopable:    true

Example:

image = ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

If not set, the container will have a private userns other containers can share.

A container with userns=host will share the host's userns.

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

task.podman

Minimal configlet:

[task#1]
type = podman
image = ghcr.io/opensvc/pause

Minimal setup command:

om test/vol/foo set \
	--kw="type=podman" \
	--kw="image=ghcr.io/opensvc/pause"

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_run

required:    false
scopable:    true

A command or script to execute after the resource run action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_run

required:    false
scopable:    true

A command or script to execute before the resource run action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

check

required:    false
scopable:    true
candidates:  last_run,

Example:

check = last_run

If set to last_run, the last run retcode is used to report a task resource status.

If not set (default), the status of a task is always n/a.

command

required:    false
scopable:    true
convert:     shlex

Example:

command = /opt/tomcat/bin/catalina.sh

The command to execute in the docker container on run.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs_environment

required:    false
scopable:    true
convert:     shlex

Example:

configs_environment = PORT=http/port webapp/app1* {name}/* {name}-debug/settings

A whitespace-separated list of <var>=<cfg name>/<key path> or <cfg name>/<key matcher>.

If the cfg or config key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <cfg name>/<key path> only or whole <var>=<cfg name>/<key path>.

Example with,

<ns>/cfg/nginx a config having a user key with value user1.
<ns>/cfg/cfg1 a config having a key1 key with value val1.

configs_environment = NGINX_USER=nginx/user cfg1/* creates the following variables in the container command execution environment:

NGINX_USER=user1
key1=val1

confirmation

required:    false
scopable:    false
convert:     bool

If set to true, ask for an interactive confirmation to run the task.

This flag can be used for dangerous tasks like data restoration.

cwd

required:    false
scopable:    true

Example:

cwd = /opt/foo

The current working directory set for the executed command.

devices

required:    false
scopable:    true
convert:     shlex

Example:

devices = myvol1:/dev/xvda myvol2:/dev/xvdb

The whitespace-separated list of <host devpath>:<containerized devpath> exposing host devices as container devices.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

dns_search

required:    false
scopable:    true
convert:     list

Example:

dns_search = opensvc.com

The whitespace-separated list of DNS domains to search for shortname lookups.

If empty or not set, the list will be <name>.<namespace>.svc.<clustername> <namespace>.svc.<clustername> svc.<clustername>.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

entrypoint

required:    false
scopable:    true
convert:     shlex

Example:

entrypoint = /bin/sh

The script or binary executed in the container.

The entrypoint args must be set in command.

environment

required:    false
scopable:    true
convert:     shlex

Example:

environment = KEY=cert1/server.key PASSWORD=db/password

A whitespace-separated list of <var>=<value>.

A shell expression spliter is applied, so double quotes can be around <value> only or whole <var>=<value>.

guest_os

required:    false
scopable:    true
candidates:  unix, windows
default:     unix

Example:

guest_os = unix

The name of the operating system in the virtual machine.

hostname

required:    false
scopable:    true

Example:

hostname = nginx1

Set the container hostname. If not set, a unique id is used.

image

required:    true
scopable:    true

Example:

image = ghcr.io/opensvc/pause

The docker image pull, and run the container with.

image_pull_policy

required:    false
scopable:    true
candidates:  once, always

Example:

image_pull_policy = once

The docker image pull policy.

always

Pull upon each container start.
once

Pull if not already pulled (default).

init

required:    false
scopable:    true
default:     true
convert:     bool

Run an init inside the container that forwards signals and reaps processes.

interactive

required:    false
scopable:    true
convert:     bool

Keep stdin open even if not attached.

To use if the container entrypoint is a shell.

ipcns

required:    false
scopable:    true

Example:

ipcns = container#0

empty

The docker daemon's default value is used.
none

Do not mount /dev/shm.

private

Create a ipcns other containers can not share.
shareable

Create a ipcns other containers can share.
container#

Share the container# ipcns.

log

required:    false
scopable:    true
default:     true
convert:     bool

Log the task outputs in the service log.

max_parallel

required:    false
scopable:    true
default:     1
convert:     int

Example:

max_parallel = 2

Support limited, concurrent runs of tasks.

The task#xx.max_parallel=2 setting limits the number of concurrent task runs to 2.

The default value is 1, ensuring backward compatibility.

The run count is determined based on PID files created in the /run/ directories.

Staleness is evaluated using the condition: (PID file mtime < process birth time).

A new status log message may appear to indicate that the maximum concurrency limit has been reached.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     Autogenerated using a `<namespace>..<object name>.container.<resource index>`

template.

Example:

name = osvcprd..rundeck.container.db

The name to assign to the container on docker run.

If not set, a <namespace>..<name>.container.<rid idx> name is automatically assigned.

netns

required:    false
scopable:    true

Example:

netns = container#0

empty or none

The container has a private netns other container, ip.netns or ip.cni resources can share.
<rid>

The id of the resource that has the network namespace this container joins.

For example, a container with netns=container#0 will share the container#0 netns.
host

Share the host network namespace.

on_error

required:    false
scopable:    true

Example:

on_error = /srv/{name}/data/scripts/task_on_error.sh

A command to execute on run action if command returned an error.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

osvc_root_path

required:    false
scopable:    true

Example:

osvc_root_path = /opt/opensvc

If the OpenSVC agent is installed via package in the container, this keyword must not be set.

Else the value can be set to the fullpath hosting the agent installed from sources.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pidns

required:    false
scopable:    true

Example:

pidns = container#0

empty

The container has a private pidns other containers can share. Usually a pidns sharer will run a pause image to reap zombies.
container#

Share container# pidns.
host

Share the host's pidns.

pool

required:    false
scopable:    true

The name of the pool this volume was allocated from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_run

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

privileged

required:    false
scopable:    true
convert:     bool

Give extended privileges to the container.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

pull_timeout

required:    false
scopable:    true
default:     2m
convert:     duration

Example:

pull_timeout = 2m

Wait for <duration> before declaring the container action a failure.

registry_creds

required:    false
scopable:    true

Example:

registry_creds = creds-registry-opensvc-com

The name of a secret in the same namespace having a config.json key which value is used to login to the container image registry.

If not specified, the node-level registry credential store is used.

retcodes

required:    false
scopable:    true
default:     0:up 1:down

Example:

retcodes = 0:up 1:down 3:warn 4: n/a 5:undef

The whitespace-separated list of <retcode>:<status name>.

All undefined retcodes are mapped to the warn status.

Valid <status names> are:

up
down
warn
n/a
undef

run_args

required:    false
scopable:    true
convert:     shlex

Example:

run_args = -v /opt/docker.opensvc.com/vol1:/vol1:rw -p 37.59.71.25:8080:8080

Extra arguments to pass to the docker run command, like volume and port mappings.

run_requires

required:    false
scopable:    false

Example:

run_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'run' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

run_timeout

required:    false
scopable:    true
convert:     duration

Example:

run_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

schedule

required:    false
scopable:    true

Example:

schedule = 00:00-01:00 mon

Set the task run schedule.

See usr/share/doc/opensvc/schedule for the schedule syntax reference.

secrets_environment

required:    false
scopable:    true
convert:     shlex

Example:

secrets_environment = CRT=cert1/server.pem sec1/*

A whitespace-separated list of <var>=<sec name>/<key path> or <sec name>/<key matcher>.

If the sec or secret key doesn't exist then start and stop actions on the resource will fail with a non 0 exit code.

A shell expression splitter is applied, so double quotes can be around <sec name>/<key path> only or whole <var>=<sec name>/<key path>.

Example with,

<ns>/sec/cert1 a secret having a server.pem key with value mycrt.
<ns>/sec/sec1 a secret having a key1 key with value val1.

secrets_environment = CRT=cert1/server.pem sec1/* creates the following variables in the container command execution environment:

CRT=mycrt
key1=val1

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

size

required:    false
scopable:    true
convert:     size

The size used by this volume in its pool.

snooze

required:    false
scopable:    true
convert:     duration

Example:

snooze = 10m

Snooze the service before running the task, so if the command is cause a status degradation the user can decide to snooze alarms for the duration set as value.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
convert:     duration

Example:

timeout = 5m

Wait for <duration> before declaring the task run action a failure.

If no timeout is set, the agent waits indefinitely for the task command to exit.

tty

required:    false
scopable:    true
convert:     bool

Allocate a pseudo-tty.

type

required:    false
scopable:    false

The resource driver name.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = guest

The user that will run the command inside the container.

Also support the <user>:<group> syntax.

userns

required:    false
scopable:    true

Example:

userns = container#0

Defines the podman container run --userns value.

the 'container#...' values are converted to container:id

utsns

required:    false
scopable:    true
candidates:  , host

Example:

utsns = container#0

empty

The container has a private utsns.
host

The container shares the host's hostname.

volume_mounts

required:    false
scopable:    true
convert:     shlex

Example:

volume_mounts = myvol1:/vol1 myvol2:/vol2:rw /localdir:/data:ro

The whitespace-separated list of <volume name|local dir>:<containerized mount path>:<mount options>.

When the source is a local dir, the default <mount option> is rw.

When the source is a volume name, the default <mount option> is taken from volume access.

volume

access

required:    false
scopable:    true
candidates:  rwo, roo, rwx, rox
default:     rwo

The access mode of the volume.

rwo is Read Write Once
roo is Read Only Once
rwx is Read Write Many
rox is Read Only Many

rox and rwx modes are served by flex volume services.

blocking_post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors interrupt the action.

blocking_post_start

required:    false
scopable:    true

A command or script to execute after the resource start action.

Errors interrupt the action.

blocking_post_stop

required:    false
scopable:    true

A command or script to execute after the resource stop action.

Errors interrupt the action.

blocking_post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource unprovision action.

Errors interrupt the action.

blocking_pre_provision

required:    false
scopable:    true

A command or script to execute before the resource provision action.

Errors interrupt the action.

blocking_pre_start

required:    false
scopable:    true

A command or script to execute before the resource start action.

Errors interrupt the action.

blocking_pre_stop

required:    false
scopable:    true

A command or script to execute before the resource stop action.

Errors interrupt the action.

blocking_pre_unprovision

required:    false
scopable:    true

A command or script to execute before the resource unprovision action.

Errors interrupt the action.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

configs

required:    false
scopable:    true
convert:     shlex

Example:

configs = conf/mycnf:/etc/mysql/my.cnf:ro conf/sysctl:/etc/sysctl.d/01-db.conf

The whitespace-separated list of <config name>/<key>:<volume relative path>:<options>.

directories

required:    false
scopable:    true
convert:     list

Example:

directories = a/b/c d /e

The whitespace-separated list of directories to create in the vol head.

dirperm

required:    false
scopable:    true
default:     700
convert:     file-mode

Example:

dirperm = 750

The permissions, in octal notation, to apply to the volume root and installed directories.

disable

required:    false
scopable:    true
convert:     bool

A disabled resource will be ignored on start, stop, provision and unprovision actions.

A disabled resource status is n/a.

If set in the DEFAULT section of an object, the object is disabled and ignores start, stop, shutdown, provision and unprovision actions.

These actions immediately return success.

om <path> disable sets DEFAULT.disable=true.

om <path> enable sets DEFAULT.disable=false.

Note: The enable and disable actions preserve the individual resource disable state.

encap

required:    false
scopable:    false
convert:     bool

Set to true to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers.

format

required:    false
scopable:    true
default:     true
convert:     bool

If true, the pool's vol configuration producer will include a fs resource layered over the disk resource.

group

required:    false
scopable:    true

Example:

group = 1001

The group name or id that will own the volume root and installed files and directories.

monitor

required:    false
scopable:    true
convert:     bool

A resource with monitor=true will trigger the monitor_action (crash or reboot the node, freezestop or switch the service) if:

The resource is down.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
All restart tentatives failed.

name

required:    false
scopable:    true
default:     {name}-vol-{rindex}

The vol name.

An object can only reference a vol in the same namespace.

nodes

required:    false
scopable:    true
default:     {.nodes}
convert:     nodes

A node selector expression filtering the creator nodes to determine the volume nodes.

If not set, all the creator nodes will be volume nodes.

optional

required:    false
scopable:    true
convert:     bool

Action errors on optional resources are logged but do not interrupt the action sequence.

The status of optional resources is not included in the instance availability status but is considered in the overall status.

The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional.

Resources tagged as noaction are considered optional by default.

Dump filesystems are a typical use case for optional=true.

perm

required:    false
scopable:    true
convert:     file-mode

Example:

perm = 660

The permissions, in octal notation, to apply to the installed files.

pg_blkio_weight

required:    false
scopable:    true

Example:

pg_blkio_weight = 50

Block IO relative weight. Value: between 10 and 1000.

The kernel default is 1000.

pg_cpu_quota

required:    false
scopable:    true

Example:

pg_cpu_quota = 50%@all

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpu_shares

required:    false
scopable:    true
convert:     size

Example:

pg_cpu_shares = 512

The kernel default value is used, which usually is 1024 shares.

In a cpu-bound situation, this setting ensures the service does not use more than its share of cpu resource. The actual percentile depends on shares allowed to other services.

pg_cpus

required:    false
scopable:    true
depends:     create_pg=true

Example:

pg_cpus = 0-2

Allow service process to bind only the specified cpus.

Cpus are specified as list or range : 0,1,2 or 0-2.

pg_mem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_mem_limit = 512m

Ensures the service does not use more than specified memory (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing.

pg_mem_oom_control

required:    false
scopable:    true

Example:

pg_mem_oom_control = 1

A flag (0 or 1) that enables or disables the Out of Memory killer for the processes of the group.

If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system.

The OOM killer is enabled by default in every cgroup using the memory controller.

pg_mem_swappiness

required:    false
scopable:    true

Example:

pg_mem_swappiness = 40

Set a swappiness percentile value for the process group.

pg_mems

required:    false
scopable:    true

Example:

pg_mems = 0-2

Allow service process to bind only the specified memory nodes.

Memory nodes are specified as list or range : 0,1,2 or 0-2.

pg_vmem_limit

required:    false
scopable:    true
convert:     size

Example:

pg_vmem_limit = 1g

Ensures the service does not use more than specified memory+swap (in bytes).

The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than pg_mem_limit.

pool

required:    false
scopable:    true

The name of the pool to allocate from.

post_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

post_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_provision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_start

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_stop

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

pre_unprovision

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

provision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the provision and unprovision actions on the resource.

Warning: provision and unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

provision_requires

required:    false
scopable:    false

Example:

provision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'provision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

provision_timeout

required:    false
scopable:    true
convert:     duration

Example:

provision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

restart

required:    false
scopable:    true
default:     0
convert:     int

The daemon will try to restart a resource if:

The resource is down, stdby down or warn.
The instance has local_expect=started in its daemon monitor data, which means the daemon considers this instance is and should remain started.
The node is not frozen
The instance is not frozen

In this case, the daemon try restart=<n> times before falling back to the monitor action.

The restart_delay keyword sets the interval after a failed restart before the next tentative.

Resources with standby=true have restart forced to a minimum of 2, to increase chances of a restart success.

restart_delay

required:    false
scopable:    true
default:     500ms
convert:     duration

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

secrets

required:    false
scopable:    true
convert:     shlex

Example:

secrets = cert/pem:server.pem cert/key:server.key

The whitespace-separated list of <secret name>/<key>:<volume relative path>:<options>.

shared

required:    false
scopable:    true
convert:     bool

If true, the resource will be considered shared during provision and unprovision actions.

A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not:

When --leader is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices.
When --leader is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices.

The daemon takes care of setting the --leader flags on the commands it submits during deploy, purge, provision and unprovision orchestrations.

Warning: If admins want to submit --local provision or unprovision commands themselves, they have to set the --leader flag correctly.

Flex objects usually don't use shared resources. But if they do, only the flex primary gets --leader commands.

Warning: All resources depending on a shared resource must also be flagged as shared.

signal

required:    false
scopable:    true

Example:

signal = hup:container#1

If only the signal is specified, all candidate resources will be signaled.

This keyword is typically used to reload daemons on certificate or configuration files changes.

size

required:    false
scopable:    true
convert:     size

The size to allocate in the pool.

standby

required:    false
scopable:    true
convert:     bool

If true, always start the resource, even on non-started instances.

The daemon is responsible for starting standby resources.

A resource can be set standby on a subset of nodes using keyword scoping.

A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem.

Warning: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example.

start_requires

required:    false
scopable:    false

Example:

start_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'start' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

start_timeout

required:    false
scopable:    true
convert:     duration

Example:

start_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

stat_timeout

required:    false
scopable:    true
convert:     duration

The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond.

When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored.

status_timeout

required:    false
scopable:    true
default:     1m
convert:     duration

Example:

status_timeout = 10s

The maximum duration of the instance status evaluation.

For example, the total start action duration is constrained by different timeouts:

the start_timeout Limiting the start action duration.
the stop_timeout Limiting the start rollback duration triggered by start errors.
the status_timeout Limiting the post-start instance status evaluation duration.

stop_requires

required:    false
scopable:    false

Example:

stop_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'stop' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

stop_timeout

required:    false
scopable:    true
convert:     duration

Example:

stop_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

subset

required:    false
scopable:    true

A command or script to execute after the resource provision action.

Errors do not interrupt the action.

sync_timeout

required:    false
scopable:    true
convert:     duration

Example:

sync_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

tags

required:    false
scopable:    true
convert:     set

A whitespace-separated list of tags.

Tags can be used for resource selection by tag.

Some tags can influence the driver behaviour:

noaction

Skip any state changing action on the resource and imply optional=true.
nostatus

Force the status n/a.

timeout

required:    false
scopable:    true
default:     1h
convert:     duration

Example:

timeout = 2h

Wait for <duration> before declaring a state-changing action a failure.

A per-action <action>_timeout can override this value.

type

required:    false
scopable:    true

The type of the pool to allocate the vol from.

The selected pool will be the one matching type and capabilities and with the maximum available space.

unprovision

required:    false
scopable:    false
default:     true
convert:     bool

Set to false to ignore the unprovision action on the resource.

Warning: unprovision use data-destructive operations like formatting.

It is recommended to set provision=false on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data.

unprovision_requires

required:    false
scopable:    false

Example:

unprovision_requires = ip#0 fs#0(down,stdby down)

A whitespace-separated list of conditions to meet to accept a 'unprovision' action.

A condition is expressed as <rid>(<state>,...).

If states are omitted, up,stdby up is used as the default expected states.

unprovision_timeout

required:    false
scopable:    true
convert:     duration

Example:

unprovision_timeout = 1m30s

Wait for <duration> before declaring the action a failure.

Takes precedence over timeout.

user

required:    false
scopable:    true

Example:

user = 1001

The user name or id that will own the volume root and installed files and directories.

DEFAULT

alt_names

required:    false
scopable:    true
convert:     list

Example:

alt_names = www.opensvc.com opensvc.com

Certificate Signing Request Alternative Domain Names.

app

required:    false
scopable:    false
default:     default

A user-defined code linking to:

who is responsible for this service.
who is billable.

This code thus provides a most useful object grouping and filtering key.

Short and simple codes, like ERP, are easier to work with.

bits

required:    false
scopable:    true
default:     4kib
convert:     size

Example:

bits = 8192

Certificate Private Key Length.

c

required:    false
scopable:    true

Example:

c = FR

Certificate Signing Request Country.

ca

required:    false
scopable:    true

Example:

ca = ca

The name of secret containing a certificate to use as a Certificate Authority. This secret must be in the same namespace.

cn

required:    false
scopable:    true

Example:

cn = test.opensvc.com

Certificate Signing Request Common Name.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

drpnodes

required:    false
scopable:    true
convert:     other-nodes

Example:

drpnodes = n1 n2

A node selector expression specifying the list of cluster nodes hosting object instances when all primary nodes are unavailable, like in a DRP situation.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

The drpnodes can be data synchronization targets for sync resources.

email

required:    false
scopable:    true

Example:

email = test@opensvc.com

Certificate Signing Request Email.

env

required:    false
scopable:    false
default:     The same as the node `env`.

A code like PRD, DEV, etc... the agent can use to enforce data protection policies:

A non-PRD object instance can not be started on a PRD node
A PRD object instance can be started on a non-PRD node (typically in a DRP situation)

The default value is read from the node env keyword.

id

required:    false
scopable:    false
default:     A random generated UUID.

A rfc4122 random uuid generated by the agent.

l

required:    false
scopable:    true

Example:

l = Gouvieux

Certificate Signing Request Location.

nodes

required:    false
scopable:    true
default:     *
convert:     nodes

A node selector expression specifying the list of cluster nodes hosting object instances.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

o

required:    false
scopable:    true

Example:

o = OpenSVC

Certificate Signing Request Organization.

ou

required:    false
scopable:    true

Example:

ou = Lab

Certificate Signing Request Organizational Unit.

st

required:    false
scopable:    true

Example:

st = Oise

Certificate Signing Request State.

validity

required:    false
scopable:    true
default:     1y
convert:     duration

Example:

validity = 10y

Certificate Validity duration.

DEFAULT

app

required:    false
scopable:    false
default:     default

A user-defined code linking to:

who is responsible for this service.
who is billable.

This code thus provides a most useful object grouping and filtering key.

Short and simple codes, like ERP, are easier to work with.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

drpnodes

required:    false
scopable:    true
convert:     other-nodes

Example:

drpnodes = n1 n2

A node selector expression specifying the list of cluster nodes hosting object instances when all primary nodes are unavailable, like in a DRP situation.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

The drpnodes can be data synchronization targets for sync resources.

env

required:    false
scopable:    false
default:     The same as the node `env`.

A code like PRD, DEV, etc... the agent can use to enforce data protection policies:

A non-PRD object instance can not be started on a PRD node
A PRD object instance can be started on a non-PRD node (typically in a DRP situation)

The default value is read from the node env keyword.

id

required:    false
scopable:    false
default:     A random generated UUID.

A rfc4122 random uuid generated by the agent.

nodes

required:    false
scopable:    true
default:     *
convert:     nodes

A node selector expression specifying the list of cluster nodes hosting object instances.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

DEFAULT

app

required:    false
scopable:    false
default:     default

A user-defined code linking to:

who is responsible for this service.
who is billable.

This code thus provides a most useful object grouping and filtering key.

Short and simple codes, like ERP, are easier to work with.

comment

required:    false
scopable:    false

Comments help the users understand the role of the object and its resources.

drpnodes

required:    false
scopable:    true
convert:     other-nodes

Example:

drpnodes = n1 n2

A node selector expression specifying the list of cluster nodes hosting object instances when all primary nodes are unavailable, like in a DRP situation.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.

The drpnodes can be data synchronization targets for sync resources.

env

required:    false
scopable:    false
default:     The same as the node `env`.

A code like PRD, DEV, etc... the agent can use to enforce data protection policies:

A non-PRD object instance can not be started on a PRD node
A PRD object instance can be started on a non-PRD node (typically in a DRP situation)

The default value is read from the node env keyword.

grant

required:    false
scopable:    true
convert:     list-lowercase

Example:

grant = admin:test* guest:*

Grant roles to the user.

A whitespace-separated list of pervasives role or per-namespace roles.

Pervasive roles:

root

Add resource triggers, non-containerized resources (non-root users can only add container.docker, container.podman task.docker, task.podman and volume)
squatter

Create a new namespace.
prioritizer

Set the priority keyword of an object.
blacklistadmin

Clear the blacklist of daemon listeners clients.
<per-namespace role>:<namespace selector>

Per-namespace roles:

admin

Create, delete objects in the namespace.
operator

Start, stop, provision, unprovision, freeze, unfreeze objects in the namespace.
guest

List and read configuration and status of the objects in the namespace.

A namespace selector is a glob pattern applied to existing namespaces.

id

required:    false
scopable:    false
default:     A random generated UUID.

A rfc4122 random uuid generated by the agent.

nodes

required:    false
scopable:    true
default:     *
convert:     nodes

A node selector expression specifying the list of cluster nodes hosting object instances.

If not specified or left empty, the node evaluating the keyword is assumed to be the only instance hosting node.

The glob syntax can be used in the node selector expression. For example n1 n[23] n4* would be expanded to n1 n2 n3 n4 in a n1 n2 n3 n4 n5 cluster.