# Driver `disk.drbd` **Minimal configlet:** [disk#1] type = drbd **Minimal setup command:** om test/vol/foo set --kw="type=drbd" **Supported keywords:** - addr - blocking_post_provision - blocking_post_start - blocking_post_stop - blocking_post_unprovision - blocking_pre_provision - blocking_pre_start - blocking_pre_stop - blocking_pre_unprovision - comment - disable - disk - encap - max_peers - monitor - network - no_preempt_abort - optional - pg_blkio_weight - pg_cpu_burst - pg_cpu_quota - pg_cpu_shares - pg_cpus - pg_mem_high - pg_mem_limit - pg_mem_oom_control - pg_mem_swappiness - pg_mems - pg_pids_max - pg_vmem_limit - port - post_provision - post_start - post_stop - post_unprovision - pre_provision - pre_start - pre_stop - pre_unprovision - prkey - promote_rw - provision - provision_requires - res - restart - restart_delay - scsireserv - shared - standby - start_requires - stat_timeout - stop_requires - subset - tags - template - type - unprovision - unprovision_requires ## Keyword `addr` required: false scopable: true provision: used only by the provision of the resource rbac: This driver group requires the root grant. **Default:** The ipaddr resolved for the nodename. **Example:** addr=1.2.3.4 **Description:** The addr to use to connect a peer. Use scoping to define each non-default address. ## Keyword `blocking_post_provision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `provision` action. Errors interrupt the action. This trigger is only executed on leaders. ## Keyword `blocking_post_start` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `start` action. Errors interrupt the action. ## Keyword `blocking_post_stop` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `stop` action. Errors interrupt the action. ## Keyword `blocking_post_unprovision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `unprovision` action. Errors interrupt the action. This trigger is only executed on leaders. ## Keyword `blocking_pre_provision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `provision` action. Errors interrupt the action. ## Keyword `blocking_pre_start` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `start` action. Errors interrupt the action. ## Keyword `blocking_pre_stop` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `stop` action. Errors interrupt the action. ## Keyword `blocking_pre_unprovision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `unprovision` action. Errors interrupt the action. ## Keyword `comment` required: false scopable: false rbac: This driver group requires the root grant. **Description:** A free form text describing the role of the object, of the node, or of the section it is set in. The keyword is accepted in any section, so the DEFAULT section can document a configuration as a whole, and a resource, pool, heartbeat, array or network section can document itself. The agent does not interpret the value. ## Keyword `disable` required: false scopable: true convert: bool rbac: This driver group requires the root grant. **Description:** A disabled resource will be ignored on `start`, `stop`, `provision` and `unprovision` actions. A disabled resource status is `n/a`. If set in the `DEFAULT` section of an object, the object is disabled and ignores `start`, `stop`, `shutdown`, `provision` and `unprovision` actions. These actions immediately return success. `om disable` sets `DEFAULT.disable=true`. `om enable` sets `DEFAULT.disable=false`. > **Note**: The `enable` and `disable` actions preserve the individual resource `disable` state. ## Keyword `disk` required: false scopable: true provision: used only by the provision of the resource, which fails without it rbac: This driver group requires the root grant. **Example:** disk=/dev/vg1/lv1 **Description:** The device to provision the drbd on, written as the other devices of a service are: * `/dev/vg1/lv1` is a device of the node. * `disk#1` is the device a resource of the service exposes, as a logical volume. The resource must expose one device, and start before this one. The resource file of the drbd names the disk of every node: a resource id is the device the resource exposes on the node provisioning it, as a logical volume of the same name on the others. A different device can be set on different nodes using the `disk@` scoping syntax. ## Keyword `encap` required: false scopable: false convert: bool rbac: This driver group requires the root grant. **Description:** Set to `true` to ignore this resource in the nodes context and consider it in the encapnodes context. The resource is thus handled by agents deployed in the service containers. ## Keyword `max_peers` required: false scopable: false provision: used only by the provision of the resource convert: int rbac: This driver group requires the root grant. **Default:** (nodes_count*2)-1 **Example:** max_peers=8 **Description:** The integer value to use in `create-md --max-peers `. The driver ensures the value is not lesser than the number of instances. ## Keyword `monitor` required: false scopable: true convert: bool rbac: This driver group requires the root grant. **Description:** A resource with `monitor=true` will trigger the `monitor_action` (crash or reboot the node, freezestop or switch the service) if: * The resource is `down`. * The instance has `local_expect=started` in its daemon monitor data, which means the daemon considers this instance is and should remain started. * All restart tentatives failed. ## Keyword `network` required: false scopable: false provision: used only by the provision of the resource rbac: This driver group requires the root grant. **Example:** network=benet1 **Description:** The name of the backend network to use for drbd traffic. Set this keyword if some node names are resolved to NATed addresses. The address of a node is the first of the subnet the network assigned it, so the network is a `routed_bridge`. A network giving the nodes no subnet of their own, as a `lan`, is refused: set `addr` instead. ## Keyword `no_preempt_abort` required: false scopable: true convert: bool rbac: This driver group requires the root grant. **Description:** If `true`, the agent will preempt the scsi3 persistent reservation with a `preempt` command instead of a `preempt and and abort`. Some scsi target implementations do not support `preempt and and abort` (esx). ## Keyword `optional` required: false scopable: true convert: bool rbac: This driver group requires the root grant. **Description:** Action errors on optional resources are logged but do not interrupt the action sequence. The status of optional resources is not included in the instance availability status but is considered in the overall status. The status of task and sync resources is always included in the overall status, regardless of whether they are marked as optional. Resources tagged as `noaction` are considered optional by default. Dump filesystems are a typical use case for optional=true. ## Keyword `pg_blkio_weight` required: false scopable: true rbac: This driver group requires the root grant. **Example:** pg_blkio_weight=50 **Description:** Block IO relative weight. Value: between `10` and `1000`. The kernel default is `100`. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_cpu_burst` required: false scopable: true rbac: This driver group requires the root grant. **Example:** pg_cpu_burst=20% **Description:** The cpu time the group banks under its quota, and spends above it. It is the time the processes of the group may bank while under their `pg_cpu_quota`, and spend above it in a short spike, in the notation of `pg_cpu_quota`: `20%` lets the group run a fifth of a cpu above its quota during a period, out of what it left unused before. It serves a latency-sensitive service capped tightly: a request arriving after an idle spell is served at the speed of the burst, rather than throttled for the rest of the period. Over a longer time the group still uses no more than its quota. It needs `pg_cpu_quota`, and cannot exceed it: the kernel refuses a burst larger than the quota. Only the unified cgroup hierarchy has it. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_cpu_quota` required: false scopable: true rbac: This driver group requires the root grant. **Example:** pg_cpu_quota=50%@all **Description:** The percentage of cpu time the processes of the group may use. Accepted expressions: * `50%` Half of one cpu. * `50%@all` Half of every cpu the node has. * `10%@2` A tenth of two cpus. Unlike `pg_cpu_shares`, which only arbitrates when the node is cpu-bound, a quota caps the group whether or not the node is busy. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_cpu_shares` required: false scopable: true convert: size rbac: This driver group requires the root grant. **Example:** pg_cpu_shares=512 **Description:** The share of cpu the group gets when the node is cpu-bound. The share is relative to the other groups. The actual percentile depends on the shares allowed to the other services. Unset, the kernel default is used, which usually is 1024 shares. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_cpus` required: false scopable: true depends: create_pg=true rbac: This driver group requires the root grant. **Example:** pg_cpus=0-2 **Description:** The cpus the processes of the group may run on. Cpus are specified as list or range : `0,1,2` or `0-2`. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_mem_high` required: false scopable: true convert: size rbac: This driver group requires the root grant. **Example:** pg_mem_high=384m **Description:** The memory past which the kernel throttles the group, in bytes. Past it, the kernel slows the processes of the group down, reclaiming their memory and throttling their allocations. Unlike `pg_mem_limit`, reaching it does not wake the Out-Of-Memory killer: a group above it keeps running, slower, while the kernel reclaims. Set below `pg_mem_limit`, it gives a process growing too much the time to be noticed, or to shrink, before it is killed. It is a soft limit, so it is not what a memory claim counts: the claim counts `pg_mem_limit`. Only the unified cgroup hierarchy has it. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_mem_limit` required: false scopable: true convert: size rbac: This driver group requires the root grant. **Example:** pg_mem_limit=512m **Description:** The memory the processes of the group may use, in bytes. The Out-Of-Memory killer is triggered in case of tresspassing. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_mem_oom_control` required: false scopable: true rbac: This driver group requires the root grant. **Example:** pg_mem_oom_control=1 **Description:** Whether the Out of Memory killer runs for the group: 0 enables it. It is a flag (0 or 1) enabling or disabling the Out of Memory killer for the processes of the group. * If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer. * If disabled (1), tasks are allowed to continue to try allocating memory, stressing the system. The OOM killer is enabled by default in every cgroup using the memory controller. This keyword caps nothing on a node holding the unified cgroup hierarchy, which has no `memory.oom_control`: that file is of the v1 hierarchy. Setting it there is warned about, and ignored. ## Keyword `pg_mem_swappiness` required: false scopable: true rbac: This driver group requires the root grant. **Example:** pg_mem_swappiness=40 **Description:** Set a swappiness percentile value for the process group. This keyword caps nothing on a node holding the unified cgroup hierarchy, which has no `memory.swappiness`: that file is of the v1 hierarchy. Setting it there is warned about, and ignored. ## Keyword `pg_mems` required: false scopable: true rbac: This driver group requires the root grant. **Example:** pg_mems=0-2 **Description:** The memory nodes the processes of the group may allocate from. Memory nodes are specified as list or range : `0,1,2` or `0-2`. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_pids_max` required: false scopable: true rbac: This driver group requires the root grant. **Example:** pg_pids_max=512 **Description:** The most processes and threads the group may run at once. A fork beyond it fails in the process that asked for it, and nothing else of the node is touched: it is what keeps a runaway fork loop from exhausting the process table of the node. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `pg_vmem_limit` required: false scopable: true convert: size rbac: This driver group requires the root grant. **Example:** pg_vmem_limit=1g **Description:** The memory plus swap the processes of the group may use, in bytes. The Out-Of-Memory killer is triggered in case of tresspassing. The specified value must be greater than `pg_mem_limit`. Setting this keyword to `default` puts the capping back where a node that never capped anything leaves it. Removing the keyword does not: what was written stays written, whether om wrote it or something else did. ## Keyword `port` required: false scopable: true provision: used only by the provision of the resource convert: int rbac: This driver group requires the root grant. **Example:** port=1234 **Description:** The port to use to connect a peer. Auto-allocated on provision if not already defined. ## Keyword `post_provision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `provision` action. Errors do not interrupt the action. This trigger is only executed on leaders. ## Keyword `post_start` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `start` action. Errors do not interrupt the action. ## Keyword `post_stop` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `stop` action. Errors do not interrupt the action. ## Keyword `post_unprovision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute after the resource `unprovision` action. Errors do not interrupt the action. This trigger is only executed on leaders. ## Keyword `pre_provision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `provision` action. Errors do not interrupt the action. ## Keyword `pre_start` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `start` action. Errors do not interrupt the action. ## Keyword `pre_stop` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `stop` action. Errors do not interrupt the action. ## Keyword `pre_unprovision` required: false scopable: true rbac: Triggers require the root grant. **Description:** A command or script to execute before the resource `unprovision` action. Errors do not interrupt the action. ## Keyword `prkey` required: false scopable: true default: {node.node.prkey} rbac: This driver group requires the root grant. **Description:** A specific scsi3 persistent reservation key for the resource. It overrides the object-level `prkey` and the node-level `prkey`. ## Keyword `promote_rw` required: false scopable: false convert: bool rbac: This driver group requires the root grant. **Description:** If `true`, OpenSVC will try to promote the exposed devices to read-write on `start` actions. ## Keyword `provision` required: false scopable: false default: true convert: bool rbac: This driver group requires the root grant. **Description:** Set to `false` to ignore the `provision` and `unprovision` actions on the resource. > **Warning**: `provision` and `unprovision` use data-destructive operations like formatting. It is recommended to set `provision=false` on long-lived critical objects, to force administrators to remove this setting when they really want to destroy data. ## Keyword `provision_requires` required: false scopable: false rbac: This driver group requires the root grant. **Example:** provision_requires=ip#0 fs#0(down,stdby down) **Description:** A whitespace-separated list of conditions to meet to accept a 'provision' action. A condition is expressed as `(,...)`. If states are omitted, `up,stdby up` is used as the default expected states. ## Keyword `res` required: false scopable: false stop first: a change is refused while the resource is up rbac: This driver group requires the root grant. **Example:** res=r1 **Description:** The name of the drbd resource associated with this service resource. OpenSVC expects the resource configuration file to reside in `/etc/drbd.d/.res` and takes care of its replication on peer nodes. ## Keyword `restart` required: false scopable: true default: 0 convert: int rbac: This driver group requires the root grant. **Description:** The daemon will try to restart a resource if: * The resource is `down`, `stdby down` or `warn`. * The instance has `local_expect=started` in its daemon monitor data, which means the daemon considers this instance is and should remain started. * The node is not frozen * The instance is not frozen In this case, the daemon try `restart=` times before falling back to the monitor action. The `restart_delay` keyword sets the interval after a failed restart before the next tentative. Resources with `standby=true` have `restart` forced to a minimum of 2, to increase chances of a restart success. ## Keyword `restart_delay` required: false scopable: true default: 500ms convert: duration rbac: This driver group requires the root grant. **Description:** The minimum delay between two restart tentatives on the resource. ## Keyword `scsireserv` required: false scopable: false convert: bool rbac: This driver group requires the root grant. **Description:** If `true`, try to acquire a type-5 (write exclusive, registrant only) scsi3 persistent reservation on every path to every disk used by this resource. Existing reservations are preempted to not block service failover. If the `start` was not legitimate the data are still protected from being written concurrently from all nodes. ## Keyword `shared` required: false scopable: true convert: bool rbac: This driver group requires the root grant. **Description:** If `true`, the resource will be considered shared during provision and unprovision actions. A shared resource driver can implement a different behaviour depending on weither it is run from the leader instance, or not: * When `--leader` is set, the driver creates and configures the system objects. For example the disk.disk driver allocates a SAN disk and discover its block devices. * When `--leader` is not set, the driver does not redo the actions already done by the leader, but may do some. For example, the disk.disk driver skips the SAN disk allocation, but discovers the block devices. The daemon takes care of setting the `--leader` flags on the commands it submits during deploy, purge, provision and unprovision orchestrations. > **Warning**: If admins want to submit `--local` provision or unprovision commands themselves, they have to set the `--leader` flag correctly. Flex objects usually don't use shared resources. But if they do, only the flex primary gets `--leader` commands. > **Warning**: All resources depending on a shared resource must also be flagged as shared. ## Keyword `standby` required: false scopable: true convert: bool rbac: This driver group requires the root grant. **Description:** If `true`, always start the resource, even on non-started instances. The daemon is responsible for starting standby resources. A resource can be set standby on a subset of nodes using keyword scoping. A typical use-case is a synchronized filesystem on non-shared disks. The remote filesystem must be mounted to not overflow the underlying filesystem. > **Warning**: In most situation, don't set shared resources standby, a non-clustered fs on shared disks for example. ## Keyword `start_requires` required: false scopable: false rbac: This driver group requires the root grant. **Example:** start_requires=ip#0 fs#0(down,stdby down) **Description:** A whitespace-separated list of conditions to meet to accept a 'start' action. A condition is expressed as `(,...)`. If states are omitted, `up,stdby up` is used as the default expected states. ## Keyword `stat_timeout` required: false scopable: true convert: duration rbac: This driver group requires the root grant. **Description:** The fs resources status evaluation includes a stat syscall test. This keyword defines the maximum wait time for those stat calls to respond. When expired, the resource status is degraded is to warn, which can trigger a monitor action (reboot or crash the node) if the resource is monitored. ## Keyword `stop_requires` required: false scopable: false rbac: This driver group requires the root grant. **Example:** stop_requires=ip#0 fs#0(down,stdby down) **Description:** A whitespace-separated list of conditions to meet to accept a 'stop' action. A condition is expressed as `(,...)`. If states are omitted, `up,stdby up` is used as the default expected states. ## Keyword `subset` required: false scopable: true rbac: This driver group requires the root grant. **Description:** The name of the subset this resource is assigned to. Resources of a same driver group and a same subset are ordered and acted upon together, and are configured as a whole in a `[subset#:]` section, or in a less precise `[subset#]` one. That section accepts `parallel`, to act on the members concurrently rather than one after the other, and the `pg_*` keywords, to place the members in their own process group. ## Keyword `tags` required: false scopable: true convert: set rbac: This driver group requires the root grant. **Description:** A whitespace-separated list of tags. Tags can be used for resource selection by tag. Some tags can influence the driver behaviour: * `noaction` Skip any state changing action on the resource and imply `optional=true`. * `nostatus` Force the status `n/a`. ## Keyword `template` required: false scopable: false provision: used only by the provision of the resource rbac: This driver group requires the root grant. **Example:** template=default **Description:** A template key name from config object `system/cfg/drbd` to define alternate drbd resource configuration template used during resource provisioning. When defined: Provision will create resource config file from the template retrieved from `om system/cfg/drbd key decode --name