Attach/Detach Disks (cbdtab) · CloudBD Documentation

Introduction

The /etc/cloudbd/cbdtab file controls which CloudBD disks attach to the server. Disks specified in the cbdtab file are automatically attached during the system startup. Manual starting and stopping of disks is also possible using the cbddisks_start and cbddisks_stop commands.

Attached CloudBD disks can be found in the /dev/mapper/ directory.

A CloudBD disk may only be attached to one server at a time. Attaching the disk to multiple servers simultaneously can cause disk corruption.

Add a line entry to /etc/cloudbd/cbdtab for each disk that will attach to the server

# nbd<N>  <remote>:<disk>         [driver_options]  
nbd0      remote:testdisk1        defaults # up to 1 GB/s (10 gigabit)
nbd1      remote:testdisk2        highperf # up to 2.5 GB/s (25 gigabit)
nbd2      remote:testdisk3        lowmem   # up to 500 MB/s (5 gigabit)

Start all disks specified in the cbdtab file

sudo cbddisks_start -a

ls /dev/mapper
control  remote:testdisk1  remote:testdisk2  remote:testdisk3

Configuring cbdtab

The /etc/cloudbd/cbdtab file controls which CloudBD disks attach to the server as well as the driver settings for the disks. The file may contain multiple disk definitions, one per line, each of which contains three space separated fields.

Fields are separated from each other by any number of space or tab characters; disk definitions are separated from one another by newline characters. The file may also contain any number of comments, which start with a '#' character and continue until the end of the line or the end of the file, whichever is first.

Fields

Each disk definition line contains the following fields:

<nbd>           <remote>:<disk>         [driver settings]

<nbd> The short name of an available nbd device.

CloudBD disks attach to the server using the Network Block Device (nbd) kernel module. The default nbd kernel module settings will create 16 nbd devices at /dev/nbd[0-15]. This field should contain the short name of an unused nbd device without the leading /dev/ part; e.g., it could say nbd0.
<remote>:<disk> The remote and disk names.

The remote name is the filename of your storage remote config file in the CloudBD remotes.d directory without the '.conf' extension; e.g., /etc/cloudbd/remotes.d/<remote>.conf.

For a list of configured remotes run:
```
cloudbd list
```
The disk name is the name of a CloudBD disk stored on <remote>.

For a list of disks on the remote run:
```
cloudbd list <remote>
```
[driver settings] The driver settings.

This field is optional and need not appear in a file if no custom settings are necessary. These settings are only read at startup of each disk's driver. To adjust a driver setting, change the value in the cbdtab file and restart the disk. Multiple options may be specified by using a comma separated list of options without spaces. Supported driver settings:
- defaults
  
  Recommended for servers with up to 10 Gigabit networking and at least 4GB of memory. Provides up to 1 GiB/s read and write throughput.
- lowmem
  
  Recommended for servers with up to 5 Gigabit networking and at least 2GB of memory. Provides up to 500 MiB/s read and write throughput.
- highperf | highmem
  
  Recommended for servers with up to 25 Gigabit networking and at least 8 GB of memory. Provides up to 2.5 GiB/s read and write throughput.
- noauto
  
  Do not attach this disk automatically at boot time or when using cbddisks_start -a. The disk can still be attached manually using the cbddisks_start <remote>:<disk> command.
- readahead=READAHEAD_MB
  
  Set the device's kernel readahead parameter in MiB (default is readahead=64 for defaults and highperf and readahead=32 for lowmem). The kernel uses this value whenever it detects a sequential read pattern on a file. The value is applied to each open file independently. The total readahead requested may be as large as the number of open files * READAHEAD_MB. A readahead value that is too small will cause lower read throughput due to round trip latency. A value too large may cause unnecessary reads of unrequested data.
  
  The optimal readahead value for a CloudBD disk will depend on the expected number of files concurrently read and the network performance between the server and remote object storage. Using the value TOTAL_READAHEAD = AVG_NUM_FILES_CONCURRENTLY_READ * READAHEAD_MB, we recommend:
  - For servers using defaults: TOTAL_READAHEAD = 128 MiB
  - For servers using lowmem: TOTAL_READAHEAD = 64 MiB
  - For servers using highperf: TOTAL_READAHEAD = 256 MiB
  For example, if optimizing a server that has 10 Gigabit networking to handle an average of 8 files being read at once, use driver options defaults,readahead=16.

Examples

Configure disk1 on remote1 (remote config file at /etc/cloudbd/remotes.d/remote1.conf) to attach to this server as a block device at /dev/mapper/remote1:disk1 and use the default driver settings.

# This file configures which CloudBD disks attach to this server.
# For more information on attaching disks or the driver settings, see the
# man page for cbdtab (5) or the documentation at https://www.cloudbd.io/docs.

# Disks attach as block devices at: '/dev/mapper/<remote>:<disk>'
# <nbd>         <remote>:<disk>         [driver settings]
nbd0            remote1:disk1

Configure multiple CloudBD disks from several remotes to attach to this server with various driver settings.

# This file configures which CloudBD disks attach to this server.
# For more information on attaching disks or the driver settings, see the
# man page for cbdtab (5) or the documentation at https://www.cloudbd.io/docs.

# Disks attach as block devices at: '/dev/mapper/<remote>:<disk>'
# <nbd>         <remote>:<disk>         [driver settings]

# Disks will use default settings.
nbd0            remote1:disk1
nbd1            remote1:disk2           defaults

# Disks will use reduced settings for lower memory usage.
nbd2            remote2:disk1           lowmem
nbd3            remote2:disk2           lowmem

# Disks will use increased settings for higher disk throughput.
nbd4            remote3:disk1           highperf
nbd5            remote3:disk2           highmem

# This disk wont attach automatically at startup and uses the lowmem settings.
# This disk can be attached by the 'cbddisks_start' command.
nbd6            remote3:disk3           lowmem,noauto

Attach and Detach Commands

CloudBD disks configured in the cbdtab file can be attached or detached from the current server using the cbddisks_start and cbddisks_stop commands. Attached disks can be found in the /dev/mapper/ directory.

To attach all disks defined in the cbdtab file (without the "noauto" setting) run:
```
cbddisks_start -a
```
To detach all attached disks run:
```
cbddisks_stop -a
```
To attach a specific disk from the cbdtab file run:
```
cbddisks_start <remote>:<disk>
```
To detach a specific attached disk run:
```
cbddisks_stop <remote>:<disk>
```

NOTE: When detaching CloudBD disks, disks that are busy (e.g., mounted disks) can be forced to stop by adding the -f flag to the cbddisks_stop command.

Systemd

For servers using Systemd, CloudBD disks are managed by Systemd service files. During system startup or whenever a systemctl daemon-reload is issued, Systemd parses the cbdtab file and generates Systemd service files named cbdsetup@<remote>:<disk>.service. These generated CloudBD service files collectively make up the Systemd target cbdsetup.target. By default, this target is started during system startup to automatically attach your CloudBD disks.

The cbddisks_start and cbddisks_stop commands will continue to work on servers using Systemd. The commands are forwarded to the CloudBD disk service files that manage the CloudBD disks.

Additionally, you can invoke systemctl commands directly to the CloudBD disk service files or the cbdsetup target as an alternative method to attach/detach disks. When directly using systemctl commands to manage CloudBD disks you must issue a systemctl daemon-reload after any modifications to the cbdtab file so that Systemd can generate updated service files.

To attach all disks defined in the cbdtab file (without the "noauto" setting) run:
```
systemctl start cbdsetup.target
```
To detach all attached disks run:
```
systemctl stop cbdsetup@*.service
```
To attach a specific disk from the cbdtab file run:
```
systemctl start cbdsetup@<remote>:<disk>.service
```

To detach a specific attached disk run:

systemctl stop cbdsetup@<remote>:<disk>.service

Upstart/SysVinit

For servers using Upstart or SysVinit, disks specified in cbdtab are managed by the cbddisks service. Starting the service will attach all disks defined in the cbdtab file (without the "noauto" setting). By default, this service is started during the system startup to automatically attach your CloudBD disks.

In addition to the cbddisks_start and cbddisks_stop commands, the following commands also attach and detach CloudBD disks.

To attach all disks defined in the cbdtab file (without the "noauto" setting) run:
```
service cbddisks start
```
To detach all attached disks run:
```
service cbddisks stop
```