Implementing Autochanger for bareos FileStorage

The implementation of Autochanger in bareos software is very far from working, so I expect many changes there in the future. The described solution is actually a workaround that works well on the 17.2 version. It may not work on a different version, you need to check before implementation on the production server.

When performing a backup using bareos, most of the backup time is wasted on the client side. The client performs an inventory of files, compares them with bareos metadata, compresses files, etc. If it were possible to perform parallel backups, this could dramatically reduce the backup window even for full backups. However, the standard FileStorage configuration does not allow running parallel tasks.

The solution includes four devices and an autochanger script. Devices are just symbolic links to real volume files somewhere in the location pointed to by the script. This solution makes concurrent backups possible yet still using the same media and pools. With minor changes, it can also use multiple file systems. The necessary wisdom can be realized in an autochanger script.

Changes to Storage daemon

I want to remind you that the storage daemon is not required to be on the same server, it can be somewhere else where there is a lot of disk space. Therefore, these changes need to be made where your storage daemon is running.

Remove the default FileStorage definition and create new devices:

# rm -f /etc/bareos/bareos-sd.d/device/FileStorage.conf
# cat /etc/bareos/bareos-sd.d/device/drive0.conf 
Device {
  AlwaysOpen = no
  Archive Device = /var/lib/bareos/storage/drive0
  Autochanger = yes
  Automatic Mount = no
  Block Checksum = no
  Device Type = File
  Drive Index = 0
  LabelMedia = yes
  Maximum Concurrent Jobs = 1
  Media Type = File
  Name = drive0
  Random Access = yes
  RemovableMedia = no
}
# sed -e 's/0/1/' /etc/bareos/bareos-sd.d/device/drive0.conf > /etc/bareos/bareos-sd.d/device/drive1.conf
# sed -e 's/0/2/' /etc/bareos/bareos-sd.d/device/drive0.conf > /etc/bareos/bareos-sd.d/device/drive2.conf
# sed -e 's/0/3/' /etc/bareos/bareos-sd.d/device/drive0.conf > /etc/bareos/bareos-sd.d/device/drive3.conf

Now create the Autochanger control device:

# cat /etc/bareos/bareos-sd.d/autochanger/VTLControl.conf
Autochanger {
  Changer Command = "/usr/local/bin/bareos-sd-link \%o \%a \%S \%v"
  Changer Device = none
  Description = "VTLControl script relink device files"
  Device = drive0
  Device = drive1
  Device = drive2
  Device = drive3
  Name = VTLControl
}

Update Maximum Concurrent Jobs in /etc/bareos/bareos-sd.d/storage/bareos-sd.conf

Storage {
  Name = bareos-sd
  Maximum Concurrent Jobs = 4 # We have 4 drives
}

And finally, create a new script that do the trick:

# cat /usr/local/bin/bareos-sd-link
#!/bin/bash
# all storage volumes mounted under /var/lib/bareos/storage hierarchy, like:
# /dev/sdc                  1.0T   34M  1.0T   1% /var/lib/bareos/storage/volume1
# /dev/sdd                  1.0T   34M  1.0T   1% /var/lib/bareos/storage/volume2

logger -t Autochanger "Called with:${@};arguments."

COMMAND=$1 ; shift
DRIVEFILE=$1 ; shift
TAPESLOT=$1 ; shift
VOLUME=$1 # Could be missing

BASEDIR=$(dirname $DRIVEFILE)
cd ${BASEDIR} || exit 1

make_new() {
        local v=$1
        # Select less used mountpoint:
        touch $(df -kP | grep $BASEDIR | sort -k4nr | awk '{print $NF;exit}')/$v
}

case $COMMAND in
load|loaded)
        if [ ! -z $VOLUME ] ; then
                rm -f $DRIVEFILE
                [ -f */${VOLUME} ] || make_new ${VOLUME}
                ln -s */${VOLUME} $DRIVEFILE
        fi
        ;;
esac
# chmod +x /usr/local/bin/bareos-sd-link

Do not forget that all files should be owned by bareos user:

# chown -R bareos:bareos /etc/bareos/bareos-sd.d

Restart storage daemon:

# service bareos-sd restart

Changes to Director

It is enough to fix the device name in the /etc/bareos/bareos-dir.d/storage/File.conf file and add a couple of other parameters:

Storage {
  Name = File
  Address = bareos                # N.B. Use a fully qualified name here (do not use "localhost" here).
  Password = "WrwIH5axzI/zzXWIsgaPygOCOBrYJ/dZxYMlKNppOUqa"
  Media Type = File
  Device = VTLControl
  Auto Changer = yes
  Maximum Concurrent Jobs = 20	# Should be more than double of devices, control session is also counts as "jobs"
}

You can restart the director service or just reload it:

# echo reload | bconsole