Solvedborg document pull-like operation

this is a FAQ (by people who have firewalls or want it for other reasons) and some people are evaluating setups with ssh -R (see some posts in #36).

this issue is to collect such setups and if evaluated successfully, add it to the documentation.

note: the debian/ubuntu package description says borg only supports push, maybe that can be removed after this ticket is closed.

so, if you successfully run a pull-like setup, the best thing you can do is to make a pull request that closes this ticket.

💰 there is a bounty for this

Note: to collect the bounty you need to run a reliable pull-like setup, do a pull request for our documentation, documenting the pull-related parts of the setup.

While having documentation for this workaround is great, wouldn't it be better to add this functionality to borg itself? This kind of syntax would be awesome:

$ borg create /path/to/repo::example.com-now user@example.com:/

While having documentation for this workaround is great, wouldn't it be better to add this functionality to borg itself? This kind of syntax would be awesome:

$ borg create /path/to/repo::example.com-now user@example.com:/

Was this ever implemented?

FWIW, I have made a small hack which works with socat, thus saving the SSH-in-SSH overhead and obliterating the need for the remote machine to have an account on the local machine. Using --append-only and --restrict-to-path, this should be as safe as Borg is, but I’d like any feedback on that.

First, we create socat-wrap.sh, which we will use as BORG_RSH:

#!/bin/bash
exec socat STDIO TCP-CONNECT:localhost:12345

Locally, we run socat to offer the borg service:

socat TCP-LISTEN:12345,fork \
    "EXEC:borg serve --append-only --restrict-to-path $PATH_TO_REPOSITORIES --umask 077"

(omit the ,fork if you want to allow only exactly one borg command to be run)

Now we invoke borg on the remote using ssh, forwarding the port:

ssh -R 12345:localhost:12345 sourcehost \
    BORG_RSH="/home/horazont/socat-wrap.sh" \
    borg init -e none ssh://foo/$PATH_TO_REPOSITORIES/some_repository

foo is completely arbitrary; one could substitute anything here, because the socat-wrap.sh ignores its arguments.

Of course, it’s also possible to do the same with UNIX sockets, providing more isolation.

socat-wrap.sh:

#!/bin/bash
exec socat STDIO UNIX-CONNECT:/home/horazont/borg-remote.sock

socat UNIX-LISTEN:/home/horazont/borg-local.sock,fork \
    "EXEC:borg serve --append-only --restrict-to-path $PATH_TO_REPOSITORIES --umask 077"

ssh -R /home/horazont/borg-local.sock:/home/horazont/borg-remote.sock sourcehost \
    BORG_RSH="/home/horazont/socat-wrap.sh" \
    borg init -e none ssh://foo/$PATH_TO_REPOSITORIES/some_repository

ssh is friendly enough to automatically set very strict permissions on the socket on the remote side.

A new round of fun with pull-like operation.

I wrapped the pulling side in systemd units:

borg-remote-repositories.socket

[Unit]
Description=Socket for accessing a specific path as borg repositories

[Socket]
ListenStream=/data/test/borg.sock
Accept=yes

borg-remote-repositories@.service

[Unit]
Description=Borg serve

[Service]
Type=simple
ExecStart=/usr/bin/borg serve --append-only --restrict-to-path /data/test/repos/ --umask 077
StandardInput=socket
StandardOutput=socket
StandardError=journal
User=remote-backups
Group=remote-backups
ProtectSystem=strict
PrivateTmp=yes
PrivateNetwork=yes
PrivateDevices=yes
ProtectKernelTunables=yes
RestrictAddressFamilies=
ReadWritePaths=/data/test/repos/

This makes the borg serve:

• run under its own user (remote-backups -- make sure that user has rwx permissions on /data/test/repos and everything therein)
• have ~no privileges on the system: no network access, no device access, no access to a shared tmp, no write access to the system etc.
• be able to run multiple times, once for each client connecting to the socket

To execute a backup, one can use for example:

ssh -R /root/borg.sock:/data/test/borg.sock root@remote-host BORG_RSH="'bash -c \"exec socat STDIO UNIX-CONNECT:/root/borg.sock\"'" borg create -p ssh://remote/data/test/repos/remotely-created::postgres-$(date --iso-8601=seconds) /var/lib/postgresql-backups/ ';' rm /root/borg.sock

The rm /root/borg.sock helps with cleanup in case the remote server cannot be configured to do StreamLocalBindUnlink.

(Of course, you’d normally not use root but instead a user with sudo privileges for exactly the required borg create commands.)

Unless I'm blind, I don't think anyone spoke about the fact a complete pull system with sshfs started before Borg is doable, without a root login (specific sudo right on the remote target is required).
The trick lies with -o sftp_server and sudo :

sshfs user@host:/  /local/mount/dir  -o ro -o sftp_server="sudo /usr/lib/openssh/sftp-server"

Adjust sftp_server arg to the sshd_config subsystems entry.

To have this working, you'll need :

1. a dedicated user on the remote server. It can be a system user without password, but a home and shell are required. No specific group or rights aside this file in the sudoers (adjust the username) :

# sudoers file : /etc/sudoers.d/borg
borg ALL=NOPASSWD:/usr/lib/openssh/sftp-server

2. this user will also need in his ~/.ssh directory the public key of the user running Borg on the backup server.

Now try to connect to the target server with ssh, and retry with sshfs. You'll see all files can be accessed, due to sftp-server running as root.
Borg can now start to backup the remote server using the mount point.

Only limitation for now is the fact the backup will have inside the full path of the mount point. And this will also need to be set as a prefix on all paths to backup and exclude.
For example : borg create ... repo::backup-set /mount/point/etc /mount/point/boot /mount/point/home /mount/point/usr --exclude /mount/point/usr/cache/