Solvedborg document pull-like operation

this is a FAQ (by people who have firewalls or want it for other reasons) and some people are evaluating setups with ssh -R (see some posts in #36).

this issue is to collect such setups and if evaluated successfully, add it to the documentation.

note: the debian/ubuntu package description says borg only supports push, maybe that can be removed after this ticket is closed.

so, if you successfully run a pull-like setup, the best thing you can do is to make a pull request that closes this ticket.


💰 there is a bounty for this

Note: to collect the bounty you need to run a reliable pull-like setup, do a pull request for our documentation, documenting the pull-related parts of the setup.

38 Answers

✔️Accepted Answer

While having documentation for this workaround is great, wouldn't it be better to add this functionality to borg itself? This kind of syntax would be awesome:

$ borg create /path/to/repo::example.com-now user@example.com:/

Other Answers:

While having documentation for this workaround is great, wouldn't it be better to add this functionality to borg itself? This kind of syntax would be awesome:

$ borg create /path/to/repo::example.com-now user@example.com:/

Was this ever implemented?

FWIW, I have made a small hack which works with socat, thus saving the SSH-in-SSH overhead and obliterating the need for the remote machine to have an account on the local machine. Using --append-only and --restrict-to-path, this should be as safe as Borg is, but I’d like any feedback on that.

First, we create socat-wrap.sh, which we will use as BORG_RSH:

#!/bin/bash
exec socat STDIO TCP-CONNECT:localhost:12345

Locally, we run socat to offer the borg service:

socat TCP-LISTEN:12345,fork \
    "EXEC:borg serve --append-only --restrict-to-path $PATH_TO_REPOSITORIES --umask 077"

(omit the ,fork if you want to allow only exactly one borg command to be run)

Now we invoke borg on the remote using ssh, forwarding the port:

ssh -R 12345:localhost:12345 sourcehost \
    BORG_RSH="/home/horazont/socat-wrap.sh" \
    borg init -e none ssh://foo/$PATH_TO_REPOSITORIES/some_repository

foo is completely arbitrary; one could substitute anything here, because the socat-wrap.sh ignores its arguments.


Of course, it’s also possible to do the same with UNIX sockets, providing more isolation.

socat-wrap.sh:

#!/bin/bash
exec socat STDIO UNIX-CONNECT:/home/horazont/borg-remote.sock
socat UNIX-LISTEN:/home/horazont/borg-local.sock,fork \
    "EXEC:borg serve --append-only --restrict-to-path $PATH_TO_REPOSITORIES --umask 077"
ssh -R /home/horazont/borg-local.sock:/home/horazont/borg-remote.sock sourcehost \
    BORG_RSH="/home/horazont/socat-wrap.sh" \
    borg init -e none ssh://foo/$PATH_TO_REPOSITORIES/some_repository

ssh is friendly enough to automatically set very strict permissions on the socket on the remote side.

A new round of fun with pull-like operation.

I wrapped the pulling side in systemd units:

borg-remote-repositories.socket

[Unit]
Description=Socket for accessing a specific path as borg repositories

[Socket]
ListenStream=/data/test/borg.sock
Accept=yes

borg-remote-repositories@.service

[Unit]
Description=Borg serve

[Service]
Type=simple
ExecStart=/usr/bin/borg serve --append-only --restrict-to-path /data/test/repos/ --umask 077
StandardInput=socket
StandardOutput=socket
StandardError=journal
User=remote-backups
Group=remote-backups
ProtectSystem=strict
PrivateTmp=yes
PrivateNetwork=yes
PrivateDevices=yes
ProtectKernelTunables=yes
RestrictAddressFamilies=
ReadWritePaths=/data/test/repos/

This makes the borg serve:

  • run under its own user (remote-backups -- make sure that user has rwx permissions on /data/test/repos and everything therein)
  • have ~no privileges on the system: no network access, no device access, no access to a shared tmp, no write access to the system etc.
  • be able to run multiple times, once for each client connecting to the socket

To execute a backup, one can use for example:

ssh -R /root/borg.sock:/data/test/borg.sock root@remote-host BORG_RSH="'bash -c \"exec socat STDIO UNIX-CONNECT:/root/borg.sock\"'" borg create -p ssh://remote/data/test/repos/remotely-created::postgres-$(date --iso-8601=seconds) /var/lib/postgresql-backups/ ';' rm /root/borg.sock

The rm /root/borg.sock helps with cleanup in case the remote server cannot be configured to do StreamLocalBindUnlink.

(Of course, you’d normally not use root but instead a user with sudo privileges for exactly the required borg create commands.)

Unless I'm blind, I don't think anyone spoke about the fact a complete pull system with sshfs started before Borg is doable, without a root login (specific sudo right on the remote target is required).
The trick lies with -o sftp_server and sudo :

sshfs user@host:/  /local/mount/dir  -o ro -o sftp_server="sudo /usr/lib/openssh/sftp-server"

Adjust sftp_server arg to the sshd_config subsystems entry.

To have this working, you'll need :

  1. a dedicated user on the remote server. It can be a system user without password, but a home and shell are required. No specific group or rights aside this file in the sudoers (adjust the username) :
# sudoers file : /etc/sudoers.d/borg
borg ALL=NOPASSWD:/usr/lib/openssh/sftp-server
  1. this user will also need in his ~/.ssh directory the public key of the user running Borg on the backup server.

Now try to connect to the target server with ssh, and retry with sshfs. You'll see all files can be accessed, due to sftp-server running as root.
Borg can now start to backup the remote server using the mount point.

Only limitation for now is the fact the backup will have inside the full path of the mount point. And this will also need to be set as a prefix on all paths to backup and exclude.
For example : borg create ... repo::backup-set /mount/point/etc /mount/point/boot /mount/point/home /mount/point/usr --exclude /mount/point/usr/cache/

Related Issues:

50
borg document pull-like operation
While having documentation for this workaround is great wouldn't it be better to add this functional...
3
borg DistributionNotFound error with msgpack-python 0.5.0 installed
A slightly dirty workaround is to pip install msgpack-python. Borg fails to launch with msgpack-pyth...
120
laravel backup Target [Spatie\Backup\Tasks\Cleanup\CleanupStrategy] is not instantiable while building
I found that ensuring the package is removed: composer remove spatie/laravel-backup clear the cache ...
63
restic Dry-Run Backup
Hm interesting idea thanks Maybe we can add this once #1494 is done e.g add some kind of backup --dr...
26
elasticsearch dump Content-Type header [] is not supported
I answered my own question with a simple read of the documentation In order to help us troubleshoot ...
26
laravel backup File Not Found Exception Message when using Google Drive
That error is because you can't use the folder name you have to get the folder ID of the google driv...
17
restic Cannot find docs for the exclude file syntax
To answer a few of your questions already: All patterns are tested against the full path of a file/d...
17
oxidized Rugged::SshError: Failed to authenticate SSH session
If you have -----BEGIN OPENSSH PRIVATE KEY----- in your private key you need to change it to PEM for...
15
backup ERROR: Failed to build gem native extension (json-1.8.2) when installing
I can't install backup gem on my UNIX server: I tried it with a newer json gem: But this didn't help...
9
backup DropboxError: server response does not have offset key
I made a working fork with api V2 it's require api_token instead key and secret Token can be generat...
8
velero Add support for restoring to another zone [GCP]
Thanks for the script @stevegore -- I had to make some changes to get it to work for me but knowing ...
3
restic support for the new Scaleway Object Storage (S3 compatible)
FYI this works both on nl-ams or fr-par region (with no ~/.aws/*): We do support SigV4 ...
320
Pillow ImportError: cannot import name 'PILLOW_VERSION' from 'PIL' (unknown location)
I'm getting the same error right now after conda install pillow on Win 10 It installed pillow 6.2.1 ...
227
neovim build fails with OSX Mojave
Starting with Mojave the headers are no longer installed under /usr/include/ by default -- look unde...
189
Pillow zlib requirement on Alpine Linux
@wiredfool Thanks I was having the same problem building a docker image hey ...
167
vim Error message printed first time python3 (version 3.7.0) dynamic library is imported
Regardless of installed plugins silently execute python3 once on the top of your vimrc: ...
122
Pillow cannot write mode RGBA as JPEG (4.2.0)
@AymericHENRY Hi While this is just reiterating what has already been said you should have no proble...
105
Pillow Can't install Pillow on Android
It does the trick: $ pkg install python libjpeg-turbo libcrypt ndk-sysroot clang zlib $ LDFLAGS=-L${...
90
neovim Build failed with Homebrew on both macOS 11.4 and 12.0.1
This should be fixed in Homebrew for now Neovim version (nvim -v) Upgrading neovim HEAD-487286b -> H...
77
arrayfire NVCC does not support Apple Clang version 8.x
@joseph-zhong it looks like you're using Xcode 8.3 which CUDA (v8.0.61) does not yet support :( Down...
69
Win32 OpenSSH Can't add keys to ssh-agent, communication with agent failed
That check in ssh-agent can be removed as sshd now runs as SYSTEM We'll fix it As a workaround to un...
56
xrdp Problem on debian - black screen
Hi I ran into the same issue on Debian testing and opened a bug on the Debian bug tracking system (D...
55
neovim Update tree-sitter to v0.19.0
Workaround for homebrew users: Uninstall tree-sitter@0.19.1 ignoring neovim dependency Extract the f...
54
systemd systemd can't handle the process previlege that belongs to user name startswith number, such as 0day
Let's look if it's actually invalid.. Submission type Bug report systemd version the issue has been ...
52
vim E363: pattern uses more memory than 'maxmempattern'
Vim 7.4 is getting a little long in the teeth The current state-of-the art Vim is 8.0.1097 as of thi...
48
systemd systemctl - Useless error message - "Failed to enable unit: Invalid Argument"
In the mean time I came across a new-to-me command for possibly resolving invalid argument errors: s...
45
scrcpy adb server version (41) doesn't match this client (40)
fixed this by doing the following: going into GenyMotion settings -> ADB tab instead of Use Genymoti...
41
Pillow Unable to install on macOS Big Sur (zlib missing)
Run export SYSTEM_VERSION_COMPAT=1 before all other scripts They say Big Sur is both 10.16 and 11.0 ...
41
vim netrw's "gx" functionality doesn't open URLs anymore in macOS
I found yet another issue with netrw not opening URLs with @ and I decided enough is enough I starte...
40
vim Can't get transparent background with termguicolors enabled
At this moment this issue is solely about: Since to me the fact of being able to get transparent bac...
39
vcpkg Remove multiple packages at once (wildcards)
How about allowing syntax .\vcpkg.exe remove *:x86-windows to remove the entire x86-windows triplet?...
38
scrcpy not working with OS Catalina
I managed to fix this: cd /usr/local/Caskroom/android-platform-tools/29.0.5/platform-tools && open ....
38
neovim homebrew build fails to start (segfault in libtermkey 0.21)
EDITOR=/usr/bin/vim brew edit libtermkey edit the URL to libtermkey-0.20.tar.gz and the hash to 6c0d...
34
neovim Feature: optional floating window borders
This works for me Not sure why but it does. #6619 Added floating windows but no way to create a bord...
34
systemd "Restart" and/or "RestartForceExitStatus" should work with oneshot services, too
oneshot services managing their own restart policy (in a bash retry loop or similar) is similar to s...
31
kitty Error opening terminal: xterm-kitty
From stackoverflow: export TERM=xterm And it works. Any ideas how to fix this? This is a brand new M...
29
vim Right click enables visual mode by default
This is by design See :help defaults.vim for detail This happens if you don't have .vimrc ...
26
vim Editing a Groovy file causes Vim to throw an error - E945: Range too large in character class
At first I could not reproduce it either Describe the bug When I edit a Groovy file Vim throws an er...
25
neovim Failed to load python host
@brycearden I had that same issue as yours later I found out that pip and pip3 were both aliased to ...
25
Pillow The headers or library files could not be found for zlib, a required dependency when compiling Pillow from source
I figured it was my pip version that was behind I close the issue as upgrade the pip version fixed i...
25
vifm How to use bat as a fileviewer
Make sure that your truecolor terminal sets the COLORTERM variable to either truecolor or 24bit Othe...
24
libvips Install via package managers on Linux?
Amazon Linux 2 seems to be based on RHEL/CentOS 7[1] so I assume you could use Remi's RPM repository...
23
fluent bit Errors when forwarding to ElasticSearch
@edsiper I've got the same problem also related to es_rejected_execution_exception but: It should st...
23
systemd Access mounted (ip netns ...) network namespaces.
FWIW I came up with a simpler minimalistic version of @CrackerJackMack's solution above: Then ...
22
allegro5 Android build fails due to deprecated 'android' command
I have searched and found no replacement for the android update project command ...
22
neovim Pip already installed neovim but module not found
Try pip uninstalling both neovim and pynvim and then reinstalling just pynvim -- that worked for me ...
22
xrdp VNC Problem connecting
[SOLVED] I got the same error in CentOS 7.5 with Xfce desktop I had struggled for several days with ...
21
awesome Intellij dialog windows close immediately
Same problems with PyCharm 2018.1 and Idea 2018.1 Popups are problematic As @p-himik in #2233 said c...
21
Pillow Unable to install Pillow on the Macbook Pro 13 M1 Silicon
If it helps anyone on Apple Silicon brew install libjpeg pip install Pillow works ! ...
20
neovim cannot copy unicode characters
I had a similar problem: when copying texts letters with accents were swapped by other characters (e...