Skip to content

Instantly share code, notes, and snippets.

@csarn

csarn/README.md

Last active May 19, 2024 20:21
Show Gist options
  • Save csarn/3084913ba3e78da6c4d511df131c4d5f to your computer and use it in GitHub Desktop.
Save csarn/3084913ba3e78da6c4d511df131c4d5f to your computer and use it in GitHub Desktop.
Download ZIP
ZFS pull backup with minimal permissions
Raw
README.md

Usage

on system that should be backed up

  • put "restrict_commands.sh" in /usr/local/bin and make it executable
  • install ts, lzop and optionally mbuffer
useradd zfsbackup --create-home --system
mkdir /home/zfsbackup/.ssh
zfs allow -u zfsbackup send,hold tank/dataset
echo 'restrict,command="restrict_commands.sh" ssh-ed25519 ...' > /home/zfsbackup/.ssh/authorized_keys
chown zfsbackup:zfsbackup /home/zfsbackup/.ssh -R

on backup server

run cronjob with:

syncoid --no-sync-snap --no-privilege-elevation --sendoptions=Rw zfsbackup@target:tank/dataset tank/dataset
Raw
restrict_commands.sh
#!/bin/bash
export PATH=/usr/sbin:$PATH
_RE_DATASET=$'[\"\']+[a-z0-9/_]+[\"\']+'
_RE_SNAPSHOT=$'[\"\']+[a-z0-9/_]+[\"\']*@[\"\']*[a-z0-9/_:-]+[\"\']+'
_RE_SIZE=$'[0-9]+[kMG]?'
_WHITELIST=(
-e "exit"
-e "echo -n"
-e "zpool get -o value -H feature@extensible_dataset $_RE_DATASET"
-e "zfs get -H syncoid:sync $_RE_DATASET"
-e "zfs get -Hpd 1 -t snapshot guid,creation $_RE_DATASET"
-e " *zfs send -R -w -nv?P $_RE_SNAPSHOT"
-e " *zfs send -R -w +$_RE_SNAPSHOT \| +lzop *(\| mbuffer +-q -s $_RE_SIZE -m $_RE_SIZE 2>/dev/null)?"
-e " *zfs send -R -w -nv?P +-I $_RE_SNAPSHOT $_RE_SNAPSHOT"
-e " *zfs send -R -w +-I $_RE_SNAPSHOT $_RE_SNAPSHOT \| lzop *(\| mbuffer +-q -s $_RE_SIZE -m $_RE_SIZE 2>/dev/null)?"
-e "command -v (lzop|mbuffer)"
)
## LOG non-whitelisted commands, execute whitelisted
echo "$SSH_ORIGINAL_COMMAND" |\
tee >(egrep -x -v "${_WHITELIST[@]}" \
| ts "non-whitelisted command issued: (client $SSH_CLIENT)" \
| logger -p local0.crit \
) |\
egrep -x "${_WHITELIST[@]}" | bash
@alembiq
Copy link

alembiq commented Apr 1, 2024

any luck? i'm stucked on the lzop error also :(

syncoid --sshkey=/var/lib/syncoid/kubera-2022-05-31 backup@10.10.10.10:zpool/nixos/etc tank/test --no-privilege-elevation --debug --sendoptions="w c " --sshoption=StrictHostKeyChecking=off
DEBUG: SSHCMD: ssh  -o StrictHostKeyChecking=off  -i /var/lib/syncoid/kubera-2022-05-31
loginShellInit
DEBUG: checking availability of lzop on source...
DEBUG: checking availability of lzop on target...
DEBUG: checking availability of lzop on local machine...
DEBUG: checking availability of mbuffer on source...
DEBUG: checking availability of mbuffer on target...
DEBUG: checking availability of pv on local machine...
DEBUG: checking availability of zfs resume feature on source...
DEBUG: checking availability of zfs resume feature on target...
DEBUG: syncing source zpool/nixos/etc to target tank/test.
DEBUG: getting current value of syncoid:sync on zpool/nixos/etc...
ssh  -o StrictHostKeyChecking=off  -i /var/lib/syncoid/kubera-2022-05-31 -S /tmp/syncoid-backup@10.10.10.10-1712001241 backup@10.10.10.10  zfs get -H syncoid:sync ''"'"'zpool/nixos/etc'"'"''
DEBUG: checking to see if tank/test on  is already in zfs receive using  ps -Ao args= ...
DEBUG: checking to see if target filesystem exists using "  zfs get -H name 'tank/test' 2>&1 |"...
DEBUG: getting list of snapshots on zpool/nixos/etc using ssh  -o StrictHostKeyChecking=off  -i /var/lib/syncoid/kubera-2022-05-31 -S /tmp/syncoid-backup@10.10.10.10-1712001241 backup@10.10.10.10  zfs get -Hpd 1 -t snapshot guid,creation ''"'"'zpool/nixos/etc'"'"'' |...
DEBUG: creating sync snapshot using "ssh  -o StrictHostKeyChecking=off  -i /var/lib/syncoid/kubera-2022-05-31 -S /tmp/syncoid-backup@10.10.10.10-1712001241 backup@10.10.10.10  zfs snapshot ''"'"'zpool/nixos/etc'"'"''@syncoid_kubera_2024-04-01:21:54:02-GMT02:00
"...
loginShellInit
DEBUG: target tank/test does not exist.  Finding oldest available snapshot on source zpool/nixos/etc ...
DEBUG: getting estimated transfer size from source -S /tmp/syncoid-backup@10.10.10.10-1712001241 backup@10.10.10.10 using "ssh  -o StrictHostKeyChecking=off  -i /var/lib/syncoid/kubera-2022-05-31 -S /tmp/syncoid-backup@10.10.10.10-1712001241 backup@10.10.10.10  zfs send -w -c  -nvP ''"'"'zpool/nixos/etc@autosnap_2024-03-28_22:13:01_monthly'"'"'' 2>&1 |"...
DEBUG: sendsize = 782528
INFO: Sending oldest full snapshot zpool/nixos/etc@autosnap_2024-03-28_22:13:01_monthly (~ 764 KB) to new target filesystem:
DEBUG: ssh  -o StrictHostKeyChecking=off  -i /var/lib/syncoid/kubera-2022-05-31 -S /tmp/syncoid-backup@10.10.10.10-1712001241 backup@10.10.10.10 ' zfs send -w -c  '"'"'zpool/nixos/etc'"'"'@'"'"'autosnap_2024-03-28_22:13:01_monthly'"'"' | lzop  | mbuffer  -q -s 128k -m 16M 2>/dev/null' | mbuffer  -q -s 128k -m 16M 2>/dev/null | lzop -dfc | pv -p -t -e -r -b -s 782528 |  zfs receive  -s -F 'tank/test'
DEBUG: checking to see if tank/test on  is already in zfs receive using  ps -Ao args= ...
lzop: <stdin>: not a lzop file
mbuffer: error: outputThread: error writing to <stdout> at offset 0x10000: Broken pipe
0.00 B 0:00:00 [0.00 B/s] [>                                                                                                                                                                                                                 ]  0%
cannot receive: failed to read from stream
mbuffer: warning: error during output to <stdout>: Broken pipe
CRITICAL ERROR: ssh  -o StrictHostKeyChecking=off  -i /var/lib/syncoid/kubera-2022-05-31 -S /tmp/syncoid-backup@10.10.10.10-1712001241 backup@10.10.10.10 ' zfs send -w -c  '"'"'zpool/nixos/etc'"'"'@'"'"'autosnap_2024-03-28_22:13:01_monthly'"'"' | lzop  | mbuffer  -q -s 128k -m 16M 2>/dev/null' | mbuffer  -q -s 128k -m 16M 2>/dev/null | lzop -dfc | pv -p -t -e -r -b -s 782528 |  zfs receive  -s -F 'tank/test' failed: 256 at /usr/sbin/syncoid line 492.

@lediur
Copy link

lediur commented Apr 27, 2024 •
edited

I also encountered the above mysterious lzop: <stdin>: not a lzop file error. I initially tried to work around it by specifying --compress=none, but that resulted in a different cannot receive: failed to read from stream. Turns out this is a bit of a red herring.

/var/log/syslog on the remote host (source of the dataset) revealed that it was catching multiple "non-whitelisted" commands. It turns out that the two _RE_DATASET and _RE_SNAPSHOT regex in the restrict_commands.sh above don't account for capital letters in either the dataset or snapshot, and syncoid was trying to send snapshots that started with TEMP_.

Changing those two lines to:

_RE_DATASET=$'[\"\']+[A-Za-z0-9/_]+[\"\']+'
_RE_SNAPSHOT=$'[\"\']+[A-Za-z0-9/_]+[\"\']*@[\"\']*[A-Za-z0-9/_:-]+[\"\']+'

resolved my issue. I'm now happily pulling snapshots from my primary server to my backup server. @Mikesco3 harder to tell if this resolves your issue specifically, but @alembiq I do see a capital GMT in your log output so you might give it a shot.

@alembiq
Copy link

alembiq commented May 16, 2024

me bad, i never used the script above, I got this error from using syncoid from default package, but I'm getting the error above

should have read the thread more in detail before posting my question :( sorry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment