Backup solution - Deja-dup is not reliable

Help
1

I made a backup with Deja-dup, but I am unable to restore without errors. I don’t think this is related to NixOS because I already have this problem on other distros. It is deplorable to see that the backup you thought was safe was in fact not.

Do you have suggestions for a replacement? I was thinking about simply creating tar archives. But the problem with this solution is that that I can’t do incremental backups. I have to back up everything each time.

I would like your feedback because I don’t want to try another solution and discover later that it is not working.

2

I do mine via borgbackup, using the NixOS service. Pika is a gui option for this case.

I haven’t had to restore, but I have successfully mounted my backup to check.

2 Likes
3

I do my backups with restic + rclone. I’ve had no issues so far.

2 Likes
4

For what it’s worth, I have been using Déjà Dup for years and have not encountered any errors when restoring.

It would be helpful if you could open an issue either in Nixpkgs, or in Déjà Dup issue tracker if it can be reproduced on other distros.

1 Like
5

This is quite curious. It never worked for me.

I would like to help but I don’t know what is causing the problem and I cannot send my personal data for testing. I will keep an eye on this to see if I can find anything relevant.

6

I doubt that Déjà Dup is just unable to restore backups: if it’s failing reliably there’s probably something strange with your setup, maybe some filesystem incompatibility or permissions issue?

Anyway, you can try bup. It’s a simple command line tool and can be used similarly to tar if you like that (see the join and split subcommands specifically) . However, unlike tar, it does incremental backups and file deduplication by using the git file format.

In my experience it’s very reliable and even if you encouter some issue the author is available to help you sort it out.

1 Like
7

Even getting the error message and console output when running DEJA_DUP_DEBUG=1 deja-dup might be helpful for determining where the issue is. Hopefully any personal data can be redacted.

8

Since Déjà Dup was causing an error while restoring my folder, I decided to make a script to decompress its archives manually. Then, I had an error while expending one of its archives.

The files I was restoring were originally on my Mac. Today, I created a tar archive of my music on the Mac. I had no issue creating the archive itself. But I had an error while decompressing the archive on NixOS. The error message was something saying that the archive ended unexpectedly.

I don’t know what to think about this experience. Do you have ideas?

9

OK, I’ll try that. Here are some characters to make the message longer, so I can post.

10

Maybe I should try with GNU Tar.

11

I’ve had an issue once from Mac to Linux with a tar archive, because there was a “/” character in a filename that was in the archive.

12

macOS has xattrs that do not exist on Linux. If your files originate from macOS, that might be the issue and there’s likely a flag to ignore these “errors”.

13

Wait, what were you backing up – we only have Déjà Dup packaged for Linux. Did you try to back up files from native macOS drive (e.g. APFS), some other file system on an external drive, or did you copy the files to Linux FS and try to back up that.

I would expect Linux FSs to work properly but the APFS might be problematic as APFS Linux drivers are probably less battle tested and duplicity (the backup tool Déjà Dup uses internally) itself might have issues with APFS metadata.

And looks like duplicity does not even use straight tar any more so I would not be surprised if trying to unpack that with tar would fail.

But if extracting regular tar files created on MacOS does not work either, that might indicate metadata issue as Atemu mentions.

14

I prefer btrbk for my backups for the simple reason that the result is just a mirror of the origin file system. Restore boils down to a simple copy, no need to extract archives or interpret and meta information about whats where.

15

This deserves clarification.

My files were on the Mac. I created .tar.gz archives of the files I wanted to transfer to NixOS using the default tar command of the Mac (non GNU). These archives were created on an external FAT32 disk that both systems can understand. I decompressed these archives on NixOS also using tar. Later on, I made a back-up with Deja Dup on NixOS. I reformatted my main NixOS disk to have full-disk encryption. After that, I wanted to restore my files from my back-up (from an ext4 partition on the same external drive I used to transfer from Mac to NixOS). Then, I was unable to do my restore without errors.

Also, as I mentioned, I was unable to decompress my music archive from the Mac on NixOS. The other archives gave full of warning about flags GNU tar was unable to understand. But, this is an expected behaviour (meta information about an alien file system). Otherwise, I had no issues with these archives.

What is curious is seeing Deja Dup unable to restore files of a native Linux file system. This is the part I don’t understand.

16

I used GNU Tar to compress my Music folder on the Mac, and I was able to decompress it without error on NixOS.

17

How large is your music archive? If it’s more than 4,294,967,295 Bytes, it won’t fit into a single file on FAT32. If your Music dir was larger than that and the archive file was truncated, that’d explain the “unexpected end of file” errors.

Always use exFAT for compatibility between modern operating systems. FAT32 should only be used for legacy systems or similarly limited systems such as embedded firmware.

1 Like
18

I didn’t mention it, but I used split to mitigate this problem. So, the “unexpected end of file” errors cannot be explained by that.

19

If you split the archive, you must concat it again when extracting.

20

Yes, of course. I did.