Potential for removing "-F" on receive

[treydock]

I've noticed that one side-effect of using "-F" for a receive is that all the snapshots that exist on the archival host are removed if they were also removed on the host being backed up. This presents an issue if the goal of an archival host is to retain snapshots longer than the system its backing up. My initial idea for archival was to keep hourly for a day, daily for a week, and weekly for a month on a production system and double those periods on the archival host.

What would be, or have been, the issues of not using the "-F" flag?

Thanks,

• Trey

[notr1ch]

I was looking into this as my requirements are similar to what you want - lots of snapshots on the backup host, not as many on the target due to space constraints. I'm very new to ZFS, but from what I can tell, -F is needed to rollback the dataset to the previous snapshot so that the incremental send can function, as without -F there is a risk that the dataset has been modified on the backuphost and thus the incremental sends won't be in sync (I hope this is is explained right!). Another effect of -F is that it deletes any snapshots not present on the sender (why?!).

Using zfs rollback seems like it could accomplish the same goal to avoid a modified dataset. For now I've removed -F from simplesnap and made the target datasets read-only on the backuphost so there's no risk of accidental modification. This seems to work so far.. fingers crossed.

Update: It seems the readonly attribute is lost after the zfs receive so this isn't an ideal solution.

[notr1ch]

After putting this into production, it's failing with errors like "cannot receive incremental stream: destination tank/backups/db has been modified" or "cannot receive incremental stream: invalid backup stream". Unfortunately without the ability to maintain snapshots on the backup host, simplesnap isn't going to work for me.

[jgoerzen]

Is that with -F removed? I've been using simplesnap (as written) for a long time now and have never seen a problem like this.

[notr1ch]

Sorry I should have been more explicit, that was with -F removed. I did notice that due to a long run time, a snapshot on the backup host was created by a cron job in the middle of a receive operation, which I assume was responsible for one of those errors.

I've wiped everything out and started from scratch, if I continue to experience issues I'll go back to using -F and try to come up with some other way to achieve what I want (cloning?)

[jgoerzen]

It is not an issue to create snapshots on the machine that is sending the backups while a backup is running. It is probably unwise to be creating new snapshots over the backup data on the system that receives the backups. They will necessarily have to be removed anyhow. - John

…

On 03/24/2017 09:44 AM, Richard Stanway wrote: Sorry I should have been more explicit, that was with -F removed. I did notice that due to a long run time, a snapshot on the backup host was created by a cron job in the middle of a receive operation, which I assume was responsible for one of those errors. I've wiped everything out and started from scratch, if I continue to experience issues I'll go back to using -F and try to come up with some other way to achieve what I want (cloning?) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAG5HR3DqXCpQTnyv33srYoSAKYKnZWPks5ro_LzgaJpZM4E8Tq6>.

[notr1ch]

As long as the automatically created simplesnap snapshot is the latest snapshot for the dataset, shouldn't it be OK to maintain older snapshots of the backup data? While admittedly I'm very new to ZFS, I don't really understand why it's necessary to remove the snapshots on the backup dataset to receive the latest one.

[jplitza]

-F doesn't remove snapshots on the receiving host that are older than the anchor snapshot, i.e. the newest snapshot by simplesnap. It does a rollback of the receiving side to the anchor snapshot, which IIRC is necessary because simplesnap creates lockfiles inside the datasets themselves. But snapshots older than the anchor snapshot that were manually created (or at least not created by simplesnap) are never touched by simplesnap.