librelist archives

« back to archive

Backing up block devices

Backing up block devices

From:
Manuel Faux
Date:
2015-04-01 @ 18:03
Hi,

I use LVM logical volumes as storage devices for my virtual machines. How 
can I backup those logical volumes using Attic?


When simply passing the device to attic create, I just receive an empty 
file named like the logical volume.


Thanks,

Manuel

Re: [attic] Backing up block devices

From:
Thomas Waldmann
Date:
2015-04-01 @ 18:49
in one of my branches there is stdin / stdout support, so you can pipe from
/ to dd.

Am Mittwoch, 1. April 2015 schrieb Manuel Faux :

>  Hi,
>
> I use LVM logical volumes as storage devices for my virtual machines. How
> can I backup those logical volumes using Attic?
>
>
>  When simply passing the device to attic create, I just receive an empty
> file named like the logical volume.
>
>
>  Thanks,
>
> Manuel
>

Re: [attic] Backing up block devices

From:
Yuri D'Elia
Date:
2015-04-02 @ 09:34
On 04/01/2015 08:03 PM, Manuel Faux wrote:
> Hi,
> 
> I use LVM logical volumes as storage devices for my virtual machines.
> How can I backup those logical volumes using Attic?

Attic is not the best tool for backing up block devices.

Re: [attic] Backing up block devices

From:
Manuel Faux
Date:
2015-04-02 @ 11:19
On Thursday, April 2, 2015 12:27, Dan Williams wrote:
> You could give ddar a try.
At first sight seems to do what I need. I'll give it a try. Thanks.

> Alternatively, I wonder if you have considered simply mounting the LVM
> volumes (as read-only) and then running Attic on them that way? If that's
> possible for you.
Unfortunately some of the virtual machines use file systems Linux cannot
(easily) deal with (e.g. OpenBSD UFS). But I'll keep that idea for later, in case
I cannot find another satisfying solution.

> Or, perhaps there's something you could do to expose the LVM volume as an
> ordinary file for Attic. I'm not sure how; I thought loopback, but that
> would also create a device... hmmmm.
Wouldn't I just have the same result than piping the block device to Attic? I
think the problem of Attric is not working with block devices per se, but with
huge single files, or am I wrong?

Regarding btrfs, I also thought about moving to btrfs, but did not have enough
arguments to do so, in the current environment. Maybe that gives that project
a little drive again.

________________________________________
From: attic@librelist.com <attic@librelist.com> on behalf of Dan Williams 
<dan@dotfive.co.uk>
Sent: Thursday, April 2, 2015 12:27
To: attic@librelist.com
Subject: RE: [attic] Backing up block devices

You could give ddar a try. I have not tried it myself, as it is not suitable
for my needs, but it accepts data from stdin so is a tar-like tool in that
regard. (Attic also accepts from stdin with Thomas's work, but as you have
found is not suitable for you, so...)

I would suggest reading the block device with dd and piping through ddar.

Alternatively, I wonder if you have considered simply mounting the LVM
volumes (as read-only) and then running Attic on them that way? If that's
possible for you.

Or, perhaps there's something you could do to expose the LVM volume as an
ordinary file for Attic. I'm not sure how; I thought loopback, but that
would also create a device... hmmmm.

This is in fact one of the reasons why I moved away from LVM to sparse file
images, because it makes things so much easier to back up and transfer. Also
I use Btrfs as much as possible so I have no need for LVM anymore. That
doesn't help you now but may possibly give a direction to investigate.



 >> -----Original Message-----
 >> From: attic@librelist.com [mailto:attic@librelist.com] On Behalf Of
 >> Manuel Faux
 >> Sent: 02 April 2015 11:08
 >> To: attic@librelist.com
 >> Subject: Re: [attic] Backing up block devices
 >>
 >> On Thursday, April 2, 2015 11:34, Yuri D'Elia wrote:
 >> > Attic is not the best tool for backing up block devices.
 >>
 >> After playing around a little bit with the stdin feature I also
 >> recognized that.
 >> The attic mount command does not work as I expected for huge single file
 >> backups, since it works very long (even longer than creating the backup
 >> took).
 >> After searching for a tool to backup my block devices I stumbled upon a
 >> page
 >> recommending attic for that. After exploring the tool a little bit I
 >> also did not
 >> expect attic to be the tool of my choice, but after reading the FAQ page
 >> (https://attic-backup.org/faq.html) where it states that VM images may
 >> be
 >> backed up with attic, I thought to give it a try.
 >>
 >> Any recommendations for a different tool?

Re: [attic] Backing up block devices

From:
Thomas Waldmann
Date:
2015-04-02 @ 16:19
> Wouldn't I just have the same result than piping the block device to
Attic?
> I think the problem of Attric is not working with block devices per se,
but with
> huge single files, or am I wrong?

What's the "problem of attic with huge single files" again?

Just wondering...

Re: [attic] Backing up block devices

From:
Dan Williams
Date:
2015-04-02 @ 11:42
 >> > Or, perhaps there's something you could do to expose the LVM volume as
 >> > an ordinary file for Attic. I'm not sure how; I thought loopback, but
 >> > that would also create a device... hmmmm.
 >> Wouldn't I just have the same result than piping the block device to
 >> Attic? I think the problem of Attric is not working with block devices
 >> per se, but with huge single files, or am I wrong?
 >> 

I would not say that you are wrong, but that's not how I understood it. In
my own testing I have backed-up some fairly large files - virtual machine
images among them - without any problems. Restoration is always extremely
quick for me with Attic. However, for me, these are ordinary files on the
filesystem, albeit sparse.

I had *thought* that the issue you were encountering was due to having to
use Attic with stdin, i.e. in piped fashion, but I could be wrong. I have
not used Attic in that way so cannot really compare.

Hence my thinking was that if your issue is related to the approach,
backing-up as actual files might help. It was pure speculation, as I don't
even know how Attic deals with the stdin stream - I also didn't really
understand the issue you mentioned with `attic mount`. Uncharted waters for
me... I just know that in my usage, large files are no problem whatsoever.

Not sure if that helps or not! :o)


Re: [attic] Backing up block devices

From:
Manuel Faux
Date:
2015-04-02 @ 10:07
On Thursday, April 2, 2015 11:34, Yuri D'Elia wrote:
> Attic is not the best tool for backing up block devices.

After playing around a little bit with the stdin feature I also recognized that.
The attic mount command does not work as I expected for huge single file
backups, since it works very long (even longer than creating the backup
took).
After searching for a tool to backup my block devices I stumbled upon a page
recommending attic for that. After exploring the tool a little bit I also did not
expect attic to be the tool of my choice, but after reading the FAQ page
(https://attic-backup.org/faq.html) where it states that VM images may be
backed up with attic, I thought to give it a try.

Any recommendations for a different tool?

Re: [attic] Backing up block devices

From:
Dan Williams
Date:
2015-04-02 @ 10:27
You could give ddar a try. I have not tried it myself, as it is not suitable
for my needs, but it accepts data from stdin so is a tar-like tool in that
regard. (Attic also accepts from stdin with Thomas's work, but as you have
found is not suitable for you, so...)

I would suggest reading the block device with dd and piping through ddar.

Alternatively, I wonder if you have considered simply mounting the LVM
volumes (as read-only) and then running Attic on them that way? If that's
possible for you.

Or, perhaps there's something you could do to expose the LVM volume as an
ordinary file for Attic. I'm not sure how; I thought loopback, but that
would also create a device... hmmmm.

This is in fact one of the reasons why I moved away from LVM to sparse file
images, because it makes things so much easier to back up and transfer. Also
I use Btrfs as much as possible so I have no need for LVM anymore. That
doesn't help you now but may possibly give a direction to investigate.



 >> -----Original Message-----
 >> From: attic@librelist.com [mailto:attic@librelist.com] On Behalf Of
 >> Manuel Faux
 >> Sent: 02 April 2015 11:08
 >> To: attic@librelist.com
 >> Subject: Re: [attic] Backing up block devices
 >> 
 >> On Thursday, April 2, 2015 11:34, Yuri D'Elia wrote:
 >> > Attic is not the best tool for backing up block devices.
 >> 
 >> After playing around a little bit with the stdin feature I also
 >> recognized that.
 >> The attic mount command does not work as I expected for huge single file
 >> backups, since it works very long (even longer than creating the backup
 >> took).
 >> After searching for a tool to backup my block devices I stumbled upon a
 >> page
 >> recommending attic for that. After exploring the tool a little bit I
 >> also did not
 >> expect attic to be the tool of my choice, but after reading the FAQ page
 >> (https://attic-backup.org/faq.html) where it states that VM images may
 >> be
 >> backed up with attic, I thought to give it a try.
 >> 
 >> Any recommendations for a different tool?

Re: [attic] Backing up block devices

From:
Manuel Faux
Date:
2015-04-05 @ 05:26
On Thursday, April 2, 2015 18:19, Thomas Waldmann wrote:

>> Wouldn't I just have the same result than piping the block device to Attic?
>> I think the problem of Attric is not working with block devices per se,
but with
>> huge single files, or am I wrong?
>
> What's the "problem of attic with huge single files" again?


I think I have judged Attic too fast. I've been using Attic now for a bit 
to backup

my virtual machine images (which are stored on LVM block devices, not files)

and in general I am happy with the backup procedure. I also had a look at ddar,

but I am not so happy with that since it lacks some basic features like prune or

mounting of images.


I use the stdin/stdout feature of Thomas' repo to backup and restore the images,

which works quite well. Also I am happy with the feature to change compression

to a minimum level, since I have enough backup disk space and want to speed up

the restore procedure.


All in all I'm impressed about Attic.