librelist archives

« back to archive

Handling of sparse files

Handling of sparse files

From:
Date:
2014-11-28 @ 14:05
Hello,

i'm experimenting with using attic as a backup for a libvirt-based 
VM-Host.

Unfortunately most disk-images libvirt works with are sparsely created, 
and attic doesn't seem to handle them intelligently - it crunches on 
gigabytes of zeroes. In my tests, attic takes about 10 Minutes for a 40GB 
qcow2 image with 3GB allocated, while taking only about 1.5 minutes for a 
tarred (tar --sparse) version of the same file.

Additionally, "attic extract" will not create those files sparsely.

Would efficient handling of sparse files be something worth implementing?

Best Regards
 Heiko

Re: [attic] Handling of sparse files

From:
Crest da Zoltral
Date:
2014-11-29 @ 10:20
> On 28 Nov 2014, at 15:05, heiko.helmle@horiba.com wrote:
> 
> Hello, 
> 
> i'm experimenting with using attic as a backup for a libvirt-based VM-Host. 
> 
> Unfortunately most disk-images libvirt works with are sparsely created, 
and attic doesn't seem to handle them intelligently - it crunches on 
gigabytes of zeroes. In my tests, attic takes about 10 Minutes for a 40GB 
qcow2 image with 3GB allocated, while taking only about 1.5 minutes for a 
tarred (tar --sparse) version of the same file. 
> 
> Additionally, "attic extract" will not create those files sparsely. 
> 
> Would efficient handling of sparse files be something worth implementing?

I would consider sparse file handling something useful. Even if it is a 
layering violation it might be a good idea to teach the chunking code to 
consider the start and end of holes as end of chunk condition. This could 
be leveraged by a fast path for the holes which are known in size and 
contain only zeroes.