librelist archives

« back to archive

How to find older versions of a file

How to find older versions of a file

From:
Zsigmond Attila
Date:
2015-05-03 @ 21:58
Hi,

I found Attic yesterday and I decided to use it on my desktop PC.
(still near good old backup2l which I use since 2008)

I know about "attic list" and "attic mount" commands but they seems to
be very unpractical when I need to find an older version of a file.
If I don't know - and usually don't - the date when the file was
changed I need to look it in many archives. I mounted the repository
(which had only 6 archives with 3.5 GB total deduplicated size) and
tried to find different versions of a file with double commander. It was
very slow, practically unacceptable. I don't think that it would
considerably quicker with an other file manager. It is simply
unpractical to find one file among another 100000. There should be a
better solution. Is there?

Attila

Re: [attic] How to find older versions of a file

From:
SanskritFritz
Date:
2015-05-04 @ 09:21
I'd write a script which restores the files to a custom location.
What is slow? Browsing the files?


On Sun, May 3, 2015 at 11:58 PM, Zsigmond Attila <david.spammy@gmail.com>
wrote:

> Hi,
>
> I found Attic yesterday and I decided to use it on my desktop PC.
> (still near good old backup2l which I use since 2008)
>
> I know about "attic list" and "attic mount" commands but they seems to
> be very unpractical when I need to find an older version of a file.
> If I don't know - and usually don't - the date when the file was
> changed I need to look it in many archives. I mounted the repository
> (which had only 6 archives with 3.5 GB total deduplicated size) and
> tried to find different versions of a file with double commander. It was
> very slow, practically unacceptable. I don't think that it would
> considerably quicker with an other file manager. It is simply
> unpractical to find one file among another 100000. There should be a
> better solution. Is there?
>
> Attila
>

Re: [attic] How to find older versions of a file

From:
Zsigmond Attila
Date:
2015-05-04 @ 11:15
On Mon, 4 May 2015 11:21:48 +0200
SanskritFritz <sanskritfritz@gmail.com> wrote:

> I'd write a script which restores the files to a custom location.
> What is slow? Browsing the files?


Browsing files is not slow. My problem is to guess which versions
of the file have I backed up and which archives contain them. Maybe
there should be an index for them or an efficient algorithm to find
them.

Attila

Re: [attic] How to find older versions of a file

From:
SanskritFritz
Date:
2015-05-04 @ 14:33
What you are asking for is way beyond the purposes of a backup program.
Dont forget, Attic knows little about files or their contents. It stores
data in chunks, which do not correspond to files they are extracted from at
all. You want to index files by their contents. That means you have to
extract them one-by-one from the archive. This was exactly the workaround I
proposed earlier.



On Mon, May 4, 2015 at 1:15 PM, Zsigmond Attila <david.spammy@gmail.com>
wrote:

> On Mon, 4 May 2015 11:21:48 +0200
> SanskritFritz <sanskritfritz@gmail.com> wrote:
>
> > I'd write a script which restores the files to a custom location.
> > What is slow? Browsing the files?
>
>
> Browsing files is not slow. My problem is to guess which versions
> of the file have I backed up and which archives contain them. Maybe
> there should be an index for them or an efficient algorithm to find
> them.
>
> Attila
>

Re: [attic] How to find older versions of a file

From:
Zsigmond Attila
Date:
2015-05-04 @ 15:10
No I don't want files to be indexed by their content. I would see
only their metadata, mostly their date and size from different
archives. These information surely need to be there because restore
cannot done without them. So the task would be to find all different
versions of a file in each archive catalogs.

Heiko: Agree and thank you.


On Mon, 4 May 2015 16:33:07 +0200
SanskritFritz <sanskritfritz@gmail.com> wrote:

> What you are asking for is way beyond the purposes of a backup program.
> Dont forget, Attic knows little about files or their contents. It stores
> data in chunks, which do not correspond to files they are extracted from at
> all. You want to index files by their contents. That means you have to
> extract them one-by-one from the archive. This was exactly the workaround I
> proposed earlier.

Sv: [attic] How to find older versions of a file

From:
Petter Gunnerud
Date:
2015-05-05 @ 13:17
I agree that this functionality should be in a backup solution.
(And for that reason I'm using Unison for backup of users datafiles.)

The easiest way to solve this I think would be a export file list for all 
repos that can be passed through grep for filtering and opened in a 
spreadsheet for sorting. The format could be:
"repoName","path","filename","size","mtime","ctime"
I'm guessing the required information is easily accessible in cache/repo.

The best (and hardest) way to solve this I think would be an "merged 
mount" of all repos (or a selection of repos within a time frame).

To explain what I called "merged mount":
Say repo 2015-05-01 contains the files(path/file , file size , modified timestamp)
/home/user/folder1/file1 1KB, 2015-05-01
/home/user/folder1/file2 1KB, 2015-05-01
/home/user/folder2/file3 1KB, 2015-05-01/home/user/folder2/file4 1KB, 2015-05-01
repo 2015-05-02 contains the files
/home/user/folder1/file1 1KB, 2015-05-01
/home/user/folder1/file2 2KB, 2015-05-02
/home/user/folder2/file4 1KB, 2015-05-01/home/user/folder2/file5 2KB, 
2015-05-02(file2 modified, file3 deleted, file5 created)
repo 2015-05-03 contains the files
/home/user/folder1/file1 1KB, 2015-05-01
/home/user/folder1/file2 3KB, 2015-05-03/home/user/folder2/file3 3KB, 
2015-05-03/home/user/folder2/file4 1KB, 2015-05-01/home/user/folder2/file5
1KB, 2015-05-02(file2 modified, file3 recreated)
With a "merged mount" each file could show as a folder named as the file, 
containing the versions of the file, and each version of the file has the 
repo-name appended to it. The paths/files could look like this:
/mnt/merged/home/user/folder1/file1/file1.2015-05-01 1KB, 
2015-05-01/mnt/merged/home/user/folder1/file1/file1.2015-05-02 1KB, 
2015-05-01/mnt/merged/home/user/folder1/file1/file1.2015-05-03 1KB, 
2015-05-01
/mnt/merged/home/user/folder1/file2/file2.2015-05-01 1KB, 
2015-05-01/mnt/merged/home/user/folder1/file2/file2.2015-05-02 2KB, 
2015-05-02/mnt/merged/home/user/folder1/file2/file2.2015-05-03 3KB, 
2015-05-03
/mnt/merged/home/user/folder2/file3/file3.2015-05-01 1KB, 
2015-05-01/mnt/merged/home/user/folder2/file3/file3.2015-05-03 3KB, 
2015-05-03
/mnt/merged/home/user/folder2/file4/file4.2015-05-01 1KB, 
2015-05-01/mnt/merged/home/user/folder2/file4/file4.2015-05-02 1KB, 
2015-05-01/mnt/merged/home/user/folder2/file4/file4.2015-05-03 1KB, 
2015-05-01
/mnt/merged/home/user/folder2/file5/file5.2015-05-02 1KB, 
2015-05-02/mnt/merged/home/user/folder2/file5/file5.2015-05-03 1KB, 
2015-05-02
When it comes to the timestamps of the folders named as the file, I think 
mtime could be the mtime of the file in the newest repo, and created time 
of the folder could be set to the mtime of the first repo containing the 
file.


     Fra: Zsigmond Attila <david.spammy@gmail.com>
 Til: attic@librelist.com 
 Sendt: Mandag, 4. mai 2015 17.10
 Emne: Re: [attic] How to find older versions of a file
   
No I don't want files to be indexed by their content. I would see
only their metadata, mostly their date and size from different
archives. These information surely need to be there because restore
cannot done without them. So the task would be to find all different
versions of a file in each archive catalogs.

Heiko: Agree and thank you.


On Mon, 4 May 2015 16:33:07 +0200
SanskritFritz <sanskritfritz@gmail.com> wrote:

> What you are asking for is way beyond the purposes of a backup program.
> Dont forget, Attic knows little about files or their contents. It stores
> data in chunks, which do not correspond to files they are extracted from at
> all. You want to index files by their contents. That means you have to
> extract them one-by-one from the archive. This was exactly the workaround I
> proposed earlier.


   

Re: [attic] How to find older versions of a file

From:
Date:
2015-05-04 @ 14:43
I disagree

having a backup catalog of the files backed up for quick lookup is a 
standard feature of many (larger) backup programs (e.g. amanda, bacula, 
veeam, even acronis has one). Even tar has rudimentary support for that.

Maybe some of those people more knowledgeable of attics internals could 
tell us if it would be theoretically possible to extract some information 
from the attic cache? I do know attic caches quite a lot of metadata 
(inodes, mtimes etc.) - perhaps it is possible to use that information? 

if not then you at least have the (quite time-consuming) way of mounting 
the whole repo and running a "find" over it. Depending on repo size and 
number of files, this of course could take a bit - but in the best case, 
you don't do restores too often so it's not that big of an issue :)

Best regards
 Heiko

attic@librelist.com wrote on 04.05.2015 16:33:07:

> From: SanskritFritz <sanskritfritz@gmail.com>
> To: attic@librelist.com
> Date: 04.05.2015 16:34
> Subject: Re: [attic] How to find older versions of a file
> Sent by: attic@librelist.com
> 
> What you are asking for is way beyond the purposes of a backup 
> program. Dont forget, Attic knows little about files or their 
> contents. It stores data in chunks, which do not correspond to files
> they are extracted from at all. You want to index files by their 
> contents. That means you have to extract them one-by-one from the 
> archive. This was exactly the workaround I proposed earlier. 
>