librelist archives

« back to archive

why do I sometimes get "Initializing cache" messages?

why do I sometimes get "Initializing cache" messages?

From:
Dan Christensen
Date:
2014-03-11 @ 00:24
Sometimes, when creating a backup in a repo that I use regularly, I get
the "Initializing cache..." message followed by "Analyzing archive:
..." for all of the archives.  But most of the time this doesn't happen.

Any idea what might cause this?

Dan

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Dan Christensen
Date:
2014-03-11 @ 00:36
Dan Christensen <jdc@uwo.ca> writes:

> Sometimes, when creating a backup in a repo that I use regularly, I get
> the "Initializing cache..." message followed by "Analyzing archive:
> ..." for all of the archives.  But most of the time this doesn't happen.
>
> Any idea what might cause this?

Could the North American switch to daylight savings time have something
to do with it?  It seems like the caches need to be rebuilt on all
of my repos, and for the ones I access over sshfs, this is quite slow.

Dan

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Jonas Borgström
Date:
2014-03-11 @ 12:01
On 2014-03-11 01:36 , Dan Christensen wrote:
> Dan Christensen <jdc@uwo.ca> writes:
> 
>> Sometimes, when creating a backup in a repo that I use regularly, I get
>> the "Initializing cache..." message followed by "Analyzing archive:
>> ..." for all of the archives.  But most of the time this doesn't happen.
>>
>> Any idea what might cause this?

This usually happens when multiple clients (and thereby different cache
directories) are used to modify a single shared repository. When Attic
detects that the cache directory is not up to date it will automatically
rebuild it.

The Attic cache stores among other things a hash table containing size
and reference count of all file chunks stored in the repository.

On archive creation this information is used to determine if a file
chunk is new and needs to be stored in the repository or if we just need
to increment the reference count.

On archive deletion the reference count is used to determine if a file
chunk can be deleted or if it's used by other archives.

A cache is currently rebuilt by scanning through all archive metadata
which is fairly slow. Ideally this should be done incrementally, but
that's a rather difficult, especially in the case where archives have
been deleted, since we need access to the deleted metadata.

> Could the North American switch to daylight savings time have something
> to do with it?  It seems like the caches need to be rebuilt on all
> of my repos, and for the ones I access over sshfs, this is quite slow.

I don't think so, is that the only change in your setup? is the
repository access pattern unchanged?

/ Jonas

PS
This should probably be added to our non-existent FAQ :)
DS

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Dan Christensen
Date:
2014-03-11 @ 12:41
Jonas Borgström <jonas@borgstrom.se> writes:

> On 2014-03-11 01:36 , Dan Christensen wrote:
>> Dan Christensen <jdc@uwo.ca> writes:
>> 
>>> Sometimes, when creating a backup in a repo that I use regularly, I get
>>> the "Initializing cache..." message followed by "Analyzing archive:
>>> ..." for all of the archives.  But most of the time this doesn't happen.
>>>
>>> Any idea what might cause this?
>
> This usually happens when multiple clients (and thereby different cache
> directories) are used to modify a single shared repository. 

Ok, that's all it is.  I backup several machines into the same repos
so they can benefit from de-duplication, and I guess I didn't notice
the cache rebuilding until the repos started to grow.  I'm using sshfs
over slow wifi, and it's very painful.  I'm guessing that if I ran 
attic on the remote host it would be a lot better, right?  It's on
my to do list to upgrade that machine.

> A cache is currently rebuilt by scanning through all archive metadata
> which is fairly slow. Ideally this should be done incrementally, but
> that's a rather difficult, especially in the case where archives have
> been deleted, since we need access to the deleted metadata.

Could the cache also be stored in the repo, so that when a client has 
an outdated cache it could just download it from the repo rather than
recreating it?  Having multiple clients backup into the same repo seems
like a standard use case that should be efficiently supported.

Dan

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Jonas Borgström
Date:
2014-03-12 @ 11:48
On 2014-03-11 13:41 , Dan Christensen wrote:
> Jonas Borgström <jonas@borgstrom.se> writes:
> 
>> On 2014-03-11 01:36 , Dan Christensen wrote:
>>> Dan Christensen <jdc@uwo.ca> writes:
>>>
>>>> Sometimes, when creating a backup in a repo that I use regularly, I get
>>>> the "Initializing cache..." message followed by "Analyzing archive:
>>>> ..." for all of the archives.  But most of the time this doesn't happen.
>>>>
>>>> Any idea what might cause this?
>>
>> This usually happens when multiple clients (and thereby different cache
>> directories) are used to modify a single shared repository. 
> 
> Ok, that's all it is.  I backup several machines into the same repos
> so they can benefit from de-duplication, and I guess I didn't notice
> the cache rebuilding until the repos started to grow.  I'm using sshfs
> over slow wifi, and it's very painful.  I'm guessing that if I ran 
> attic on the remote host it would be a lot better, right?  It's on
> my to do list to upgrade that machine.

Using a proper remote repository will make it a bit faster but
rebuilding the cache from scratch will always be slow since all archive
metadata will have to be downloaded and processed.

So until this is fixed you probably want to switch to a setup where only
a single client/cache is regularly modifying a specific repository.

>> A cache is currently rebuilt by scanning through all archive metadata
>> which is fairly slow. Ideally this should be done incrementally, but
>> that's a rather difficult, especially in the case where archives have
>> been deleted, since we need access to the deleted metadata.
> 
> Could the cache also be stored in the repo, so that when a client has 
> an outdated cache it could just download it from the repo rather than
> recreating it?  Having multiple clients backup into the same repo seems
> like a standard use case that should be efficiently supported.

I have some ideas for how to avoid full cache rebuilds. One approach is
to leave the archive metadata of deleted archives in the repository as
some kind of archive tombstones. This would make it possible for
outdated caches to replay those changes. These tombstones could later be
automatically purged after X days to recover disk space.

But it's tricky to implement this correctly and at the same time not
break compatibility too much.

I'll have to give it some more thought, but this might become the main
feature for 0.12 or 0.13. Especially since everyone seems to be running
into this limitation after discovering how nice it is with proper
deduplication and decides to store everything into one big repository :)

/ Jonas

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Dan Christensen
Date:
2014-03-12 @ 13:03
Jonas Borgström <jonas@borgstrom.se> writes:

>> Could the cache also be stored in the repo, so that when a client has 
>> an outdated cache it could just download it from the repo rather than
>> recreating it?  Having multiple clients backup into the same repo seems
>> like a standard use case that should be efficiently supported.
>
> I have some ideas for how to avoid full cache rebuilds. One approach is
> to leave the archive metadata of deleted archives in the repository as
> some kind of archive tombstones. This would make it possible for
> outdated caches to replay those changes. These tombstones could later be
> automatically purged after X days to recover disk space.
>
> But it's tricky to implement this correctly and at the same time not
> break compatibility too much.

What you describe might be most efficient, but wouldn't it be easier to
just store a copy of the cache in the repo, and if the local cache is
out of date, just download the remote cache?  It's not very large
compared to the size of the repo.  And one could even download it
efficiently, e.g. using an rsync-like protocol, although that wouldn't
even be necessary.

Dan

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Jonas Borgström
Date:
2014-03-13 @ 21:48
On 2014-03-12 14:03, Dan Christensen wrote:
> Jonas Borgström <jonas@borgstrom.se> writes:
> 
>>> Could the cache also be stored in the repo, so that when a client has 
>>> an outdated cache it could just download it from the repo rather than
>>> recreating it?  Having multiple clients backup into the same repo seems
>>> like a standard use case that should be efficiently supported.
>>
>> I have some ideas for how to avoid full cache rebuilds. One approach is
>> to leave the archive metadata of deleted archives in the repository as
>> some kind of archive tombstones. This would make it possible for
>> outdated caches to replay those changes. These tombstones could later be
>> automatically purged after X days to recover disk space.
>>
>> But it's tricky to implement this correctly and at the same time not
>> break compatibility too much.
> 
> What you describe might be most efficient, but wouldn't it be easier to
> just store a copy of the cache in the repo, and if the local cache is
> out of date, just download the remote cache?  It's not very large
> compared to the size of the repo.  And one could even download it
> efficiently, e.g. using an rsync-like protocol, although that wouldn't
> even be necessary.

Yeah, that could work but doesn't sound very easy either...

Anyway, I've just pushed a minor change that caches archive metadata to
a temporary file during the cache rebuild. This should give a noticeable
speed improvement especially for archives created with Attic >= 0.11.

Even with this change a cache rebuild takes too long in some cases. But
I'm working on an approach that will allow the rebuild to be incremental
(only processing metadata for new and deleted archives, not all archives).
This will however require a change in the repository manifest format.
This means that repositories/archives created by Attic 0.12 will not be
compatible with older versions of Attic (The other way around will work).
Would it be acceptable to break compatibility in this way?

/ Jonas

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Date:
2014-03-14 @ 06:50
> This means that repositories/archives created by Attic 0.12 will not be
> compatible with older versions of Attic (The other way around will 
work).
> Would it be acceptable to break compatibility in this way?

probably yes, but the possibility to upgrade/rebuild archives <0.11 would 
be really nice :)

Best Regards
 Heiko

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Petros Moisiadis
Date:
2014-03-14 @ 07:45
On 03/13/2014 11:48 PM, Jonas Borgström wrote:
> Even with this change a cache rebuild takes too long in some cases.
> But I'm working on an approach that will allow the rebuild to be
> incremental (only processing metadata for new and deleted archives,
> not all archives). This will however require a change in the
> repository manifest format. This means that repositories/archives
> created by Attic 0.12 will not be compatible with older versions of
> Attic (The other way around will work). Would it be acceptable to
> break compatibility in this way? / Jonas 

I think it would be acceptable.
But, it would be also awesome, if possible for such changes, to have an
'--upgrade-format' option for in-place upgrading the format of older
repositories and their archives to the latest format (with a
confirmation step and a warning to keep backup copies first).

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Petros Moisiadis
Date:
2014-03-14 @ 08:13
On 03/14/2014 09:45 AM, Petros Moisiadis wrote:
> On 03/13/2014 11:48 PM, Jonas Borgström wrote:
>> Even with this change a cache rebuild takes too long in some cases.
>> But I'm working on an approach that will allow the rebuild to be
>> incremental (only processing metadata for new and deleted archives,
>> not all archives). This will however require a change in the
>> repository manifest format. This means that repositories/archives
>> created by Attic 0.12 will not be compatible with older versions of
>> Attic (The other way around will work). Would it be acceptable to
>> break compatibility in this way? / Jonas 
> I think it would be acceptable.
> But, it would be also awesome, if possible for such changes, to have an
> '--upgrade-format' option for in-place upgrading the format of older
> repositories and their archives to the latest format (with a
> confirmation step and a warning to keep backup copies first).

Well, maybe an 'upgrade' action (like the 'init') would be better for this.

Also, Jonas, in your last post you suggested a silent upgrade, so I am
thinking if this could be risky in some cases. Maybe this time the
change is not that risky (only the manifest format changes), but if
there are other, more risky changes in the future, it seems to me
reasonable to have people control the upgrade process.

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Jonas Borgström
Date:
2014-03-14 @ 11:44
On 2014-03-14 09:13 , Petros Moisiadis wrote:
> On 03/14/2014 09:45 AM, Petros Moisiadis wrote:
>> On 03/13/2014 11:48 PM, Jonas Borgström wrote:
>>> Even with this change a cache rebuild takes too long in some cases.
>>> But I'm working on an approach that will allow the rebuild to be
>>> incremental (only processing metadata for new and deleted archives,
>>> not all archives). This will however require a change in the
>>> repository manifest format. This means that repositories/archives
>>> created by Attic 0.12 will not be compatible with older versions of
>>> Attic (The other way around will work). Would it be acceptable to
>>> break compatibility in this way? / Jonas 
>> I think it would be acceptable.
>> But, it would be also awesome, if possible for such changes, to have an
>> '--upgrade-format' option for in-place upgrading the format of older
>> repositories and their archives to the latest format (with a
>> confirmation step and a warning to keep backup copies first).
> 
> Well, maybe an 'upgrade' action (like the 'init') would be better for this.
> 
> Also, Jonas, in your last post you suggested a silent upgrade, so I am
> thinking if this could be risky in some cases. Maybe this time the
> change is not that risky (only the manifest format changes), but if
> there are other, more risky changes in the future, it seems to me
> reasonable to have people control the upgrade process.

From a user's point of view it would be best if upgrades always were
optional and only required to enable new features (But at the same time
making the repository incompatible with older versions).

But that of course also translates into more complexity when the source
code needs to support reading and writing multiple different versions.

Anyway, I think I'll start with actually getting this feature to work
before deciding on how to handle the format change.

/ Jonas

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Petros Moisiadis
Date:
2014-03-14 @ 12:13
On 03/14/14 13:44, Jonas Borgström wrote:
> On 2014-03-14 09:13 , Petros Moisiadis wrote:
>> On 03/14/2014 09:45 AM, Petros Moisiadis wrote:
>>> On 03/13/2014 11:48 PM, Jonas Borgström wrote:
>>>> Even with this change a cache rebuild takes too long in some cases.
>>>> But I'm working on an approach that will allow the rebuild to be
>>>> incremental (only processing metadata for new and deleted archives,
>>>> not all archives). This will however require a change in the
>>>> repository manifest format. This means that repositories/archives
>>>> created by Attic 0.12 will not be compatible with older versions of
>>>> Attic (The other way around will work). Would it be acceptable to
>>>> break compatibility in this way? / Jonas 
>>> I think it would be acceptable.
>>> But, it would be also awesome, if possible for such changes, to have an
>>> '--upgrade-format' option for in-place upgrading the format of older
>>> repositories and their archives to the latest format (with a
>>> confirmation step and a warning to keep backup copies first).
>> Well, maybe an 'upgrade' action (like the 'init') would be better for this.
>>
>> Also, Jonas, in your last post you suggested a silent upgrade, so I am
>> thinking if this could be risky in some cases. Maybe this time the
>> change is not that risky (only the manifest format changes), but if
>> there are other, more risky changes in the future, it seems to me
>> reasonable to have people control the upgrade process.
> >From a user's point of view it would be best if upgrades always were
> optional and only required to enable new features (But at the same time
> making the repository incompatible with older versions).
>
> But that of course also translates into more complexity when the source
> code needs to support reading and writing multiple different versions.
Perhaps it is not that bad to deprecate and finally drop
backwards-compatibility from time to time, in order to keep the
complexity of source code in 'sane' levels. For example,
backwards-compatibility could exist only between two successive format
changes. Besides, people insisting on keeping their old stuff (and lose
newer functionality), would still have the choice of not upgrading their
attic installations.
> Anyway, I think I'll start with actually getting this feature to work
> before deciding on how to handle the format change.
>
> / Jonas
>
>

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Jonas Borgström
Date:
2014-03-19 @ 20:03
On 2014-03-14 13:13, Petros Moisiadis wrote:
> On 03/14/14 13:44, Jonas Borgström wrote:
>> On 2014-03-14 09:13 , Petros Moisiadis wrote:
>>> On 03/14/2014 09:45 AM, Petros Moisiadis wrote:
>>>> On 03/13/2014 11:48 PM, Jonas Borgström wrote:
>>>>> Even with this change a cache rebuild takes too long in some cases.
>>>>> But I'm working on an approach that will allow the rebuild to be
>>>>> incremental (only processing metadata for new and deleted archives,
>>>>> not all archives). This will however require a change in the
>>>>> repository manifest format. This means that repositories/archives
>>>>> created by Attic 0.12 will not be compatible with older versions of
>>>>> Attic (The other way around will work). Would it be acceptable to
>>>>> break compatibility in this way? / Jonas 
>>>> I think it would be acceptable.
>>>> But, it would be also awesome, if possible for such changes, to have an
>>>> '--upgrade-format' option for in-place upgrading the format of older
>>>> repositories and their archives to the latest format (with a
>>>> confirmation step and a warning to keep backup copies first).
>>> Well, maybe an 'upgrade' action (like the 'init') would be better for this.
>>>
>>> Also, Jonas, in your last post you suggested a silent upgrade, so I am
>>> thinking if this could be risky in some cases. Maybe this time the
>>> change is not that risky (only the manifest format changes), but if
>>> there are other, more risky changes in the future, it seems to me
>>> reasonable to have people control the upgrade process.
>> >From a user's point of view it would be best if upgrades always were
>> optional and only required to enable new features (But at the same time
>> making the repository incompatible with older versions).
>>
>> But that of course also translates into more complexity when the source
>> code needs to support reading and writing multiple different versions.
> Perhaps it is not that bad to deprecate and finally drop
> backwards-compatibility from time to time, in order to keep the
> complexity of source code in 'sane' levels. For example,
> backwards-compatibility could exist only between two successive format
> changes. Besides, people insisting on keeping their old stuff (and lose
> newer functionality), would still have the choice of not upgrading their
> attic installations.
>> Anyway, I think I'll start with actually getting this feature to work
>> before deciding on how to handle the format change.

A short status update on this topic. I've done some experiments with
different approaches to get cache rebuild to be "fast enough" to be a
non-issue. Unfortunately I've not yet found a way that I think is good
enough.

So far I've tested these approaches:

Leave enough data in the repository during archive deletion to make
incremental cache updates possible. This approach sounds very promising
on paper. Unfortunately this both a repository format change (minor) and
lots of code changes and added complexity. The added complexity might
also make it more difficult to implement other features in the future.

Another approach is to cache the full decrypted un-deduplicated metadata
for each archive in the cache directory. This is easy to implement and
allows a full cache rebuild in seconds. The downside with this approach
is that a lot of diskspace is wasted (around 2% of the total repository
size). This might be ok in some cases but probably not something that we
can enable by default.

So it looks like this feature will have to wait a while...

/ Jonas

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Dan Christensen
Date:
2014-04-10 @ 17:14
Jonas Borgström <jonas@borgstrom.se> writes:

> A short status update on this topic. I've done some experiments with
> different approaches to get cache rebuild to be "fast enough" to be a
> non-issue. Unfortunately I've not yet found a way that I think is good
> enough.

When the cache is rebuilt, it prints messages like

Analyzing archive: first-backup
Analyzing archive: second-backup

etc.

Would it be possible for attic to get a list of the archives that exist
in the remote repo and only rebuild the cache for the newly added
archives?  (And delete the cached info for archives that have been
deleted?)

Alternatively, I'm still not sure why my original suggestion of storing
the cache directory with the repo and transferring it over as needed
(maybe using the rsync algorithm) isn't a good solution.  It seems like
it would be very fast, even in the case where the local cache is deleted
and needs to be rebuilt from scratch, or where a new client starts
accessing an existing repo.

Dan

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Jonas Borgström
Date:
2014-04-10 @ 19:33
On 2014-04-10 19:14, Dan Christensen wrote:
> Jonas Borgström <jonas@borgstrom.se> writes:
> 
>> A short status update on this topic. I've done some experiments with
>> different approaches to get cache rebuild to be "fast enough" to be a
>> non-issue. Unfortunately I've not yet found a way that I think is good
>> enough.
> 
> When the cache is rebuilt, it prints messages like
> 
> Analyzing archive: first-backup
> Analyzing archive: second-backup
> 
> etc.
> 
> Would it be possible for attic to get a list of the archives that exist
> in the remote repo and only rebuild the cache for the newly added
> archives?  (And delete the cached info for archives that have been
> deleted?)

Unfortunately the archive metadata is needed to "unload" an archive from
the cache. And since the archive has been deleted the metadata is also gone.

> Alternatively, I'm still not sure why my original suggestion of storing
> the cache directory with the repo and transferring it over as needed
> (maybe using the rsync algorithm) isn't a good solution.  It seems like
> it would be very fast, even in the case where the local cache is deleted
> and needs to be rebuilt from scratch, or where a new client starts
> accessing an existing repo.

That would work but I think the overhead would be quite high. As you
probably have noticed the cache is fairly large, and I suspect that even
using rsync we would probably end up having to transfer almost the
entire cache every time the repository is modified. The "cache sync"
would probably consume more bandwidth than the actual backup itself.

/ Jonas

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Dan Christensen
Date:
2014-03-13 @ 23:27
Jonas Borgström <jonas@borgstrom.se> writes:

> On 2014-03-12 14:03, Dan Christensen wrote:
> 
>> What you describe might be most efficient, but wouldn't it be easier to
>> just store a copy of the cache in the repo, and if the local cache is
>> out of date, just download the remote cache?  It's not very large
>> compared to the size of the repo.
>
> Yeah, that could work but doesn't sound very easy either...

I don't know the protocol with the server, but when the client updates
the cache, I was thinking that it would just store the same data on the
server.  And when a client sees that its cache is out of date, it would
just download the cache from the server.

> Anyway, I've just pushed a minor change that caches archive metadata to
> a temporary file during the cache rebuild. This should give a noticeable
> speed improvement especially for archives created with Attic >= 0.11.

Great!  I just tried it, and it seems to be about twice as fast on
a repo that mainly contains archives from before 0.11.  What changed
in 0.11 to make a difference here?

> Even with this change a cache rebuild takes too long in some cases. But
> I'm working on an approach that will allow the rebuild to be incremental
> (only processing metadata for new and deleted archives, not all archives).

That sounds promising, as long as it doesn't add extra fragility.

> This will however require a change in the repository manifest format.
> This means that repositories/archives created by Attic 0.12 will not be
> compatible with older versions of Attic (The other way around will work).
> Would it be acceptable to break compatibility in this way?

There's no easy way to add the extra data to older repos if it doesn't
exist?

Dan

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Jonas Borgström
Date:
2014-03-14 @ 07:44
On 2014-03-14 24:27 , Dan Christensen wrote:
> Jonas Borgström <jonas@borgstrom.se> writes:
> 
>> On 2014-03-12 14:03, Dan Christensen wrote:
>>
>>> What you describe might be most efficient, but wouldn't it be easier to
>>> just store a copy of the cache in the repo, and if the local cache is
>>> out of date, just download the remote cache?  It's not very large
>>> compared to the size of the repo.
>>
>> Yeah, that could work but doesn't sound very easy either...
> 
> I don't know the protocol with the server, but when the client updates
> the cache, I was thinking that it would just store the same data on the
> server.  And when a client sees that its cache is out of date, it would
> just download the cache from the server.
> 
>> Anyway, I've just pushed a minor change that caches archive metadata to
>> a temporary file during the cache rebuild. This should give a noticeable
>> speed improvement especially for archives created with Attic >= 0.11.
> 
> Great!  I just tried it, and it seems to be about twice as fast on
> a repo that mainly contains archives from before 0.11.  What changed
> in 0.11 to make a difference here?

https://github.com/jborg/attic/commit/c394a31d6238ceea84a4605daac2dc2469e284b7

This change made the serialization of dictionaries in archive metadata
stable (before this the key order was random). So it's now much more
likely that individual metadata chunks will be shared between multiple
archives.

(attic check --repair will update this for existing archives.)

>> Even with this change a cache rebuild takes too long in some cases. But
>> I'm working on an approach that will allow the rebuild to be incremental
>> (only processing metadata for new and deleted archives, not all archives).
> 
> That sounds promising, as long as it doesn't add extra fragility.
> 
>> This will however require a change in the repository manifest format.
>> This means that repositories/archives created by Attic 0.12 will not be
>> compatible with older versions of Attic (The other way around will work).
>> Would it be acceptable to break compatibility in this way?
> 
> There's no easy way to add the extra data to older repos if it doesn't
> exist?

I probably wasn't clear enough about this:

- Attic <= 0.11 Only supports manifest version 1.
- Attic >= 0.12 Supports manifest version 1 and 2.

- Attic 0.12 will create new repositories with manifest version 2.
- Attic 0.12 will perform a quick silent upgrade of old repositories the
first time an archive is created or deleted.

So most upgrading users should not notice anything. The only scenario
where this will be noticed is for users who modify a single repository
from multiple machines and then only upgrade Attic on one machine.

Does this make more sense?

/ Jonas

Re: [attic] why do I sometimes get "Initializing cache" messages?

From:
Dan Christensen
Date:
2014-03-14 @ 13:09
Jonas Borgström <jonas@borgstrom.se> writes:

>> Great!  I just tried it, and it seems to be about twice as fast on
>> a repo that mainly contains archives from before 0.11.  What changed
>> in 0.11 to make a difference here?
>
> https://github.com/jborg/attic/commit/c394a31d6238ceea84a4605daac2dc2469e284b7
>
> This change made the serialization of dictionaries in archive metadata
> stable (before this the key order was random). So it's now much more
> likely that individual metadata chunks will be shared between multiple
> archives.
>
> (attic check --repair will update this for existing archives.)

Is it enough to add --repository-only to this?

> - Attic <= 0.11 Only supports manifest version 1.
> - Attic >= 0.12 Supports manifest version 1 and 2.
>
> - Attic 0.12 will create new repositories with manifest version 2.
> - Attic 0.12 will perform a quick silent upgrade of old repositories the
> first time an archive is created or deleted.
>
> So most upgrading users should not notice anything. The only scenario
> where this will be noticed is for users who modify a single repository
> from multiple machines and then only upgrade Attic on one machine.
>
> Does this make more sense?

Yes.  That seems reasonable to me, especially with 0.x versions.
Keeping one version of backwards compatibility, as Petros mentioned,
would also be nice, e.g. for those using versions packaged in a
distro.  But since pip is easy to use, it doesn't seem crucial.

Dan