librelist archives

« back to archive

attic check routine messages to stdout?

attic check routine messages to stdout?

From:
Dan Christensen
Date:
2014-03-01 @ 18:40
My backup scripts e-mail me only if there is output to stderr.  The
routine progress messages produced by "attic check" are currently
sent to stderr.  Would it be reasonable to send the routine messages
to stdout, and only important messages to stderr?

Dan

Re: [attic] attic check routine messages to stdout?

From:
Jonas Borgström
Date:
2014-03-04 @ 11:32
On 2014-03-01 19:40 , Dan Christensen wrote:
> My backup scripts e-mail me only if there is output to stderr.  The
> routine progress messages produced by "attic check" are currently
> sent to stderr.  Would it be reasonable to send the routine messages
> to stdout, and only important messages to stderr?

Yeah, that makes sense. Unfortunately the repository phase can only use
stderr since stdout is used for the client->server RPC.

Earlier iterations of "attic check" had some elaborate logic where these
non-critical messages chould be enabled/disabled with --progress and
--no-progress. And the default depended on if stdout was connected to a
tty or not (modeled after git fsck).
Anyway, I removed that a few commits ago since I thought it was overly
complex and hard to explain.

How about this: We keep the current default behavior but add a --quiet
option that can be used to suppress the progress messages?

While we're on the topic, do you have any better idea what we should
call "attic check --phase repostiory|archive". I'm not quite happy with
the current naming...

/ Jonas

Re: [attic] attic check routine messages to stdout?

From:
Dan Christensen
Date:
2014-03-04 @ 14:46
Jonas Borgström <jonas@borgstrom.se> writes:

> On 2014-03-01 19:40 , Dan Christensen wrote:
>> My backup scripts e-mail me only if there is output to stderr.  The
>> routine progress messages produced by "attic check" are currently
>> sent to stderr.  Would it be reasonable to send the routine messages
>> to stdout, and only important messages to stderr?
>
> Yeah, that makes sense. Unfortunately the repository phase can only use
> stderr since stdout is used for the client->server RPC.

Oh, I see.  Maybe the progress messages could be sent as part of the
client->server RPC?

> Earlier iterations of "attic check" had some elaborate logic where these
> non-critical messages chould be enabled/disabled with --progress and
> --no-progress. And the default depended on if stdout was connected to a
> tty or not (modeled after git fsck).
> Anyway, I removed that a few commits ago since I thought it was overly
> complex and hard to explain.

I agree.

> How about this: We keep the current default behavior but add a --quiet
> option that can be used to suppress the progress messages?

That would help.  But ideally, I would save all output in a file, and
send stderr by e-mail.  Then if I get an e-mail message, I can look in
the file and see at what point it found a problem.

> While we're on the topic, do you have any better idea what we should
> call "attic check --phase repostiory|archive". I'm not quite happy with
> the current naming...

I'm not sure.  But another thought that might affect this is that I'd
like to be able to just check one archive, since as a repository grows,
checking all of the historical archives is going to get very slow.

Maybe

  attic check repo.attic                             [repo and all archives]
  attic check --repository-only repo.attic           [repo]
  attic check --archives-only   repo.attic           [all archives]
  attic check repo.attic::archive1                   [repo and one archive]
  attic check --repository-only repo.attic::archive1 [repo]
  attic check --archives-only   repo.attic::archive1 [one archive]

with --archive-only a synonym for --archives-only and --repo-only a
synonym for --repository-only?

(Or the word order could be reversed, e.g. --only-repository)

Dan

Re: [attic] attic check routine messages to stdout?

From:
Jonas Borgström
Date:
2014-03-04 @ 21:05
On 2014-03-04 15:46, Dan Christensen wrote:
> Jonas Borgström <jonas@borgstrom.se> writes:
> 
>> On 2014-03-01 19:40 , Dan Christensen wrote:
>>> My backup scripts e-mail me only if there is output to stderr.  The
>>> routine progress messages produced by "attic check" are currently
>>> sent to stderr.  Would it be reasonable to send the routine messages
>>> to stdout, and only important messages to stderr?
>>
>> Yeah, that makes sense. Unfortunately the repository phase can only use
>> stderr since stdout is used for the client->server RPC.
> 
> Oh, I see.  Maybe the progress messages could be sent as part of the
> client->server RPC?

Actually, after taking a second look at this I realized that all
non-critical messages during the repository check were either before or
after the actual check so I was able to move them to archiver.py. So
there's now no output to stderr unless an error is found.

>> Earlier iterations of "attic check" had some elaborate logic where these
>> non-critical messages chould be enabled/disabled with --progress and
>> --no-progress. And the default depended on if stdout was connected to a
>> tty or not (modeled after git fsck).
>> Anyway, I removed that a few commits ago since I thought it was overly
>> complex and hard to explain.
> 
> I agree.
> 
>> How about this: We keep the current default behavior but add a --quiet
>> option that can be used to suppress the progress messages?
> 
> That would help.  But ideally, I would save all output in a file, and
> send stderr by e-mail.  Then if I get an e-mail message, I can look in
> the file and see at what point it found a problem.
> 
>> While we're on the topic, do you have any better idea what we should
>> call "attic check --phase repostiory|archive". I'm not quite happy with
>> the current naming...
> 
> I'm not sure.  But another thought that might affect this is that I'd
> like to be able to just check one archive, since as a repository grows,
> checking all of the historical archives is going to get very slow.

Yeah, we'll probably support checking specific archives in the future
but that will not make it into 0.11. These checks will also be a bit
less effective since you really need to look at all metadata to be able
to detect some problems.

> Maybe
> 
>   attic check repo.attic                             [repo and all archives]
>   attic check --repository-only repo.attic           [repo]
>   attic check --archives-only   repo.attic           [all archives]
>   attic check repo.attic::archive1                   [repo and one archive]
>   attic check --repository-only repo.attic::archive1 [repo]
>   attic check --archives-only   repo.attic::archive1 [one archive]

I've just pushed a change that does this, except for the specific
archive part.

/ Jonas

Re: [attic] attic check routine messages to stdout?

From:
Dan Christensen
Date:
2014-03-04 @ 21:33
Great!

I have to say that attic is really an amazing program.  My backups now
all get done automatically during the night, to repositories in
different locations, and it is extremely fast, network efficient, and
space efficient.  It's so efficient that I now do backups every night,
instead of just occasionally.  The de-duplication is especially useful,
since I backup different machines into the same repositories, so
anything those machines have in common is shared.

Thanks!

Dan