librelist archives

« back to archive

file locking in libgit2 ?

file locking in libgit2 ?

From:
Han-Wen Nienhuys
Date:
2012-04-03 @ 18:05
Hi there,

I have a bunch of users that roughly do the following through a program I wrote:

 * manipulate a git repository using both libgit and git command line.
 * remove the repository on disk using shutil.rmtree (in python)

I'm getting a bunch of reports that indicate the rmtree failing with:

OSError: [Errno 16] Device or resource busy:
'/path/to/repository/some/directory/.nfs0000000002128ef300001dd4'

this is for users on NFS.  Does libgit2 employ any sort of implicit or
explicit file locking that could result in this message?

thanks,

-- 
Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

Re: [libgit2] file locking in libgit2 ?

From:
Matthieu Moy
Date:
2012-04-04 @ 07:41
Han-Wen Nienhuys <hanwenn@gmail.com> writes:

> Hi there,
>
> I have a bunch of users that roughly do the following through a program I wrote:
>
>  * manipulate a git repository using both libgit and git command line.
>  * remove the repository on disk using shutil.rmtree (in python)
>
> I'm getting a bunch of reports that indicate the rmtree failing with:
>
> OSError: [Errno 16] Device or resource busy:
> '/path/to/repository/some/directory/.nfs0000000002128ef300001dd4'

These .nfs* files are created when you delete a file with a file
descriptor open. When you do that on a local file, the actual file isn't
deleted becase the file descriptor counts for 1 in the reference count,
and this .nfs* is a way to simulate this 1 reference count for the NFS
server. There's no need to use locking for this to happen:

$ sleep 100 > test/foo.txt &
[1] 18830
$ rm -fr test
rm: cannot remove `test': Directory not empty
$ rm -fr test
rm: cannot remove `test/.nfs00000000014687230000022e': Device or resource busy
$ kill %1
[1]  + terminated  sleep 100 > test/foo.txt

$ rm -fr test
$

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [libgit2] file locking in libgit2 ?

From:
Vicent Marti
Date:
2012-04-03 @ 18:21
This is weird. We do create lockfiles to ensure atomic writing most
objects (the index, ODB data, etc), but these are not NFS-aware. From
what I've researched, `flock` is a no-op for NFS in most kernels, so
it shouldn't prevent the directory removal.

Can you post more details, including the OS and kernel used, the
version of NFS, the contents of the `.nfs` file, and its exact path?

Cheerios,
Vicent

On Tue, Apr 3, 2012 at 8:05 PM, Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
> Hi there,
>
> I have a bunch of users that roughly do the following through a program I wrote:
>
>  * manipulate a git repository using both libgit and git command line.
>  * remove the repository on disk using shutil.rmtree (in python)
>
> I'm getting a bunch of reports that indicate the rmtree failing with:
>
> OSError: [Errno 16] Device or resource busy:
> '/path/to/repository/some/directory/.nfs0000000002128ef300001dd4'
>
> this is for users on NFS.  Does libgit2 employ any sort of implicit or
> explicit file locking that could result in this message?
>
> thanks,
>
> --
> Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

Re: [libgit2] file locking in libgit2 ?

From:
Carlos Martín Nieto
Date:
2012-04-03 @ 18:40
On Tue, 2012-04-03 at 20:21 +0200, Vicent Marti wrote:
> This is weird. We do create lockfiles to ensure atomic writing most
> objects (the index, ODB data, etc), but these are not NFS-aware. From
> what I've researched, `flock` is a no-op for NFS in most kernels, so
> it shouldn't prevent the directory removal.

Do we even use flock anywhere? A grep of libgit2 and git only gives me
some contrib code in git that uses it.

> 
> Can you post more details, including the OS and kernel used, the
> version of NFS, the contents of the `.nfs` file, and its exact path?
> 
> Cheerios,
> Vicent
> 
> On Tue, Apr 3, 2012 at 8:05 PM, Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
> > Hi there,
> >
> > I have a bunch of users that roughly do the following through a 
program I wrote:
> >
> >  * manipulate a git repository using both libgit and git command line.
> >  * remove the repository on disk using shutil.rmtree (in python)
> >
> > I'm getting a bunch of reports that indicate the rmtree failing with:
> >
> > OSError: [Errno 16] Device or resource busy:
> > '/path/to/repository/some/directory/.nfs0000000002128ef300001dd4'
> >
> > this is for users on NFS.  Does libgit2 employ any sort of implicit or
> > explicit file locking that could result in this message?
> >
> > thanks,
> >
> > --
> > Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen


Re: [libgit2] file locking in libgit2 ?

From:
Vicent Marti
Date:
2012-04-03 @ 19:03
On Tue, Apr 3, 2012 at 8:40 PM, Carlos Martín Nieto <carlos@cmartin.tk> wrote:
> On Tue, 2012-04-03 at 20:21 +0200, Vicent Marti wrote:
>> This is weird. We do create lockfiles to ensure atomic writing most
>> objects (the index, ODB data, etc), but these are not NFS-aware. From
>> what I've researched, `flock` is a no-op for NFS in most kernels, so
>> it shouldn't prevent the directory removal.
>
> Do we even use flock anywhere? A grep of libgit2 and git only gives me
> some contrib code in git that uses it.

Hm. You're right. We're creating `.lock` files, but not actually
locking them with a syscall. We dropped the `flock` calls because of
compat issues, I figure.

Han-Wen: filebuf.c:17 is a good starting point to debug locking, if
your NDA doesn't allow you to share more information.

Re: [libgit2] file locking in libgit2 ?

From:
Han-Wen Nienhuys
Date:
2012-04-04 @ 02:19
Also, does libgit2 keep file descriptors open across library calls ?

On Tue, Apr 3, 2012 at 3:21 PM, Vicent Marti <vicent@github.com> wrote:
> This is weird. We do create lockfiles to ensure atomic writing most
> objects (the index, ODB data, etc), but these are not NFS-aware. From
> what I've researched, `flock` is a no-op for NFS in most kernels, so
> it shouldn't prevent the directory removal.
>
> Can you post more details, including the OS and kernel used, the
> version of NFS, the contents of the `.nfs` file, and its exact path?
>
> Cheerios,
> Vicent
>
> On Tue, Apr 3, 2012 at 8:05 PM, Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
>> Hi there,
>>
>> I have a bunch of users that roughly do the following through a program
I wrote:
>>
>>  * manipulate a git repository using both libgit and git command line.
>>  * remove the repository on disk using shutil.rmtree (in python)
>>
>> I'm getting a bunch of reports that indicate the rmtree failing with:
>>
>> OSError: [Errno 16] Device or resource busy:
>> '/path/to/repository/some/directory/.nfs0000000002128ef300001dd4'
>>
>> this is for users on NFS.  Does libgit2 employ any sort of implicit or
>> explicit file locking that could result in this message?
>>
>> thanks,
>>
>> --
>> Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen



-- 
Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

Re: [libgit2] file locking in libgit2 ?

From:
Vicent Marti
Date:
2012-04-04 @ 02:31
On Wed, Apr 4, 2012 at 4:19 AM, Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
> Also, does libgit2 keep file descriptors open across library calls ?

Yes, during several complex operations. There are many objects in the
API that hold FDs until they are manually freed with the relevant
`free` call.

From the top of my head, `git_odb` holds descriptors open to keep
mapped pages in memory, and `git_odb_wstream` holds a descriptor open
as long as the user is streaming.

Re: [libgit2] file locking in libgit2 ?

From:
Han-Wen Nienhuys
Date:
2012-04-03 @ 18:29
On Tue, Apr 3, 2012 at 3:21 PM, Vicent Marti <vicent@github.com> wrote:
> This is weird. We do create lockfiles to ensure atomic writing most
> objects (the index, ODB data, etc), but these are not NFS-aware. From
> what I've researched, `flock` is a no-op for NFS in most kernels, so
> it shouldn't prevent the directory removal.

Can you point me to where in the source code that is happening?

from wikipedia:

Kernel 2.6.12 and above implement flock calls on NFS files using POSIX
byte-range locks. These locks will be visible to other NFS clients
that implement fcntl-style POSIX locks, but invisibile to those that
do not.[4]


> Can you post more details, including the OS and kernel used, the
> version of NFS, the contents of the `.nfs` file, and its exact path?

Unfortunately, I'm not sure how far I can go with given the NDAs I'm under.



> Cheerios,
> Vicent
>
> On Tue, Apr 3, 2012 at 8:05 PM, Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
>> Hi there,
>>
>> I have a bunch of users that roughly do the following through a program
I wrote:
>>
>>  * manipulate a git repository using both libgit and git command line.
>>  * remove the repository on disk using shutil.rmtree (in python)
>>
>> I'm getting a bunch of reports that indicate the rmtree failing with:
>>
>> OSError: [Errno 16] Device or resource busy:
>> '/path/to/repository/some/directory/.nfs0000000002128ef300001dd4'
>>
>> this is for users on NFS.  Does libgit2 employ any sort of implicit or
>> explicit file locking that could result in this message?
>>
>> thanks,
>>
>> --
>> Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen



-- 
Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen