librelist archives

Homebrew: flaky linking behaviour blows away real files

From:

Justin Clift

Date:

2010-09-26 @ 10:41

Hi,

Trying _really hard_ to get a Homebrew formula to work.  But Homebrew
is being just plain unpredictable and frustrating. :(

What I'm trying to achieve is have soft links created after the
"make install" portion of a formula.  The soft links need to point
FROM files in the Cellar etc/ directory, TO their equivalent
locations in #{HOMEBREW_PREFIX}/etc/.  i.e.:

   /usr/local/etc/pki/CA/cacert.pem  (certificate file - link target)
      should be pointed TO by:
   /usr/local/Cellar/libvirt/0.8.4-4/etc/pki/CA/cacert.pem (link source)

This is so the certificate files have a stable location through upgrades
of libvirt.

Sounds simple enough in theory.  Something like this should work, after
the make install:

   system "ln -s #{HOMEBREW_PREFIX}/etc/pki/CA/cacert.pem 
#{prefix}etc/pki/CA/cacert.pem"

(that's all one line, in case the mail reader wraps it)

But in practice it doesn't work.  Using that approach, Homebrew has the
link reversed, blowing away the certificate file.  Swapping around
source and target arguments makes no difference, it still destroys the
certificate file. (ugh)

Taking a different approach, by looking how the "ln_s" command is used
in other formula's (ie. mysql.rb) instead gives unreliable behaviour.
For example, using this (copied from the MySQL formula and modified):

   ln_s "#{HOMEBREW_PREFIX}/bin/testtarget", "#{libexec}/mysqld2"

Works.  It creates the (nonexistent) libexec dir in the libvit Cellar
after the make install, then creates a link from mysqld2 in there to
/usr/local/bin/testtarget.

Slightly adjusting the source file name to closer meet what we really
need, gives:

   ln_s "#{HOMEBREW_PREFIX}/bin/testtarget", "#{libexec}/cacert.pem"

This doesn't work. No libexec dir is created, no cacert.pem link appears
anywhere, and no mention of them is in the output of using -vd with brew 
install.  From just a change of source name.  (that's REALLY flaky)

Been trying to get this working for _many_ hours today, with a lot
of variations of things, and no luck.

The ln_s command doesn't appear in any Homebrew documentation I could
see, and I didn't get lucky having an open question in IRC for
a few hours either.

Does anyone know how to do this with Homebrew, or if it's even capable 
of doing links like we need?

Regards and best wishes,

Justin Clift

Hi Justin,

This should work, I think the problem is that you aren't creating any
of the directories before trying to symlink.

The ln_s function isn't documented in Homebrew because it's a Ruby
standard library function. This is also why you don't get any verbose
output when you install with -vd. I'm not sure however why it doesn't
throw an exception if it fails to make the symlink though. Something
is amiss.

I suggest that you instead configure libvrt to set its etc directory
to HOMEBREW_PREFIX/etc. This is usually a simple configure argument.
We do this for some other formula.

Max

On 09/27/2010 05:13 AM, Max Howell wrote:
<snip>
> I suggest that you instead configure libvrt to set its etc directory
> to HOMEBREW_PREFIX/etc. This is usually a simple configure argument.
> We do this for some other formula.

Thanks Max.  That was the correct direction to take.  Instructing
the configure script to use $HOMEBREW_PREFIX/etc instead was exactly
the right move.  This also put me onto the right track for configuring
other locations too. :)


 > This should work, I think the problem is that you aren't creating any
 > of the directories before trying to symlink.

Tried many approaches prior to emailing, including creating the
required containing directories manually before linking using:

   system "mkdir -p foo"

It didn't help. :(  Maybe the problem was in using the system ""
approach, rather than using Ruby's inbuilt "mkdir_p" or something.

As a consideration for future people, just updated the Formula-Cookbook
to now mention using ln_s and similar is possible, plus added a link to
the relevant Ruby doc.

(that Formula Cookbook page really needs an index)

Hope that helps.

Regards and best wishes,

Justin Clift

> What I'm trying to achieve is have soft links created after the
> "make install" portion of a formula.  The soft links need to point
> FROM files in the Cellar etc/ directory, TO their equivalent
> locations in #{HOMEBREW_PREFIX}/etc/.  i.e.:
> 
>    /usr/local/etc/pki/CA/cacert.pem  (certificate file - link target)
>       should be pointed TO by:
>    /usr/local/Cellar/libvirt/0.8.4-4/etc/pki/CA/cacert.pem (link source)
> 
> This is so the certificate files have a stable location through upgrades
> of libvirt.

My guess is that “brew link” is the part that's clobbering the cert file
in #{HOMEBREW_PREFIX}/etc, since it's trying to install a link there to
the file that it sees in the Cellar.  I think it's perfectly happy to
overwrite whatever's already there, since the intention is that anything
you place into the Cellar should be linked into #{HOMEBREW_PREFIX} when
you activate a particular version.

Do you need to have a symlink in the Cellar?  If all you want is to
prevent the cert file from being overwritten upgrades, it should be
enough to just place it in #{HOMEBREW_PREFIX}/etc.  After you run “brew
link” on the formula, you'll have a working copy (including cert file)
in #{HOMEBREW_PREFIX}.  You only need the symlink if there's something
that expects to find the cert file in the Cellar — is that the case?

cheers
–doug

On 09/27/2010 12:44 AM, Douglas Creager wrote:
> My guess is that “brew link” is the part that's clobbering the cert file
> in #{HOMEBREW_PREFIX}/etc, since it's trying to install a link there to
> the file that it sees in the Cellar.  I think it's perfectly happy to
> overwrite whatever's already there, since the intention is that anything
> you place into the Cellar should be linked into #{HOMEBREW_PREFIX} when
> you activate a particular version.

Thanks Doug.  Agreed, this seems to be the concept and approach it was
taking.  Still, it's kind of dangerous for it to be blowing away real
files without checking though. :(


> Do you need to have a symlink in the Cellar?

As it turned out, no.  The better option, suggested by Max, was to just
tell configure to use HOMEBREW_PREFIX/etc as the package config
directory instead.  No need to mess around with linking at all this
way. :)

Regards and best wishes,

Justin Clift

>> My guess is that “brew link” is the part that's clobbering the cert file
>> in #{HOMEBREW_PREFIX}/etc, since it's trying to install a link there to
>> the file that it sees in the Cellar.  I think it's perfectly happy to
>> overwrite whatever's already there, since the intention is that anything
>> you place into the Cellar should be linked into #{HOMEBREW_PREFIX} when
>> you activate a particular version.
>
> Thanks Doug.  Agreed, this seems to be the concept and approach it was
> taking.  Still, it's kind of dangerous for it to be blowing away real
> files without checking though. :(

Symlinks in /usr/local are considered fair game for clobbering. Really
it's against Homebrew policy to do anything outside of your keg.

However etc is special. I'm designing a better system for etc.

On 09/27/2010 09:09 PM, Max Howell wrote:
<snip>
> Symlinks in /usr/local are considered fair game for clobbering.

Sure.  It was blowing away *real files* though, not symlinks.

> <snip>
>>
>> Symlinks in /usr/local are considered fair game for clobbering.
>
> Sure.  It was blowing away *real files* though, not symlinks.

I found this incredulous. So I tested it. It really happens. The
problem is we force the symlink creation. And apparently if you force
it, it will even overwrite existing files. I guess this makes sense,
but I naively believed it would only replace existing symlinks.

Simple fix coming up.