librelist archives

Re: [homebrew] Re: Uninstall section in formulas?

From:

Justin Clift

Date:

2010-11-18 @ 19:25

On 18/11/2010, at 10:19 PM, Max Howell wrote:
> With Homebrew you have to tell the user what funny stuff they have to do
in the caveats.

Yeah.  My feeling is that will be a complete and total pain for users, for
the scenario I'm addressing. :(


> Homebrew was always designed to be a bare minimal package manager, the 
user should feel they are in complete control.

Understood.  It's been a good approach to get Homebrew up and running as a
useful package manager, but it's starting to get to the point where more 
advanced packaging will need to be catered for.  (in a way that doesn't 
increase complexity for people, which can definitely be done for this 
case)


> What do you need to do? As we have strategies.

It's the follow on, from an email a few weeks ago.

On *nix systems (except OSX, possibly some others), there is a central XML
catalog file:

  /etc/xml/catalog    <-- this is a text file (with no filename 
extension), not a directory

It "comes with the system" (seems to be installed by default), is world 
readable, but can only be written to by an admin (ie root).

Adding XML packages to a system (HTML DTD's, DocBook DTD's, whatever), 
follows two main steps:

  a) Install the actual DTD files in some location (can be flexible), such as

      /etc/mypackage/stuff/dtds/somefile.xml
      /etc/mypackage/stuff/dtds/anotherfile.xml
      /etc/mypackage/stuff/dtds/morefiles.xml

  b) Update the central XML catalog (/etc/xml/catalog), so packages which 
need the DTDs can find them.  For the XHTML DTDs, this means:

      system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Strict//EN' 'xhtml1-strict.dtd' /etc/xml/catalog"
      system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Transitional//EN' 'xhtml1-transitional.dtd' /etc/xml/catalog"
      system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Frameset//EN' 'xhtml1-frameset.dtd' /etc/xml/catalog"
      system "xmlcatalog --noout -add public '-//W3C//ENTITIES Latin 1 for
XHTML//EN' 'xhtml-lat1.ent' /etc/xml/catalog"
      system "xmlcatalog --noout -add public '-//W3C//ENTITIES Special for
XHTML//EN' 'xhtml-special.ent' /etc/xml/catalog"
      system "xmlcatalog --noout -add public '-//W3C//ENTITIES Symbols for
XHTML//EN' 'xhtml-symbol.ent' /etc/xml/catalog"
      system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/xhtml1/DTD/' './' /etc/xml/catalog"
      system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
/etc/xml/catalog"
      system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/xhtml1/DTD/' './' /etc/xml/catalog"
      system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
/etc/xml/catalog"

All package that use DTD's, know of the /etc/xml/catalog file, and will 
use it automatically.

With Homebrew though, the /etc/xml/catalog location isn't feasible.  It 
doesn't exist by default on OSX, and if we use this, then one Homebrew 
installation "wins" and has it's stuff used, while no other Homebrew 
installation on a system will then be able have it's own set.

So, to address this in a more Homebrew proper fashion, with the required 
location flexibility, I created a new formula called 
"xml-catalog-base.rb".

It creates an empty XML catalog file here:

  #{etc}/xml/catalog

The empty XML catalog file looks like this:

  <?xml version="1.0"?>
  <!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog 
V1.0//EN" 
"http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">
  <catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/>

When a person installs a DTD package (ie xhtml1-dtds.rb), the files for it
are added to the filesystem, plus the DTDs are added to this base catalog 
file:

  system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Strict//EN' 'xhtml1-strict.dtd' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Transitional//EN' 'xhtml1-transitional.dtd' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Frameset//EN' 'xhtml1-frameset.dtd' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add public '-//W3C//ENTITIES Latin 1 for 
XHTML//EN' 'xhtml-lat1.ent' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add public '-//W3C//ENTITIES Special for 
XHTML//EN' 'xhtml-special.ent' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add public '-//W3C//ENTITIES Symbols for 
XHTML//EN' 'xhtml-symbol.ent' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/xhtml1/DTD/' './' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
#{etc}/xml/catalog"
  system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/xhtml1/DTD/' './' #{etc}/xml/catalog"
  system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
/etc/xml/catalogue"

This results in the Homebrew base location catalog file looking like this:

  <?xml version="1.0"?>
  <!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog 
V1.0//EN" 
"http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">
  <catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
    <public publicId="-//W3C//DTD XHTML 1.0 Strict//EN" uri="xhtml1-strict.dtd"/>
    <public publicId="-//W3C//DTD XHTML 1.0 Transitional//EN" 
uri="xhtml1-transitional.dtd"/>
    <public publicId="-//W3C//DTD XHTML 1.0 Frameset//EN" 
uri="xhtml1-frameset.dtd"/>
    <public publicId="-//W3C//ENTITIES Latin 1 for XHTML//EN" 
uri="xhtml-lat1.ent"/>
    <public publicId="-//W3C//ENTITIES Special for XHTML//EN" 
uri="xhtml-special.ent"/>
    <public publicId="-//W3C//ENTITIES Symbols for XHTML//EN" 
uri="xhtml-symbol.ent"/>
    <rewriteSystem systemIdStartString="http://www.w3.org/TR/xhtml1/DTD/" 
rewritePrefix="./"/>
    <rewriteSystem 
systemIdStartString="http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/" 
rewritePrefix="./"/>
    <rewriteURI uriStartString="http://www.w3.org/TR/xhtml1/DTD/" 
rewritePrefix="./"/>
    <rewriteURI 
uriStartString="http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/" 
rewritePrefix="./"/>
  </catalog>

All good so far.  However, uninstalling the formula, because Homebrew 
presently doesn't support an "uninstall" target, means it can't run the 
xmlcatalog commands to remove those entries, thus breaking things:

  def uninstall
    # This cleanly removes the entries added to the central XML catalog
    system "xmlcatalog --noout -del '-//W3C//DTD XHTML 1.0 Strict//EN' 
#{etc}/xml/catalog"
    system "xmlcatalog --noout -del '-//W3C//DTD XHTML 1.0 
Transitional//EN' #{etc}/xml/catalog"
    system "xmlcatalog --noout -del '-//W3C//DTD XHTML 1.0 Frameset//EN' 
#{etc}/xml/catalog"
    system "xmlcatalog --noout -del '-//W3C//ENTITIES Latin 1 for 
XHTML//EN' #{etc}/xml/catalog"
    system "xmlcatalog --noout -del '-//W3C//ENTITIES Special for 
XHTML//EN' #{etc}/xml/catalog"
    system "xmlcatalog --noout -del '-//W3C//ENTITIES Symbols for 
XHTML//EN' #{etc}/xml/catalog"
    system "xmlcatalog --noout -del 'http://www.w3.org/TR/xhtml1/DTD/' 
#{etc}/xml/catalog"
    system "xmlcatalog --noout -del 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' #{etc}/xml/catalog"
    system "xmlcatalog --noout -del 'http://www.w3.org/TR/xhtml1/DTD/' 
#{etc}/xml/catalog"
    system "xmlcatalog --noout -del 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' #{etc}/xml/catalog"
  end

What would be nice, and keep things usable for the user with no real 
hassle, is if Homebrew supports running uninstall targets, PLUS if the 
user wants to uninstall without running that target (for maximum control),
they can tell it not to.  ie:

 + Normal uninstall, which would run the above commands to update the XML catalog:

    brew uninstall xhtml1-dtds
    (and that's it)

 + User specified "don't run the uninstall target" uninstall

    brew uninstall --someoption xhtml1-dtds
   (the DTD files are removed, but the above xmlcatalog commands don't get run)

That would seem to be the best of both worlds, expands Homebrew 
capabilities, and doesn't make writing formulas harder.   (new formula 
writers won't even notice)

?

Regards and best wishes,

Justin Clift

On 19/11/2010, at 6:25 AM, Justin Clift wrote:
>  b) Update the central XML catalog (/etc/xml/catalog), so packages which
need the DTDs can find them.  For the XHTML DTDs, this means:
> 
>      system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Strict//EN' 'xhtml1-strict.dtd' /etc/xml/catalog"
>      system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Transitional//EN' 'xhtml1-transitional.dtd' /etc/xml/catalog"
>      system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Frameset//EN' 'xhtml1-frameset.dtd' /etc/xml/catalog"
>      system "xmlcatalog --noout -add public '-//W3C//ENTITIES Latin 1 
for XHTML//EN' 'xhtml-lat1.ent' /etc/xml/catalog"
>      system "xmlcatalog --noout -add public '-//W3C//ENTITIES Special 
for XHTML//EN' 'xhtml-special.ent' /etc/xml/catalog"
>      system "xmlcatalog --noout -add public '-//W3C//ENTITIES Symbols 
for XHTML//EN' 'xhtml-symbol.ent' /etc/xml/catalog"
>      system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/xhtml1/DTD/' './' /etc/xml/catalog"
>      system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
/etc/xml/catalog"
>      system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/xhtml1/DTD/' './' /etc/xml/catalog"
>      system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
/etc/xml/catalog"

Ugh.  Munged those path names, missing an important point.  The DTD files 
are added to the XML catalog, with a complete path name to their location 
on the file system.  Like:

     system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Strict//EN' '/path/to/xhtml1-strict.dtd' /etc/xml/catalog"
     system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Transitional//EN' '/path/to/xhtml1-transitional.dtd' /etc/xml/catalog"
     system "xmlcatalog --noout -add public '-//W3C//DTD XHTML 1.0 
Frameset//EN' '/path/to/xhtml1-frameset.dtd' /etc/xml/catalog"
     system "xmlcatalog --noout -add public '-//W3C//ENTITIES Latin 1 for 
XHTML//EN' '/path/to/xhtml-lat1.ent' /etc/xml/catalog"
     system "xmlcatalog --noout -add public '-//W3C//ENTITIES Special for 
XHTML//EN' '/path/to/xhtml-special.ent' /etc/xml/catalog"
     system "xmlcatalog --noout -add public '-//W3C//ENTITIES Symbols for 
XHTML//EN' '/path/to/xhtml-symbol.ent' /etc/xml/catalog"
     system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/xhtml1/DTD/' './' /etc/xml/catalog"
     system "xmlcatalog --noout -add rewriteSystem 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
/etc/xml/catalog"
     system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/xhtml1/DTD/' './' /etc/xml/catalog"
     system "xmlcatalog --noout -add rewriteURI 
'http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/' './' 
/etc/xml/catalog"
 

On Thu, Nov 18, 2010 at 7:49 PM, Justin Clift <jclift@redhat.com> wrote:
> On 19/11/2010, at 6:25 AM, Justin Clift wrote:
> Ugh.  Munged those path names, missing an important point.  The DTD 
files are added to the XML catalog, with a complete path name to their 
location on the file system.  Like:

Hmm...

Referencing fully expanded paths is a problem for homebrew. Because if
you want to move your homebrew folder to another location - then those
hardcoded paths will break. If its at all possible, use relative
paths.

Also...

I would suggest that an xml catalog file is something that can be
generated dynamically (on the fly). If this is possible, consider
writing a homebrew formula to install a script which can generate the
xml.catalog file periodically. Like with a cron job.

The proper way to do this is install a launchd service that calls your
script (installed by homebrew). I have a branch in homwbrew for all
this launchd functionality. However the feature just hasn't been
reviewed yet by the homebrew maintainers.

dreamcat4
dreamcat4@gmail.com

On 19/11/2010, at 7:20 AM, Dreamcat4 wrote:
> Hmm...
> 
> Referencing fully expanded paths is a problem for homebrew. Because if
> you want to move your homebrew folder to another location - then those
> hardcoded paths will break. If its at all possible, use relative
> paths.

Interesting point.  Yeah, I don't know how to work around that. :/


> I would suggest that an xml catalog file is something that can be
> generated dynamically (on the fly).

Not the central one, no.  It needs to be an accurate list of the xml 
documents installed, which themselves can be installed "anywhere on the 
system" (from it's point of view).

On Thu, Nov 18, 2010 at 8:23 PM, Justin Clift <jclift@redhat.com> wrote:
> On 19/11/2010, at 7:20 AM, Dreamcat4 wrote:
>> Hmm...
>>
>> Referencing fully expanded paths is a problem for homebrew. Because if
>> you want to move your homebrew folder to another location - then those
>> hardcoded paths will break. If its at all possible, use relative
>> paths.
>
> Interesting point.  Yeah, I don't know how to work around that. :/
>
>
>> I would suggest that an xml catalog file is something that can be
>> generated dynamically (on the fly).
>
> Not the central one, no.  It needs to be an accurate list of the xml 
documents installed, which themselves can be installed "anywhere on the 
system" (from it's point of view).
>

Sorry - I thought you said that the xml catalog goes in #{etc} as in
"$HOMEBREW_PREFIX/etc/" ? Then it would be specific to a certain
homebrew folder and the generation of that file could therefore be
managed from a periodically scheduled script.

dreamcat4
dreamcat4@gmail.com

On 19/11/2010, at 7:26 AM, Dreamcat4 wrote:
<snip>
> Sorry - I thought you said that the xml catalog goes in #{etc} as in
> "$HOMEBREW_PREFIX/etc/" ?

Sorry.  Probably getting my variable names mixed up.

I'm meaning (in the common scenario):

  /usr/local/etc/xml/catalog


> Then it would be specific to a certain
> homebrew folder and the generation of that file could therefore be
> managed from a periodically scheduled script.

Yeah, there's no way at all that would work.

What are we installing (via Homebrew) that depends on xhtml1-dtds?

Does it need to be hard dependency?
You can always have some other script that knows how to install and
uninstall xhtml1-dtds and do whatever it needs to do.

And then you could do "brew install osx-xml-catalog-manager-thing" and
then use osx-xml-catalog-manager-thing to actually manage the catalog.

On 19/11/2010, at 6:32 AM, Adam Vandenberg wrote:
> What are we installing (via Homebrew) that depends on xhtml1-dtds?

libvirt.  If a person want's to build the docs, it requires the DTDs.

The xhtml1-dtds are just an example.  There are a bunch more DTDs out there. :)


> Does it need to be hard dependency?
> You can always have some other script that knows how to install and
> uninstall xhtml1-dtds and do whatever it needs to do.

Sure.  This is the approach the DocBook formula takes, writing to 
/etc/xml/catalog through sudo, in that example.

And, it's a bad way to go.  There are lots of DTD's around, and having 
support for uninstall targets seems like a much simpler and nicer solution
than having each DTD author have to invent/code/etc their own scripted 
approach.

I know that for me, even writing the xhtml1-dtd formula is being a pain 
because these curly problems haven't been solved (yet).  I honestly wish 
someone before me had got it sorted, so I could just write the formula and
it would work for people.

I kind of expect other potential DTD authors would have had the same 
thought, and just not bothered. :/


> And then you could do "brew install osx-xml-catalog-manager-thing" and
> then use osx-xml-catalog-manager-thing to actually manage the catalog.

Not really a fan of this idea, it seems like it's adding more scripting, 
to work around one specific problem, instead of addressing the core 
problem itself.

The other problem with this approach, not just that it's another script, 
is how do we tell the user they need to run this after the uninstall?

With the install, it's easy.  Use the caveats.  There doesn't seem to be a
similar thing for uninstall info though. :/

On 18 November 2010 19:25, Justin Clift <jclift@redhat.com> wrote:
> Understood.  It's been a good approach to get Homebrew up and running as
a useful package manager, but it's starting to get to the point where more
advanced packaging will need to be catered for.  (in a way that doesn't 
increase complexity for people, which can definitely be done for this 
case)

Sorry, I don't agree here. brew uninstall will increase complexity for
the users and us as it will allow packages to interact far more than
they do already.

I don't think we need to cater for more advanced packaging. We're not
packaging the whole system, just applications so we can (and should,
frankly) remain a lot more simple than Linux package managers.


>  a) Install the actual DTD files in some location (can be flexible), such as

This much is easy, they'll go into the Homebrew etc directory fine.

>  b) Update the central XML catalog (/etc/xml/catalog), so packages which
need the DTDs can find them.  For the XHTML DTDs, this means:

> That would seem to be the best of both worlds, expands Homebrew 
capabilities, and doesn't make writing formulas harder.   (new formula 
writers won't even notice)

Expanding capabilities does make stuff harder as packages will start
to rely on it and we'll have to implement it and fix the bugs that
result.


On 18 November 2010 19:47, Justin Clift <jclift@redhat.com> wrote:
> On 19/11/2010, at 6:32 AM, Adam Vandenberg wrote:
>> What are we installing (via Homebrew) that depends on xhtml1-dtds?
>
> libvirt.  If a person want's to build the docs, it requires the DTDs.
>
> The xhtml1-dtds are just an example.  There are a bunch more DTDs out there. :)

This is something that I think is managed in an idiotic fashion on
Linux and I'd rather we didn't do the same. These centralised
databases are great when every package depends on a bunch of others
but it just makes managing packages a pain and installing them take
longer.

Why can't we just prebuild the documentation and ship them? Most
formula we don't even ship the docs but I don't see a good reason why
the user needs to build them from scratch, particularly in the case
that we need to make extensions to Homebrew to do so.

I think we need a sexier reason than building documentation.


>> Does it need to be hard dependency?
>> You can always have some other script that knows how to install and
>> uninstall xhtml1-dtds and do whatever it needs to do.
>
> Sure.  This is the approach the DocBook formula takes, writing to 
/etc/xml/catalog through sudo, in that example.
>
> And, it's a bad way to go.  There are lots of DTD's around, and having 
support for uninstall targets seems like a much simpler and nicer solution
than having each DTD author have to invent/code/etc their own scripted 
approach.
>
> I know that for me, even writing the xhtml1-dtd formula is being a pain 
because these curly problems haven't been solved (yet).  I honestly wish 
someone before me had got it sorted, so I could just write the formula and
it would work for people.
>
> I kind of expect other potential DTD authors would have had the same 
thought, and just not bothered. :/

Sorry but this is another case where I think we're somewhat missing
the point of Homebrew. Homebrew should be about delivering
applications to users that aren't available in decent prepackaged
other forms or are console applications that make sense to build and
run from the command-line. I don't think Homebrew needs to become
something that builds every single feature of every single library and
I'm seriously thinking that this DTD stuff is a solution that's
looking for a problem if it's just for managing building
documentation.

Why can't we just prebuild the documentation and ship that to the
users? Can't we just turn off building documentation altogether? Is
this documentation not available to be viewed online or prebuilt
anywhere?


> Not really a fan of this idea, it seems like it's adding more scripting,
to work around one specific problem, instead of addressing the core 
problem itself.
>
> The other problem with this approach, not just that it's another script,
is how do we tell the user they need to run this after the uninstall?
>
> With the install, it's easy.  Use the caveats.  There doesn't seem to be
a similar thing for uninstall info though. :/

It's a matter of making DTDs harder to manage or making Homebrew
harder to manage. I can't see a reason why most of our users will care
about the prior.

One of the biggest problems I keep running into as a software engineer
that Max has inspired me to try and avoid is the compulsive need to be
"neat" and "consistent" over being "useful" and "simple". I think this
is an admirable goal; if you need a more comprehensive emulation of
Linux package management, Macports and Fink are always an option. As
with OS X itself, it's not trying to be all things to all people, it's
just trying to make certain kinds of users happier. Some people will
stay on Windows and that's fine.

-- 
Mike McQuaid
http://mikemcquaid.com

Hi all

Thoughts on the benefits of uninstall capability, and some observations
that the 'tricky stuff' doesn't have to be that hard to manage...

Mike McQuaid wrote:
> On 18 November 2010 19:25, Justin Clift <jclift@redhat.com> wrote:
>   
>> Understood.  It's been a good approach to get Homebrew up and running 
as a useful package manager, but it's starting to get to the point where 
more advanced packaging will need to be catered for.  (in a way that 
doesn't increase complexity for people, which can definitely be done for 
this case)
>>     
>
> Sorry, I don't agree here. brew uninstall will increase complexity for
> the users and us as it will allow packages to interact far more than
> they do already.
>
> I don't think we need to cater for more advanced packaging. We're not
> packaging the whole system, just applications so we can (and should,
> frankly) remain a lot more simple than Linux package managers.
>   

I find that the uninstall feature of a package manager is a very useful
feature for for development work. On Linux, sometimes I want to check
that the dependencies are complete and correct, so I can uninstall all
the dependencies and check that the package still installs correctly.
Sometimes I want to be able to test optional features in my package,
such as some graphics capability that becomes active in the present of
certain packages. I need to check that the code doesn't baulk when these
optional packages are missing, either.

Uninstall is also useful for getting one's system back to an earlier
state, eg if uninstalling a new version of a package and reinstalling an
older one. Packages like SWIG and SCons have caused me problems in the
past when new versions come along, and for expediency sometimes you just
want to use the old version, until you have time to resolve the problem
properly.

In RPM and Debian, the uninstall feature basically works by maintaining
lists of the files that are installed (and subsequently can be removed),
plus the provision for pre- and post- install and uninstall script
actions. Debian provides a wide range of scripts associated that do
tasks like registering shared libraries, adding and removing MIME types,
creating 'open' and 'edit' actions for files, etc. These various scripts
can of course be bundled up as a package themselves, which then becomes
a dependencies.

I think that this stuff doesn't have to add hugely to the complexity of
the package manager, but packages with script actions need to be
carefully checked to ensure there are no residual changes left behind by
the pre/post scripts.

Cheers
JP

On 19/11/2010, at 7:08 AM, Mike McQuaid wrote:
<snip>
> Why can't we just prebuild the documentation and ship them?

UGH.  I'm _building documentation_ on my system.  As in _I'm a 
documentation author_.

There's no _packaging up of it_ that can be done.  I need to have this 
stuff installed, so I can generate it, and test it.


> I think we need a sexier reason than building documentation.

Huh?

Isn't documentation one of the biggest things that people bitch about open
source software not having much depth nor quality with?

How's that going to improve, if we keep on pushing back of the efforts to 
at least make it workable?


> Sorry but this is another case where I think we're somewhat missing
> the point of Homebrew.

Sure.  I thought Homebrew was a "package manager".  There's nowhere on the
Homebrew site, _at all_, that says "Homebrew is an underpowered pacakge 
manager, that can't even uninstall stuff properly.  Support for 
uninstalling things, or doing even not-real-complex operations during 
install isn't, and won't be supported."

Can someone please update the Homebrew page to mention this?  With 
libvirt, that would have saved us from wasting a bunch of time, as I 
wouldn't have bothered with Homebrew (rightfully so).

When needing to choose a package manager, they options were Fink or 
MacPorts.  Homebrew only came up later, through someone's url.

If the Homebrew page actually mentioned this "philosophy" of 
incredibly-limited-install-types, that would have been of great help in 
AVOIDING it.


> One of the biggest problems I keep running into as a software engineer
> that Max has inspired me to try and avoid is the compulsive need to be
> "neat" and "consistent" over being "useful" and "simple". I think this
> is an admirable goal; if you need a more comprehensive emulation of
> Linux package management, Macports and Fink are always an option.

Damn straight.  Homebrew was the wrong choice.  Won't be continuing this mistake.

Please update the Homebrew pages to reflect this previously not mentioned 
"philosophy", so other people don't put their time into a system that just
doesn't work properly.

On 18 November 2010 20:19, Justin Clift <jclift@redhat.com> wrote:
> On 19/11/2010, at 7:08 AM, Mike McQuaid wrote:
> <snip>
>> Why can't we just prebuild the documentation and ship them?
> UGH.  I'm _building documentation_ on my system.  As in _I'm a 
documentation author_.
> There's no _packaging up of it_ that can be done.  I need to have this 
stuff installed, so I can generate it, and test it.

That's fine, I'm just saying that your particular use-case isn't one I
think Homebrew needs be changed to cater for. You could build your
documentation your own way and then package it for other users.


>> I think we need a sexier reason than building documentation.
>
> Huh?
>
> Isn't documentation one of the biggest things that people bitch about 
open source software not having much depth nor quality with?

I don't hear them say that and not about Homebrew or the other
open-source projects I work on.


> How's that going to improve, if we keep on pushing back of the efforts 
to at least make it workable?

I'm not. I'm saying that Homebrew doesn't need to be a core part of
your documentation workflow.


> Sure.  I thought Homebrew was a "package manager".  There's nowhere on 
the Homebrew site, _at all_, that says "Homebrew is an underpowered 
pacakge manager, that can't even uninstall stuff properly.  Support for 
uninstalling things, or doing even not-real-complex operations during 
install isn't, and won't be supported."

You're the first person I've heard complain about this. I didn't say
it would never be supported, just that your use case doesn't sound
compelling enough for us to complicate things and deal with the
resulting bugs.


> Can someone please update the Homebrew page to mention this?  With 
libvirt, that would have saved us from wasting a bunch of time, as I 
wouldn't have bothered with Homebrew (rightfully so).
>
> When needing to choose a package manager, they options were Fink or 
MacPorts.  Homebrew only came up later, through someone's url.
>
> If the Homebrew page actually mentioned this "philosophy" of 
incredibly-limited-install-types, that would have been of great help in 
AVOIDING it.
>
> Damn straight.  Homebrew was the wrong choice.  Won't be continuing this
mistake.
>
> Please update the Homebrew pages to reflect this previously not 
mentioned "philosophy", so other people don't put their time into a system
that just doesn't work properly.

I assume all your requests above are rhetorical. Generally insulting
people's work isn't the best way to encourage people who all have busy
day jobs (that don't involved Homebrew) and work on Homebrew in their
free time to document that their project sucks because you think it
does.

I was trying to have a technical discussion here. If you want to
resort to insults, feel free to follow me on Twitter or email me
personally rather than wasting the time of the people who read this
list.

-- 
Mike McQuaid
http://mikemcquaid.com

On 19/11/2010, at 8:07 AM, Mike McQuaid wrote:
> I assume all your requests above are rhetorical. Generally insulting
> people's work isn't the best way to encourage people who all have busy
> day jobs (that don't involved Homebrew) and work on Homebrew in their
> free time to document that their project sucks because you think it
> does.

Heh, yeah, sorry about that.  Long, long, long week.  :/

On the Homebrew website aspect of things though, I do reckon it would be 
good to clearly state Homebrew's design philosophy, with it's strengths 
and weaknesses.

Maybe a "feature set matrix" (like a table or something), with what 
Homebrew does, and probably also MacPorts and _maybe_ Fink?

As background, over the years, I've done packaging of apps across a fairly
wide variety of systems (Solaris, many BSD's, Linux rpm's, Linux deb's 
(ugh), various Windows types).

Pretty much every packaging system has the equivalent of "uninstall 
scripting" of some form or another.  To the point where I personally just 
expected it to be present, kind of regarding it as a basic requirement for
a packaging system.

This is obviously not the case in reality though, as Homebrew doesn't do it. ;)

So, learn something every day I s'pose. :/

For what its worth, I started some companion documentation here:
https://github.com/adamv/homebrew-cookbook
(but have slowed way down on it.)

On Sat, Nov 20, 2010 at 9:07 PM, Justin Clift <jclift@redhat.com> wrote:
> On 19/11/2010, at 8:07 AM, Mike McQuaid wrote:
>> I assume all your requests above are rhetorical. Generally insulting
>> people's work isn't the best way to encourage people who all have busy
>> day jobs (that don't involved Homebrew) and work on Homebrew in their
>> free time to document that their project sucks because you think it
>> does.
>
> Heh, yeah, sorry about that.  Long, long, long week.  :/
>
> On the Homebrew website aspect of things though, I do reckon it would be
good to clearly state Homebrew's design philosophy, with it's strengths 
and weaknesses.
>
> Maybe a "feature set matrix" (like a table or something), with what 
Homebrew does, and probably also MacPorts and _maybe_ Fink?
>
> As background, over the years, I've done packaging of apps across a 
fairly wide variety of systems (Solaris, many BSD's, Linux rpm's, Linux 
deb's (ugh), various Windows types).
>
> Pretty much every packaging system has the equivalent of "uninstall 
scripting" of some form or another.  To the point where I personally just 
expected it to be present, kind of regarding it as a basic requirement for
a packaging system.
>
> This is obviously not the case in reality though, as Homebrew doesn't do it. ;)
>
> So, learn something every day I s'pose. :/
>

On 21 November 2010 05:07, Justin Clift <jclift@redhat.com> wrote:
> On 19/11/2010, at 8:07 AM, Mike McQuaid wrote:
>> I assume all your requests above are rhetorical. Generally insulting
>> people's work isn't the best way to encourage people who all have busy
>> day jobs (that don't involved Homebrew) and work on Homebrew in their
>> free time to document that their project sucks because you think it
>> does.
>
> Heh, yeah, sorry about that.  Long, long, long week.  :/

Don't worry about it.


> On the Homebrew website aspect of things though, I do reckon it would be
good to clearly state Homebrew's design philosophy, with it's strengths 
and weaknesses.

Agreed.


> Maybe a "feature set matrix" (like a table or something), with what 
Homebrew does, and probably also MacPorts and _maybe_ Fink?

I'm not sure about this. I've never liked feature set matrices, they
tend to be fairly biased and also don't count who good you are at
certain aspects.


> As background, over the years, I've done packaging of apps across a 
fairly wide variety of systems (Solaris, many BSD's, Linux rpm's, Linux 
deb's (ugh), various Windows types).
>
> Pretty much every packaging system has the equivalent of "uninstall 
scripting" of some form or another.  To the point where I personally just 
expected it to be present, kind of regarding it as a basic requirement for
a packaging system.

Python setuptools and Perl CPAN don't, perhaps it's worth us referencing these.


> This is obviously not the case in reality though, as Homebrew doesn't do it. ;)
>
> So, learn something every day I s'pose. :/

Aye.

-- 
Mike McQuaid
http://mikemcquaid.com

On 22/11/2010, at 7:29 PM, Mike McQuaid wrote:
>> As background, over the years, I've done packaging of apps across a 
fairly wide variety of systems (Solaris, many BSD's, Linux rpm's, Linux 
deb's (ugh), various Windows types).
>> 
>> Pretty much every packaging system has the equivalent of "uninstall 
scripting" of some form or another.  To the point where I personally just 
expected it to be present, kind of regarding it as a basic requirement for
a packaging system.
> 
> Python setuptools and Perl CPAN don't, perhaps it's worth us referencing these.

That's actually a very good point.  I'd never thought of CPAN as a 
"packaging system", but just as an "easy installer" for perl.  But yeah, I
guess it _could_ be viewed as a packaging system. ;)

Ironically, one of the core mis-features of Perl/CPAN that stops it from 
wide use in most of the enterprises (I've been in), is it's inability to 
cleanly back out installation changes.

When a SysAdmin has to write up a "Change Plan" for a system, it needs to 
include a "Back out Plan" if operation (as a whole) being performed 
doesn't work for some reason.

So, with Perl/CPAN not having a solid way of reversing changes... things 
that rely on it don't get approved (unless people seek 
exceptions/exemptions/etc).  Consequently, people try to use non Perl/CPAN
stuff.

But, that's just my perspective.  It's been a few years since I've last 
looked at the Perl thing, so maybe there are workarounds now. ;)

(also not familiar with Python yet, so no idea about the setuptools you mentioned)

On 22 Nov 2010, at 09:59, Justin Clift wrote:

>> Python setuptools and Perl CPAN don't, perhaps it's worth us referencing these.

> also not familiar with Python yet, so no idea about the setuptools you mentioned

Anecdotally, I'd say the move is away from setuptools (perhaps in part 
because of uninstall issues).

The mindshare seems to be with pip [1] and virtualenv these days. A 
combination of these allows easy "Back out Plans" -- pip has uninstall; 
virtualenvs are cheap to create/destroy.

As such, citing setuptools may be a red-herring.

[1] http://pypi.python.org/pypi/pip

On 22 November 2010 10:15, Carlton Gibson <carlton.gibson@noumenal.co.uk> wrote:
> Anecdotally, I'd say the move is away from setuptools (perhaps in part 
because of uninstall issues).
>
> The mindshare seems to be with pip [1] and virtualenv these days. A 
combination of these allows easy "Back out Plans" -- pip has uninstall; 
virtualenvs are cheap to create/destroy.
>
> As such, citing setuptools may be a red-herring.

It's not a red herring because it's a packaging system that doesn't
support uninstalling, regardless of if it's the cool thing to use at
the moment.

Also, we're in a slightly different situation anyway in that we
support very clean uninstalling (arguably, far more so than most
package managers) _because_ we don't allow uninstall hooks and
formulas writing outside of their prefix on installation. As a result,
I only have a single path I can manually delete even if brew goes
horribly wrong.

-- 
Mike McQuaid
http://mikemcquaid.com

On 22 Nov 2010, at 11:10, Mike McQuaid wrote:

> It's not a red herring because it's a packaging system that doesn't
> support uninstalling, regardless of if it's the cool thing to use at
> the moment.

It's not really about what's cool or not. Presumably you'll grant that 
it's not much of an argument to say "it's okay that we don't do A because 
neither does obsolescent system X". (setuptools is obsolescent, not just 
uncool.)

> Also, we're in a slightly different situation anyway in that we
> support very clean uninstalling (arguably, far more so than most
> package managers) _because_ we don't allow uninstall hooks and
> formulas writing outside of their prefix on installation. As a result,
> I only have a single path I can manually delete even if brew goes
> horribly wrong.

This seems the relevant point. (Going back to the Python example, one of 
the good things about virtualenvs is that you you can just delete your 
environment if it all goes wrong.)

On 22 November 2010 11:37, Carlton Gibson <carlton.gibson@noumenal.co.uk> wrote:
>
> On 22 Nov 2010, at 11:10, Mike McQuaid wrote:
>
>> It's not a red herring because it's a packaging system that doesn't
>> support uninstalling, regardless of if it's the cool thing to use at
>> the moment.
>
> It's not really about what's cool or not. Presumably you'll grant that 
it's not much of an argument to say "it's okay that we don't do A because 
neither does obsolescent system X". (setuptools is obsolescent, not just 
uncool.)

It's not much of an argument. That's why I didn't use it. Justin said
'[p]retty much every packaging system has the equivalent of "uninstall
scripting"'. I said 'Python setuptools and Perl CPAN don't, perhaps
it's worth us referencing these.'

Can we bury the horse and stop beating it?

-- 
Mike McQuaid
http://mikemcquaid.com

Adam's rude opinion: building docs from source is lame, which is why
we grab pre-built versions for git for instance. Users shouldn't have
to care about the XML format that docs come from.

On 18 November 2010 21:19, Adam Vandenberg <flangy@gmail.com> wrote:
> Adam's rude opinion: building docs from source is lame, which is why
> we grab pre-built versions for git for instance. Users shouldn't have
> to care about the XML format that docs come from.

This is the thing, it's not rude, it's sensible.

The binaries we built vary depending on the processor. The
documentation is _always the same_ and frequently relies on a whole
LaTeX or Docbook buildchain that our users don't care about at all :)


-- 
Mike McQuaid
http://mikemcquaid.com

        
        Justin, dude, chill out.Homebrew is meant to make you feel safe, 
the formula can't fuck your system up. You have to do that yourself. Hence
anything that requires scripting is discouraged and won't be merged and if
you have to do more than rm to remove the package then you would have had 
to do more than `brew install` when you installed it.Make a caveats 
function that tells the user they need to run xmlcatalog and you are done.
If this is not good enough then Homebrew is not the right tool for your 
needs.If this isn't in the wiki then yeah, it probably should be. As a 
documentation writer yourself I thought you'd perhaps be more sympathetic 
to the fact that writing documentation is hard and that often there are 
bits missing.Max