Re: [homebrew] Howto create/distribute bottles?
- From:
- William Woodall
- Date:
- 2011-08-25 @ 19:57
Great that was what I was looking for.
But, let me ask another question. I am working on getting the ROS (Robot
Operating System) working better on OS X and I have moved to using Homebrew
from Macports for many reasons, but what I am interesting in is using the
bottle system to package some of the larger parts of ROS to simplify and
expedite installation. Specifically, OGRE, rViz, opencv, and a few other
components in ROS take more than 10 minutes to build on my i7 mbp w/ a solid
state drive. I would like to perhaps have a fork of Homebrew specifically
for ROS with separate hosting (probably Willowgarage or source forge) for
the bottles. What is the best path to this? Is there (should there be) a
mechanism for having "overlays" or third party Formula and Bottle
repositories which still uses the master Homebrew system from github but
overrides/extends the Formula and Bottle list? I am thinking specifically
of something similar to ppa's in apt or overlays in Gentoo's portage system.
Thanks for everyone's help,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
William Woodall
Graduate Software Engineering
Auburn University
w@auburn.edu
wjwwood@gmail.com
williamjwoodall.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Thu, Aug 25, 2011 at 2:46 PM, Charlie Sharpsteen <chuck@sharpsteen.net>wrote:
> On Thu, Aug 25, 2011 at 12:16 PM, William Woodall <wjwwood@gmail.com>wrote:
>
>> Hi, I have recently updated my local opencv Formula to use the 2.3.1
>> version of OpenCV and I was wondering if it would be a good candidate for
>> bottling?
>>
>> More generally what the limitations of bottling and what types of packages
>> or characteristics of packages lend themselves to bottling?
>>
>
> The idea behind bottling is to speed up installation of large monolithic
> packages that take a long time to compile. The prime candidate is Qt which
> takes 45 minutes on my 8-core Mac Pro and five our six hours on single-core
> VMs.
>
> The limitations are that bottles are a packaged version of what the formula
> produces when installed on OS X 10.6 with no arguments. So a user that
> passes an argument to a formula will end up building from source and a user
> running OS 10.5 will be forced to build from source as the bottle was
> compiled on Snow Leopard. Another limitation is that bottling a formula
> causes a lot of maintenance overhead with ensuring that the binary builds
> are up to date with the source builds.
>
> So, to recap---the prime candidates for bottling are formula where a good
> chunk of time, say 5 minutes or more, is spent compiling a single formula.
> For formula that take a long time to install because they have 5 dozen
> dependencies that each build rather quickly, bottling probably isn't
> appropriate.
>
>
> Also, if I were going to, what is the best (prescribed) method for creating
>> and distributing bottles?
>>
>
> Bottles can be created using the external command `brew bottle`:
>
>
>
https://github.com/mxcl/homebrew/blob/master/Library/Contributions/examples/brew-bottle.rb
>
> After the bottle is created, open an issue with a link to the file. A
> member of the core team will then review the bottle and add it to the binary
> repository which is hosted at SourceForge:
>
> http://sf.net/projects/machomebrew
>
> The only valid for distributing bottles for formula in the mxcl's master is
> this SourceForge site---this is done in order to insure that the binaries
> don't contain anything harmful or malicious.
>
> -Charlie
>
Re: [homebrew] Howto create/distribute bottles?
- From:
- Charlie Sharpsteen
- Date:
- 2011-08-25 @ 20:35
On Thu, Aug 25, 2011 at 12:57 PM, William Woodall <wjwwood@gmail.com> wrote:
> Great that was what I was looking for.
>
> But, let me ask another question. I am working on getting the ROS (Robot
> Operating System) working better on OS X and I have moved to using Homebrew
> from Macports for many reasons, but what I am interesting in is using the
> bottle system to package some of the larger parts of ROS to simplify and
> expedite installation. Specifically, OGRE, rViz, opencv, and a few other
> components in ROS take more than 10 minutes to build on my i7 mbp w/ a solid
> state drive. I would like to perhaps have a fork of Homebrew specifically
> for ROS with separate hosting (probably Willowgarage or source forge) for
> the bottles. What is the best path to this? Is there (should there be) a
> mechanism for having "overlays" or third party Formula and Bottle
> repositories which still uses the master Homebrew system from github but
> overrides/extends the Formula and Bottle list? I am thinking specifically
> of something similar to ppa's in apt or overlays in Gentoo's portage system.
>
> Thanks for everyone's help,
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> William Woodall
>
Sure. We're working on making it easier for people to manage and access
external repositories that contain specialized formula. I wrote a tool
called `brew tap` that is currently being re-worked and is on the list of
"features to add" for the next release of Homebrew. Info on `brew-tap` can
be found in this pull request:
https://github.com/mxcl/homebrew/pull/6086
The best way to get started with it is to:
1. Fork Adam's Homebrew-alt repository,
https://github.com/adamv/homebrew-alt, as `brew-tap` uses the fork
network of that repository to generate its list of available external
repositories.
2. If you want to start from scratch, clean out the master branch of your
Homebrew-alt fork by `git reset --hard aa06758` followed by a `git push -f
origin master`
3. Commit your customized formulae to the Homebrew-alt fork---organize
however you want. Currently, the only rules are that formula files must end
in `.rb` and that any file ending in `.rb` must be a valid formula (no odd
ruby scripts floating around).
Dependencies on formula in external repositories can be declared via the
`:alt` keyword. Here is an example:
https://github.com/Sharpie/homebrew-science/blob/for-science/scientific/hdf-eos2.rb#L24
Be aware that this aspect may undergo some changes due to dependency/option
upgrades that Adam is currently working on.
Then, people can use `brew-tap` to access your repository of ROS-related
formulae:
1. Go to the Homebrew prefix using "cd `brew --prefix`"
2. Check out a new branch into which `brew tap` will be merged using `git
checkout -b brew-tap` so that the history of master stays clean.
3. Add `brew-tap` into this new branch by running `brew pull
https://github.com/mxcl/homebrew/pull/6086`
4. Add the ROS repository via `brew tap add <GitHub username that forked
Homebrew-alt>`
5. Install stuff from ROS via `brew tap install <formula> [options]` or
for cases where Homebrew already has a formula: `brew tap install <GitHub
username that forked Homebrew-alt>/<formula> [options]`
6. Keep the `brew-tap` branch up to date with changes to Homebrew master
by `git checkout brew-tap && git rebase master`
That should simplify to just steps 4 and 5 once `brew tap` is integrated
into Homebrew.
As for custom bottles---you can bottle whatever you like and host it
wherever you like---it is just for formula in mxcl/master that the bottle
has to be hosted by us at SourceForge so that it can't be tampered with.
-Charlie
Re: [homebrew] Howto create/distribute bottles?
- From:
- William Woodall
- Date:
- 2011-08-25 @ 20:47
Thanks for the detailed response Charlie, it is greatly appreciated, I am
really looking forward to trying this out.
I'm sure this isn't the last question I'll have!
Thanks again,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
William Woodall
Graduate Software Engineering
Auburn University
w@auburn.edu
wjwwood@gmail.com
williamjwoodall.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Thu, Aug 25, 2011 at 3:35 PM, Charlie Sharpsteen <chuck@sharpsteen.net>wrote:
> On Thu, Aug 25, 2011 at 12:57 PM, William Woodall <wjwwood@gmail.com>wrote:
>
>> Great that was what I was looking for.
>>
>> But, let me ask another question. I am working on getting the ROS (Robot
>> Operating System) working better on OS X and I have moved to using Homebrew
>> from Macports for many reasons, but what I am interesting in is using the
>> bottle system to package some of the larger parts of ROS to simplify and
>> expedite installation. Specifically, OGRE, rViz, opencv, and a few other
>> components in ROS take more than 10 minutes to build on my i7 mbp w/ a solid
>> state drive. I would like to perhaps have a fork of Homebrew specifically
>> for ROS with separate hosting (probably Willowgarage or source forge) for
>> the bottles. What is the best path to this? Is there (should there be) a
>> mechanism for having "overlays" or third party Formula and Bottle
>> repositories which still uses the master Homebrew system from github but
>> overrides/extends the Formula and Bottle list? I am thinking specifically
>> of something similar to ppa's in apt or overlays in Gentoo's portage system.
>>
>> Thanks for everyone's help,
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> William Woodall
>>
>
> Sure. We're working on making it easier for people to manage and access
> external repositories that contain specialized formula. I wrote a tool
> called `brew tap` that is currently being re-worked and is on the list of
> "features to add" for the next release of Homebrew. Info on `brew-tap` can
> be found in this pull request:
>
> https://github.com/mxcl/homebrew/pull/6086
>
> The best way to get started with it is to:
>
> 1. Fork Adam's Homebrew-alt repository,
> https://github.com/adamv/homebrew-alt, as `brew-tap` uses the fork
> network of that repository to generate its list of available external
> repositories.
> 2. If you want to start from scratch, clean out the master branch of
> your Homebrew-alt fork by `git reset --hard aa06758` followed by a `git push
> -f origin master`
> 3. Commit your customized formulae to the Homebrew-alt fork---organize
> however you want. Currently, the only rules are that formula files must end
> in `.rb` and that any file ending in `.rb` must be a valid formula (no odd
> ruby scripts floating around).
>
> Dependencies on formula in external repositories can be declared via the
> `:alt` keyword. Here is an example:
>
>
>
https://github.com/Sharpie/homebrew-science/blob/for-science/scientific/hdf-eos2.rb#L24
>
> Be aware that this aspect may undergo some changes due to dependency/option
> upgrades that Adam is currently working on.
>
>
> Then, people can use `brew-tap` to access your repository of ROS-related
> formulae:
>
> 1. Go to the Homebrew prefix using "cd `brew --prefix`"
> 2. Check out a new branch into which `brew tap` will be merged using
> `git checkout -b brew-tap` so that the history of master stays clean.
> 3. Add `brew-tap` into this new branch by running `brew pull
>
https://github.com/mxcl/homebrew/pull/6086`<https://github.com/mxcl/homebrew/pull/6086>
> 4. Add the ROS repository via `brew tap add <GitHub username that
> forked Homebrew-alt>`
> 5. Install stuff from ROS via `brew tap install <formula> [options]` or
> for cases where Homebrew already has a formula: `brew tap install <GitHub
> username that forked Homebrew-alt>/<formula> [options]`
> 6. Keep the `brew-tap` branch up to date with changes to Homebrew
> master by `git checkout brew-tap && git rebase master`
>
> That should simplify to just steps 4 and 5 once `brew tap` is integrated
> into Homebrew.
>
>
> As for custom bottles---you can bottle whatever you like and host it
> wherever you like---it is just for formula in mxcl/master that the bottle
> has to be hosted by us at SourceForge so that it can't be tampered with.
>
> -Charlie
>
Re: [homebrew] Howto create/distribute bottles?
- From:
- Mike McQuaid
- Date:
- 2011-08-25 @ 21:49
On 25 Aug 2011, at 21:47, William Woodall wrote:
> Thanks for the detailed response Charlie, it is greatly appreciated, I
am really looking forward to trying this out.
Great description Charlie. Perhaps work sticking your email almost as-is
on the wiki?
William, I wrote the bottle support so if you find it lacking in any areas
you'd like addressed then let me know.
--
Mike McQuaid
http://mikemcquaid.com