librelist archives

Request for Comments: Adding support for mirrors

From:

Charlie Sharpsteen

Date:

2011-09-09 @ 15:36

Hey All,

This post concerns adding mirror support to Homebrew so that formula fetches
can have fallback options if the primary download site is unreachable. Off
the top of my head I can think of two solutions:

Extend `url` into a list. During fetch, if the download strategy fails using
the first URL in the list brew will iterate through the list until the
download strategy succeeds or all URLs liste fail. On the surface this seems
like the cleanest implementation, but the problem is that `url` is used for
much much more than specifying the download site---the `url` field is also
affects things like formula naming and auto-detection of version numbers.
All of those functions would need to be made aware of any changes to the
`url` implementation or bugs will appear.

Create an optional attribute called `mirrors` that contains a list of
alternate download URLs. In this case, only the formula fetch method needs
to be updated so that it iterates through `mirrors`, if available, when
`url` fails. Everything else could continue using `url` as usual.


So, wot do you guys and gals think? Should we do this? Is there a better way
to solve this problem?

-Charlie

On Fri, Sep 9, 2011 at 10:36 AM, Charlie Sharpsteen
<chuck@sharpsteen.net> wrote:
> Extend `url` into a list.

So is the idea here url() could be called multiple times in a formula,
a la depends_on()? (Let me use () to denote a function for clarity.)

> Create an optional attribute called `mirrors` that contains a list of
> alternate download URLs. In this case, only the formula fetch method needs
> to be updated so that it iterates through `mirrors`, if available, when
> `url` fails. Everything else could continue using `url` as usual.

I think this is cleaner. At least, it makes the formula DSL prettier.
A formula could look, in part, like this:

    url '...'
    mirror '...'
    mirror '...'
    version '...'
    md5 '...'

where mirror() pushes a URL onto an array of URLs.

> So, wot do you guys and gals think? Should we do this? Is there a better way
> to solve this problem?

In light of the recent kernel.org outage, I think this is worth doing
now, even if it ends up being kind of hackish and a better
implementation doesn't arrive until 2.0.

Jack

On Sat, Sep 10, 2011 at 7:20 PM, Jack Nagel <jacknagel@gmail.com> wrote:

> On Fri, Sep 9, 2011 at 10:36 AM, Charlie Sharpsteen
> > Create an optional attribute called `mirrors` that contains a list of
> > alternate download URLs. In this case, only the formula fetch method
> needs
> > to be updated so that it iterates through `mirrors`, if available, when
> > `url` fails. Everything else could continue using `url` as usual.
>
> I think this is cleaner. At least, it makes the formula DSL prettier.
> A formula could look, in part, like this:
>
>    url '...'
>    mirror '...'
>    mirror '...'
>    version '...'
>    md5 '...'
>
> where mirror() pushes a URL onto an array of URLs.


Originally I was going to go for a straight array like `def patches`, but
this sounds better to me. Preliminary implementation up for review at:

  https://github.com/mxcl/homebrew/pull/7574

 -Charlie

On 9 Sep 2011, at 16:36, Charlie Sharpsteen wrote:

> So, wot do you guys and gals think? Should we do this? Is there a better
way to solve this problem?

Self-hosting is another solution otherwise I like mirrors as long as you 
don't need to manually update the filenames for each one.

--
Mike McQuaid
http://mikemcquaid.com

+1, generally.

I think the later solution isn't as clean but provides better backwards
compatibility.  Then again you could probably implement the first solution
in a fashion that won't break current Formulas.

--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
William Woodall
Graduate Software Engineering
Auburn University
w@auburn.edu
wjwwood@gmail.com
williamjwoodall.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



On Fri, Sep 9, 2011 at 10:58 AM, Mike McQuaid <mike@mikemcquaid.com> wrote:

>
> On 9 Sep 2011, at 16:36, Charlie Sharpsteen wrote:
>
> > So, wot do you guys and gals think? Should we do this? Is there a better
> way to solve this problem?
>
> Self-hosting is another solution otherwise I like mirrors as long as you
> don't need to manually update the filenames for each one.
>
> --
> Mike McQuaid
> http://mikemcquaid.com
>
>

On Fri, Sep 9, 2011 at 11:22 AM, William Woodall <wjwwood@gmail.com> wrote:

> +1, generally.
>
> I think the later solution isn't as clean but provides better backwards
> compatibility.  Then again you could probably implement the first solution
> in a fashion that won't break current Formulas.


Theoretically possible, but if I write the code I guarantee the probability
of my nailing all the corner cases on the first try is within a machine
epsilon of 0.

Max has started design work for Homebrew2, so a clean solution could
eventually be implemented in that project.

-Charlie