librelist archives

« back to archive

Re: [leiningen] Lucene 3.6.0 dependency conflict for plugins

Re: [leiningen] Lucene 3.6.0 dependency conflict for plugins

From:
Phil Hagelberg
Date:
2013-08-27 @ 16:30
James Aley writes:

> Leiningen has an indirect dependency on Lucene 3.6.0 through maven-indexer.
> When my code attempts to use anything from the org.apache.lucene.document.*
> package, I get a VerifyError.

Ideally we could upgrade maven-indexer to get a newer version of Lucene,
but I don't know know if a newer version that does this is out.

Depending on the nature of the differences between Lucene 3 and 4, (if
the backwards-incompatibilities are minor and don't affect the subset of
Lucene that maven-indexer uses) it might be possible to bump the version
in Leiningen itself. This would mean your plugin would not work with
older Leiningen versions though.

> What's the best way to work around this? As far as I know, there's not a
> way to put in exclusions for Leiningen's dependencies itself, and even if
> there was, that may presumably break something?

Yeah, for boot speed reasons the whole Leiningen uberjar is on the
bootclasspath, which means it can't be shadowed by other classes in the
same JVM, so even a separate classloader won't work. If you need to be
isolated from Lucene 3, you'd need to spin off a subprocess. This can be
done without too much hassle using `leiningen.core.eval/eval-in` with a
dummy project map, but it will invoke a time penalty for the new JVM.

-Phil

Re: [leiningen] Lucene 3.6.0 dependency conflict for plugins

From:
James Aley
Date:
2013-08-27 @ 17:03
Hmm... so I checked later versions of maven-indexer, and unfortunately even
the latest version only uses Lucene 3.6.2. The API changed quite
substantially from 3 to 4, so a lot of projects are still on Lucene 3.

My end goal here is to get Leiningen to tell me the project dependencies,
so that I can launch a little web app serving API docs for the user's
project and all of its dependencies. Kind of like Python's pydoc server
thing, except just for one project at a time, not the whole "site" folder
(not that we have an equivalent anyway), and with a good search! Search is
important, hence Lucene :-)

I guess I should be able to spin the whole thing off as a sub-process once
I have the dependency information for the current project from Leiningen?
How will this eval-in call work? Would I effectively need to run my web
server from its jar and pass in the deps info as command line args, or is
there some nicer way? Perhaps there's an example in one of the
tasks/plugins already using it I can follow?

(For reference, my code's here: https://github.com/jaley/cloc - but it
sounds like you get the picture anyway.)

Thanks for the help!


On Tue, Aug 27, 2013 at 5:30 PM, Phil Hagelberg <phil@hagelb.org> wrote:

>
> James Aley writes:
>
> > Leiningen has an indirect dependency on Lucene 3.6.0 through
> maven-indexer.
> > When my code attempts to use anything from the
> org.apache.lucene.document.*
> > package, I get a VerifyError.
>
> Ideally we could upgrade maven-indexer to get a newer version of Lucene,
> but I don't know know if a newer version that does this is out.
>
> Depending on the nature of the differences between Lucene 3 and 4, (if
> the backwards-incompatibilities are minor and don't affect the subset of
> Lucene that maven-indexer uses) it might be possible to bump the version
> in Leiningen itself. This would mean your plugin would not work with
> older Leiningen versions though.
>
> > What's the best way to work around this? As far as I know, there's not a
> > way to put in exclusions for Leiningen's dependencies itself, and even if
> > there was, that may presumably break something?
>
> Yeah, for boot speed reasons the whole Leiningen uberjar is on the
> bootclasspath, which means it can't be shadowed by other classes in the
> same JVM, so even a separate classloader won't work. If you need to be
> isolated from Lucene 3, you'd need to spin off a subprocess. This can be
> done without too much hassle using `leiningen.core.eval/eval-in` with a
> dummy project map, but it will invoke a time penalty for the new JVM.
>
> -Phil
>

Re: [leiningen] Lucene 3.6.0 dependency conflict for plugins

From:
Phil Hagelberg
Date:
2013-08-27 @ 23:21
James Aley writes:

> I guess I should be able to spin the whole thing off as a sub-process once
> I have the dependency information for the current project from Leiningen?
> How will this eval-in call work?

Something like this:

    (ns leiningen.cloc
      (:require [leiningen.core.eval :as eval]))

    (def dummy-project {:dependencies [[cloc "..."]]})

    (defn doc-server [project port]
      (eval/eval-in project
        `(jetty/start-server (partial #'cloc.web/app ~(:dependencies 
project)) ~port)
        '(require 'cloc.web)))

This is one of those rare cases that does justify a separate Leiningen plugin
instead of just a `lein run` invocation because the :dependencies vector
needs to get passed into the code running inside the project.

You might need a few more keys in dummy-project; I'm not sure what the
minimum requirement to get eval-in-project to work is.

-Phil