librelist archives

« back to archive

Release task

Release task

From:
Phil Hagelberg
Date:
2014-05-01 @ 16:45
Hello Leiningeners.

People have been asking why there hasn't been a Leiningen release in a
while, which is a good question. Mostly it's because Leiningen has been
pretty stable, and there haven't been a lot of pressing issues. But part
of it has been that we wanted version 2.4.0 to include a `lein release`
task which could automate release processes which are now done by hand.

    https://github.com/technomancy/leiningen/issues/1389

A typical release process consists of a number of steps:

* bump version number to release
* tag in scm
* deploy to clojars (which performs compilation and jarring)
* bump version number up to the next snapshot
* push to scm

Obviously this needs to be decomposed into a number of other tasks. My
plan is to make it so :release-tasks follows the same general model as
:prep-tasks; it should be a vector of strings or string vectors
indicating which tasks to run with which arguments.

The main blocker was that I wanted to implement the version bumps in
terms of a generalized project.clj-rewriting mechanism that could handle
any kinds of changes to the project map without blowing away
comments. It turns out this is a lot harder than it sounds, especially
if you want to save changes coming from the update-in task to disk.

    https://github.com/technomancy/leiningen/issues/1388

Anyway, I think the best way forward is to include a `lein change` task
that hopefully in the future can support making those kinds of changes,
but for the time being is simply hard-coded only to be able to change
the :version key. Hopefully at some future point this task could be
expanded to support generalized project.clj changes, but for the time
being it shouldn't block the release of 2.4.0.

So in addition to the simplified `change` task, we'll need an scm task
to handle tagging and pushing. Obviously we'll start with git support,
but it'll need to be extensible with a multimethod to support other SCM
systems.

Once we have those pieces, writing the release task itself should be
pretty easy. But I thought I'd throw that out there for review if anyone
has comments or would like to tackle implementing a specific part of
that to help us get out the door more quickly.

thanks,
Phil

Re: [leiningen] Release task

From:
Phil Hagelberg
Date:
2014-06-07 @ 02:24
After getting the release functionality out and in the hands of a few
testers we found a few issues, the last of which I just closed. I think
current master looks good, and I hope to push a release early next
week. Please give it a try in the mean time and open issues for any
problems you may run into.

Thanks!

-Phil

Re: Release task

From:
Paul Stadig
Date:
2014-05-28 @ 13:29
Thanks to everyone who's done the work on the release task! I'm excited
about it as right now at work we have a collection of copy-and-pasted
scripts across our projects.

In order to make releases clean and reproducible, I'm wondering whether it
should also fail to release if there are any checkouts?


Paul

Re: [leiningen] Re: Release task

From:
Phil Hagelberg
Date:
2014-05-28 @ 15:25
Paul Stadig <paul@stadig.name> writes:

> In order to make releases clean and reproducible, I'm wondering whether it
> should also fail to release if there are any checkouts?

The release task currently invokes `lein vcs assert-committed`. Arguably
the presence of checkouts should count as uncommitted changes, so I
would be fine with that. We could proceed if the checkouts are on a
revision tagged with the dependency name and have no uncommitted
changes, but it's probably not worth doing at this time.

-Phil

Re: [leiningen] Re: Release task

From:
Hugo Duncan
Date:
2014-05-28 @ 21:52

phil@hagelb.org writes:

> Paul Stadig <paul@stadig.name> writes:
>
>> In order to make releases clean and reproducible, I'm wondering whether it
>> should also fail to release if there are any checkouts?
>
> The release task currently invokes `lein vcs assert-committed`. Arguably
> the presence of checkouts should count as uncommitted changes, so I
> would be fine with that.

We currently use a :no-checkouts profile for this, overriding the
:checkouts-deps-shares, which seems to be a more orthogonal way of
ensuring the release is properly tested.

Hugo

Re: [leiningen] Re: Release task

From:
Phil Hagelberg
Date:
2014-05-28 @ 22:07
Hugo Duncan <hugo@hugoduncan.org> writes:

>> The release task currently invokes `lein vcs assert-committed`. Arguably
>> the presence of checkouts should count as uncommitted changes, so I
>> would be fine with that.
>
> We currently use a :no-checkouts profile for this, overriding the
> :checkouts-deps-shares, which seems to be a more orthogonal way of
> ensuring the release is properly tested.

Do you mean that you run the tests the :no-checkouts profile applied?

I think that's definitely a good idea that we should encourage, but as
it is currently the release task doesn't bring testing into the picture
at all. Perhaps that should change; if we added ["test"] to
`:releases-tasks` it might help prevent people from pushing broken code.

-Phil

Re: [leiningen] Re: Release task

From:
Alex Ott
Date:
2014-05-28 @ 20:35
Hi Phil

How the release task could be adapted to the lein sub (
https://github.com/kumarshantanu/lein-sub), or something like? For example,
Incanter consists of several sub modules, and now we're using the scripts
to to deploy all modules, but the tagging, changing the version, etc. is
performed manually


On Wed, May 28, 2014 at 5:25 PM, Phil Hagelberg <phil@hagelb.org> wrote:

>
> Paul Stadig <paul@stadig.name> writes:
>
> > In order to make releases clean and reproducible, I'm wondering whether
> it
> > should also fail to release if there are any checkouts?
>
> The release task currently invokes `lein vcs assert-committed`. Arguably
> the presence of checkouts should count as uncommitted changes, so I
> would be fine with that. We could proceed if the checkouts are on a
> revision tagged with the dependency name and have no uncommitted
> changes, but it's probably not worth doing at this time.
>
> -Phil
>



-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott

Re: [leiningen] Re: Release task

From:
Phil Hagelberg
Date:
2014-05-28 @ 21:33
Alex Ott <alexott@gmail.com> writes:

> How the release task could be adapted to the lein sub (
> https://github.com/kumarshantanu/lein-sub), or something like? For example,
> Incanter consists of several sub modules, and now we're using the scripts
> to to deploy all modules, but the tagging, changing the version, etc. is
> performed manually

Yeah, I don't see any reason why you couldn't use lein-sub to make `lein
release` on one project trigger a release on a number of
subprojects. You should be able to change `:release-tasks` to include
calls to the `sub` task.

:release-tasks [["vcs" "assert-committed"]
                ["change" "version" "leiningen.release/bump-version" "release"]
                ["vcs" "commit"]
                ["vcs" "tag"]
                ["sub" "release"] ; or something
                ["change" "version" "leiningen.release/bump-version"]
                ["vcs" "commit"]
                ["vcs" "push"]]

It really depends on what the relationship between the top-level project
and the others are and whether there are any top-level artifacts.

-Phil

Re: [leiningen] Re: Release task

From:
Jim Crossley
Date:
2014-05-29 @ 00:47
I may be confused, but wouldn't it be as simple as running 'lein sub
release'? I would expect each subproject to use the default :release-tasks.

Jim

On Wed, May 28, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:

>
> Alex Ott <alexott@gmail.com> writes:
>
> > How the release task could be adapted to the lein sub (
> > https://github.com/kumarshantanu/lein-sub), or something like? For
> example,
> > Incanter consists of several sub modules, and now we're using the scripts
> > to to deploy all modules, but the tagging, changing the version, etc. is
> > performed manually
>
> Yeah, I don't see any reason why you couldn't use lein-sub to make `lein
> release` on one project trigger a release on a number of
> subprojects. You should be able to change `:release-tasks` to include
> calls to the `sub` task.
>
> :release-tasks [["vcs" "assert-committed"]
>                 ["change" "version" "leiningen.release/bump-version"
> "release"]
>                 ["vcs" "commit"]
>                 ["vcs" "tag"]
>                 ["sub" "release"] ; or something
>                 ["change" "version" "leiningen.release/bump-version"]
>                 ["vcs" "commit"]
>                 ["vcs" "push"]]
>
> It really depends on what the relationship between the top-level project
> and the others are and whether there are any top-level artifacts.
>
> -Phil
>

Re: [leiningen] Re: Release task

From:
Jim Crossley
Date:
2014-05-29 @ 01:07
I'm sorry, Alex, I was in fact, confused. I interpreted your question to
imply you were already using lein-sub. After looking at the source, I see
the shell scripts. I feel like I could significantly simplify your build
with lein-modules. Are you open to that? :)


On Wed, May 28, 2014 at 8:47 PM, Jim Crossley <jim@crossleys.org> wrote:

> I may be confused, but wouldn't it be as simple as running 'lein sub
> release'? I would expect each subproject to use the default :release-tasks.
>
> Jim
>
> On Wed, May 28, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>
>>
>> Alex Ott <alexott@gmail.com> writes:
>>
>> > How the release task could be adapted to the lein sub (
>> > https://github.com/kumarshantanu/lein-sub), or something like? For
>> example,
>> > Incanter consists of several sub modules, and now we're using the
>> scripts
>> > to to deploy all modules, but the tagging, changing the version, etc. is
>> > performed manually
>>
>> Yeah, I don't see any reason why you couldn't use lein-sub to make `lein
>> release` on one project trigger a release on a number of
>> subprojects. You should be able to change `:release-tasks` to include
>> calls to the `sub` task.
>>
>> :release-tasks [["vcs" "assert-committed"]
>>                 ["change" "version" "leiningen.release/bump-version"
>> "release"]
>>                 ["vcs" "commit"]
>>                 ["vcs" "tag"]
>>                 ["sub" "release"] ; or something
>>                 ["change" "version" "leiningen.release/bump-version"]
>>                 ["vcs" "commit"]
>>                 ["vcs" "push"]]
>>
>> It really depends on what the relationship between the top-level project
>> and the others are and whether there are any top-level artifacts.
>>
>> -Phil
>>
>
>

Re: [leiningen] Re: Release task

From:
Alex Ott
Date:
2014-05-29 @ 08:29
Hello Jim

we have a pull request for incorporation of lein-sub, I hope that it will
be merged soon...


On Thu, May 29, 2014 at 3:07 AM, Jim Crossley <jim@crossleys.org> wrote:

> I'm sorry, Alex, I was in fact, confused. I interpreted your question to
> imply you were already using lein-sub. After looking at the source, I see
> the shell scripts. I feel like I could significantly simplify your build
> with lein-modules. Are you open to that? :)
>
>
> On Wed, May 28, 2014 at 8:47 PM, Jim Crossley <jim@crossleys.org> wrote:
>
>> I may be confused, but wouldn't it be as simple as running 'lein sub
>> release'? I would expect each subproject to use the default :release-tasks.
>>
>> Jim
>>
>> On Wed, May 28, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>>
>>>
>>> Alex Ott <alexott@gmail.com> writes:
>>>
>>> > How the release task could be adapted to the lein sub (
>>> > https://github.com/kumarshantanu/lein-sub), or something like? For
>>> example,
>>> > Incanter consists of several sub modules, and now we're using the
>>> scripts
>>> > to to deploy all modules, but the tagging, changing the version, etc.
>>> is
>>> > performed manually
>>>
>>> Yeah, I don't see any reason why you couldn't use lein-sub to make `lein
>>> release` on one project trigger a release on a number of
>>> subprojects. You should be able to change `:release-tasks` to include
>>> calls to the `sub` task.
>>>
>>> :release-tasks [["vcs" "assert-committed"]
>>>                 ["change" "version" "leiningen.release/bump-version"
>>> "release"]
>>>                 ["vcs" "commit"]
>>>                 ["vcs" "tag"]
>>>                 ["sub" "release"] ; or something
>>>                 ["change" "version" "leiningen.release/bump-version"]
>>>                 ["vcs" "commit"]
>>>                 ["vcs" "push"]]
>>>
>>> It really depends on what the relationship between the top-level project
>>> and the others are and whether there are any top-level artifacts.
>>>
>>> -Phil
>>>
>>
>>
>


-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott

Re: [leiningen] Re: Release task

From:
Jim Crossley
Date:
2014-05-29 @ 22:03
Hi Alex,

I took a look at the PR. Essentially, it adds this to your root project.clj:

  :sub ["modules/incanter-charts"
        "modules/incanter-core"
        "modules/incanter-excel"
        "modules/incanter-io"
        "modules/incanter-latex"
        "modules/incanter-mongodb"
        "modules/incanter-pdf"
        "modules/incanter-sql"
        "modules/incanter-svg"
        "modules/incanter-zoo"
        ]
  :plugins [[lein-sub "0.3.0"]]

There are a couple of problems with that, though. The first is the lack of
your root project in the list, so 'lein sub install' won't build the root
as script/install will. That's easily fixed by adding "." to the list,
though.

The bigger problem is the order of the list, as lein-sub builds the modules
in the order you specify. So 'lein sub install' will fail if the
interdependent modules aren't already in  ~/.m2/repository/incanter or
available from clojars, i.e. incanter-charts requires incanter-io which
requires incanter-core. It's up to you to give lein-sub the list in
dependency order.

The lein-modules plugin provides a superset of lein-sub's features. Here's
its config, equivalent to the above in your root project.clj:

  :modules {:dirs ["."
                   "modules/incanter-charts"
                   "modules/incanter-core"
                   "modules/incanter-excel"
                   "modules/incanter-io"
                   "modules/incanter-latex"
                   "modules/incanter-mongodb"
                   "modules/incanter-pdf"
                   "modules/incanter-sql"
                   "modules/incanter-svg"
                   "modules/incanter-zoo"
                   ]
            :subprocess false}
  :plugins [[lein-modules "0.3.3"]]

But lein-modules will build the modules in dependency order.

Ideally, you wouldn't specify :dirs at all, since the modules will be
discovered automatically if they reside in immediate child directories.
Yours don't, unfortunately, so for discovery to work, you'd need the
following in each of your projects beneath modules/:

  :modules {:parent "../.."}}
  :plugins [[lein-modules "0.3.3"]]

This way, adding/removing modules doesn't require me to have to keep
updating :dirs in the parent project.clj.

Plus, adding the lein-modules plugin to each child exposes you to
additional features, specifically version management and project
inheritance. Right now, you have versions for org.clojure and the various
incanter modules littered throughout your project.clj files. With
lein-modules, you only have to maintain them in a single place, e.g. your
root project.clj:

  :modules {:versions {clojure "1.5.1", incanter "1.5.6-SNAPSHOT"}}

And all the redundant boilerplate stuff in each module could be kept in the
root project.clj, too:

  :modules {:inherited
            {:url "http://incanter.org/"
             :license {:name "Eclipse Public License"
                       :url "http://www.eclipse.org/legal/epl-v10.html"}
             :scm {:name "git"
                   :url "https://github.com/incanter/incanter"}
             :min-lein-version "2.0.0"}}

Anyway, probably way more info than you wanted, but lein-modules was
written to address projects exactly like Incanter. I'm happy to help you
migrate if it's appealing to you at all.

Jim


On Thu, May 29, 2014 at 4:29 AM, Alex Ott <alexott@gmail.com> wrote:

> Hello Jim
>
> we have a pull request for incorporation of lein-sub, I hope that it will
> be merged soon...
>
>
> On Thu, May 29, 2014 at 3:07 AM, Jim Crossley <jim@crossleys.org> wrote:
>
>> I'm sorry, Alex, I was in fact, confused. I interpreted your question to
>> imply you were already using lein-sub. After looking at the source, I see
>> the shell scripts. I feel like I could significantly simplify your build
>> with lein-modules. Are you open to that? :)
>>
>>
>> On Wed, May 28, 2014 at 8:47 PM, Jim Crossley <jim@crossleys.org> wrote:
>>
>>> I may be confused, but wouldn't it be as simple as running 'lein sub
>>> release'? I would expect each subproject to use the default :release-tasks.
>>>
>>> Jim
>>>
>>> On Wed, May 28, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>>>
>>>>
>>>> Alex Ott <alexott@gmail.com> writes:
>>>>
>>>> > How the release task could be adapted to the lein sub (
>>>> > https://github.com/kumarshantanu/lein-sub), or something like? For
>>>> example,
>>>> > Incanter consists of several sub modules, and now we're using the
>>>> scripts
>>>> > to to deploy all modules, but the tagging, changing the version, etc.
>>>> is
>>>> > performed manually
>>>>
>>>> Yeah, I don't see any reason why you couldn't use lein-sub to make `lein
>>>> release` on one project trigger a release on a number of
>>>> subprojects. You should be able to change `:release-tasks` to include
>>>> calls to the `sub` task.
>>>>
>>>> :release-tasks [["vcs" "assert-committed"]
>>>>                 ["change" "version" "leiningen.release/bump-version"
>>>> "release"]
>>>>                 ["vcs" "commit"]
>>>>                 ["vcs" "tag"]
>>>>                 ["sub" "release"] ; or something
>>>>                 ["change" "version" "leiningen.release/bump-version"]
>>>>                 ["vcs" "commit"]
>>>>                 ["vcs" "push"]]
>>>>
>>>> It really depends on what the relationship between the top-level project
>>>> and the others are and whether there are any top-level artifacts.
>>>>
>>>> -Phil
>>>>
>>>
>>>
>>
>
>
> --
> With best wishes,                    Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
> Skype: alex.ott
>

Re: [leiningen] Re: Release task

From:
Alex Ott
Date:
2014-05-30 @ 13:34
Thank you Jim!

I'll look to both options - I really already though about adopting
lein-modules...


On Fri, May 30, 2014 at 12:03 AM, Jim Crossley <jim@crossleys.org> wrote:

> Hi Alex,
>
> I took a look at the PR. Essentially, it adds this to your root
> project.clj:
>
>   :sub ["modules/incanter-charts"
>         "modules/incanter-core"
>         "modules/incanter-excel"
>         "modules/incanter-io"
>         "modules/incanter-latex"
>         "modules/incanter-mongodb"
>         "modules/incanter-pdf"
>         "modules/incanter-sql"
>         "modules/incanter-svg"
>         "modules/incanter-zoo"
>         ]
>   :plugins [[lein-sub "0.3.0"]]
>
> There are a couple of problems with that, though. The first is the lack of
> your root project in the list, so 'lein sub install' won't build the root
> as script/install will. That's easily fixed by adding "." to the list,
> though.
>
> The bigger problem is the order of the list, as lein-sub builds the
> modules in the order you specify. So 'lein sub install' will fail if the
> interdependent modules aren't already in  ~/.m2/repository/incanter or
> available from clojars, i.e. incanter-charts requires incanter-io which
> requires incanter-core. It's up to you to give lein-sub the list in
> dependency order.
>
> The lein-modules plugin provides a superset of lein-sub's features. Here's
> its config, equivalent to the above in your root project.clj:
>
>   :modules {:dirs ["."
>                    "modules/incanter-charts"
>                    "modules/incanter-core"
>                    "modules/incanter-excel"
>                    "modules/incanter-io"
>                    "modules/incanter-latex"
>                    "modules/incanter-mongodb"
>                    "modules/incanter-pdf"
>                    "modules/incanter-sql"
>                    "modules/incanter-svg"
>                    "modules/incanter-zoo"
>                    ]
>             :subprocess false}
>   :plugins [[lein-modules "0.3.3"]]
>
> But lein-modules will build the modules in dependency order.
>
> Ideally, you wouldn't specify :dirs at all, since the modules will be
> discovered automatically if they reside in immediate child directories.
> Yours don't, unfortunately, so for discovery to work, you'd need the
> following in each of your projects beneath modules/:
>
>   :modules {:parent "../.."}}
>   :plugins [[lein-modules "0.3.3"]]
>
> This way, adding/removing modules doesn't require me to have to keep
> updating :dirs in the parent project.clj.
>
> Plus, adding the lein-modules plugin to each child exposes you to
> additional features, specifically version management and project
> inheritance. Right now, you have versions for org.clojure and the various
> incanter modules littered throughout your project.clj files. With
> lein-modules, you only have to maintain them in a single place, e.g. your
> root project.clj:
>
>   :modules {:versions {clojure "1.5.1", incanter "1.5.6-SNAPSHOT"}}
>
> And all the redundant boilerplate stuff in each module could be kept in
> the root project.clj, too:
>
>   :modules {:inherited
>             {:url "http://incanter.org/"
>              :license {:name "Eclipse Public License"
>                        :url "http://www.eclipse.org/legal/epl-v10.html"}
>              :scm {:name "git"
>                    :url "https://github.com/incanter/incanter"}
>              :min-lein-version "2.0.0"}}
>
> Anyway, probably way more info than you wanted, but lein-modules was
> written to address projects exactly like Incanter. I'm happy to help you
> migrate if it's appealing to you at all.
>
> Jim
>
>
> On Thu, May 29, 2014 at 4:29 AM, Alex Ott <alexott@gmail.com> wrote:
>
>> Hello Jim
>>
>> we have a pull request for incorporation of lein-sub, I hope that it will
>> be merged soon...
>>
>>
>> On Thu, May 29, 2014 at 3:07 AM, Jim Crossley <jim@crossleys.org> wrote:
>>
>>> I'm sorry, Alex, I was in fact, confused. I interpreted your question to
>>> imply you were already using lein-sub. After looking at the source, I see
>>> the shell scripts. I feel like I could significantly simplify your build
>>> with lein-modules. Are you open to that? :)
>>>
>>>
>>> On Wed, May 28, 2014 at 8:47 PM, Jim Crossley <jim@crossleys.org> wrote:
>>>
>>>> I may be confused, but wouldn't it be as simple as running 'lein sub
>>>> release'? I would expect each subproject to use the default :release-tasks.
>>>>
>>>> Jim
>>>>
>>>> On Wed, May 28, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org>
>>>> wrote:
>>>>
>>>>>
>>>>> Alex Ott <alexott@gmail.com> writes:
>>>>>
>>>>> > How the release task could be adapted to the lein sub (
>>>>> > https://github.com/kumarshantanu/lein-sub), or something like? For
>>>>> example,
>>>>> > Incanter consists of several sub modules, and now we're using the
>>>>> scripts
>>>>> > to to deploy all modules, but the tagging, changing the version,
>>>>> etc. is
>>>>> > performed manually
>>>>>
>>>>> Yeah, I don't see any reason why you couldn't use lein-sub to make
>>>>> `lein
>>>>> release` on one project trigger a release on a number of
>>>>> subprojects. You should be able to change `:release-tasks` to include
>>>>> calls to the `sub` task.
>>>>>
>>>>> :release-tasks [["vcs" "assert-committed"]
>>>>>                 ["change" "version" "leiningen.release/bump-version"
>>>>> "release"]
>>>>>                 ["vcs" "commit"]
>>>>>                 ["vcs" "tag"]
>>>>>                 ["sub" "release"] ; or something
>>>>>                 ["change" "version" "leiningen.release/bump-version"]
>>>>>                 ["vcs" "commit"]
>>>>>                 ["vcs" "push"]]
>>>>>
>>>>> It really depends on what the relationship between the top-level
>>>>> project
>>>>> and the others are and whether there are any top-level artifacts.
>>>>>
>>>>> -Phil
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> With best wishes,                    Alex Ott
>> http://alexott.net/
>> Twitter: alexott_en (English), alexott (Russian)
>> Skype: alex.ott
>>
>
>


-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott

Re: [leiningen] Re: Release task

From:
Dave Tenny
Date:
2014-05-29 @ 11:42
Apologies, I haven't been following this too closely, and have only done
about three clojars deployments with lein, so I'll just ask while the
asking's good.

Will the new functionality ensure that I have bumped the version ID so that
I don't overwrite a version that's already out there?  Apologies if this is
done already, I'm still getting my lein/clojure/version management feet
under me and have had trouble accidentially smashing jars in my ~/.m2 with
newer versions bearing the same ID.  No fault of lein's, purely mine.  Just
want to make sure I don't smash clojars the same way.


On Thu, May 29, 2014 at 4:29 AM, Alex Ott <alexott@gmail.com> wrote:

> Hello Jim
>
> we have a pull request for incorporation of lein-sub, I hope that it will
> be merged soon...
>
>
> On Thu, May 29, 2014 at 3:07 AM, Jim Crossley <jim@crossleys.org> wrote:
>
>> I'm sorry, Alex, I was in fact, confused. I interpreted your question to
>> imply you were already using lein-sub. After looking at the source, I see
>> the shell scripts. I feel like I could significantly simplify your build
>> with lein-modules. Are you open to that? :)
>>
>>
>> On Wed, May 28, 2014 at 8:47 PM, Jim Crossley <jim@crossleys.org> wrote:
>>
>>> I may be confused, but wouldn't it be as simple as running 'lein sub
>>> release'? I would expect each subproject to use the default :release-tasks.
>>>
>>> Jim
>>>
>>> On Wed, May 28, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>>>
>>>>
>>>> Alex Ott <alexott@gmail.com> writes:
>>>>
>>>> > How the release task could be adapted to the lein sub (
>>>> > https://github.com/kumarshantanu/lein-sub), or something like? For
>>>> example,
>>>> > Incanter consists of several sub modules, and now we're using the
>>>> scripts
>>>> > to to deploy all modules, but the tagging, changing the version, etc.
>>>> is
>>>> > performed manually
>>>>
>>>> Yeah, I don't see any reason why you couldn't use lein-sub to make `lein
>>>> release` on one project trigger a release on a number of
>>>> subprojects. You should be able to change `:release-tasks` to include
>>>> calls to the `sub` task.
>>>>
>>>> :release-tasks [["vcs" "assert-committed"]
>>>>                 ["change" "version" "leiningen.release/bump-version"
>>>> "release"]
>>>>                 ["vcs" "commit"]
>>>>                 ["vcs" "tag"]
>>>>                 ["sub" "release"] ; or something
>>>>                 ["change" "version" "leiningen.release/bump-version"]
>>>>                 ["vcs" "commit"]
>>>>                 ["vcs" "push"]]
>>>>
>>>> It really depends on what the relationship between the top-level project
>>>> and the others are and whether there are any top-level artifacts.
>>>>
>>>> -Phil
>>>>
>>>
>>>
>>
>
>
> --
> With best wishes,                    Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
> Skype: alex.ott
>

Re: [leiningen] Re: Release task

From:
Phil Hagelberg
Date:
2014-05-29 @ 16:24
Dave Tenny <dave.tenny@gmail.com> writes:

> Will the new functionality ensure that I have bumped the version ID so that
> I don't overwrite a version that's already out there?

Yeah, the release task does this. However, even if you mix it up,
Clojars will still reject a deploy that attempts to overwrite an
existing version, so you should be safe in any case.

-Phil

Re: [leiningen] Re: Release task

From:
Jim Crossley
Date:
2014-05-30 @ 17:38
FYI, I was really confused about how to release a lein-sub (or
lein-modules) project. You can't just run 'lein sub release' because all
the submodules reside in the same repo, so you'd want to change all the
modules' project.clj files before commting/tagging. So instead of 'lein sub
release', you run 'lein release' and adjust :release tasks to prepend "sub"
(or "modules") to the "change" and "deploy" tasks, like so:

  :release-tasks
  [["vcs" "assert-committed"]
   ["sub" "change" "version" "leiningen.release/bump-version" "release"]
   ["vcs" "commit"]
   ["vcs" "tag"]
   ["sub" "deploy"]
   ["sub" "change" "version" "leiningen.release/bump-version"]
   ["vcs" "commit"]
   ["vcs" "push"]])

I just confirmed that works.

Hope it helps,
Jim



On Wed, May 28, 2014 at 8:47 PM, Jim Crossley <jim@crossleys.org> wrote:

> I may be confused, but wouldn't it be as simple as running 'lein sub
> release'? I would expect each subproject to use the default :release-tasks.
>
> Jim
>
> On Wed, May 28, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>
>>
>> Alex Ott <alexott@gmail.com> writes:
>>
>> > How the release task could be adapted to the lein sub (
>> > https://github.com/kumarshantanu/lein-sub), or something like? For
>> example,
>> > Incanter consists of several sub modules, and now we're using the
>> scripts
>> > to to deploy all modules, but the tagging, changing the version, etc. is
>> > performed manually
>>
>> Yeah, I don't see any reason why you couldn't use lein-sub to make `lein
>> release` on one project trigger a release on a number of
>> subprojects. You should be able to change `:release-tasks` to include
>> calls to the `sub` task.
>>
>> :release-tasks [["vcs" "assert-committed"]
>>                 ["change" "version" "leiningen.release/bump-version"
>> "release"]
>>                 ["vcs" "commit"]
>>                 ["vcs" "tag"]
>>                 ["sub" "release"] ; or something
>>                 ["change" "version" "leiningen.release/bump-version"]
>>                 ["vcs" "commit"]
>>                 ["vcs" "push"]]
>>
>> It really depends on what the relationship between the top-level project
>> and the others are and whether there are any top-level artifacts.
>>
>> -Phil
>>
>
>

Re: [leiningen] Re: Release task

From:
Phil Hagelberg
Date:
2014-05-30 @ 17:44
Jim Crossley <jim@crossleys.org> writes:

> So instead of 'lein sub release', you run 'lein release' and adjust
> :release tasks to prepend "sub" (or "modules") to the "change" and
> "deploy" tasks, like so:

Thanks for the example. This makes me think it'd be nice if loading
profiles from a plugin could be done in a first-class way without
middleware. Maybe if we namespaced the profiles it could work.

    $ lein with-profile +lein-modules/release release

Probably not something that would make it into 2.4.0, but this isn't the
first time I've come across a use case for such a thing.

-Phil

Re: [leiningen] Re: Release task

From:
Jim Crossley
Date:
2014-05-30 @ 18:07
That would be nice, although I personally kinda like seeing the release
steps explicitly defined in my top-level project.clj.

One thing I neglected to mention is the problem of interdependent
submodules, where one will have the other in its :dependencies, likely with
a -SNAPSHOT suffixed version matching its own. Ideally, these would be
modified by the "change" task, too. I know you've mentioned expanding that
task's capabilities in the future.

Jim


On Fri, May 30, 2014 at 1:44 PM, Phil Hagelberg <phil@hagelb.org> wrote:

>
> Jim Crossley <jim@crossleys.org> writes:
>
> > So instead of 'lein sub release', you run 'lein release' and adjust
> > :release tasks to prepend "sub" (or "modules") to the "change" and
> > "deploy" tasks, like so:
>
> Thanks for the example. This makes me think it'd be nice if loading
> profiles from a plugin could be done in a first-class way without
> middleware. Maybe if we namespaced the profiles it could work.
>
>     $ lein with-profile +lein-modules/release release
>
> Probably not something that would make it into 2.4.0, but this isn't the
> first time I've come across a use case for such a thing.
>
> -Phil
>

Re: [leiningen] Re: Release task

From:
Phil Hagelberg
Date:
2014-05-31 @ 04:26
Jim Crossley <jim@crossleys.org> writes:

> One thing I neglected to mention is the problem of interdependent
> submodules, where one will have the other in its :dependencies, likely with
> a -SNAPSHOT suffixed version matching its own. Ideally, these would be
> modified by the "change" task, too. I know you've mentioned expanding that
> task's capabilities in the future.

I would certainly like the change task to support more nuanced
modifications like dependency version changes, but I don't want it to
block the release. Maybe for 2.4.1. But it's easy to pass functions from
plugins to the change task too, so this could be perfected outside the
Leiningen release cycle.

-Phil

Re: [leiningen] Release task

From:
Phil Hagelberg
Date:
2014-05-27 @ 21:33
Phil Hagelberg <phil@hagelb.org> writes:

> But part of it has been that we wanted version 2.4.0 to include a
> `lein release` task which could automate release processes which are
> now done by hand.

Thanks to a flurry of patches for a bunch of new contributors, the
release task is landed and ready to go. The deploy guide has been
updated with the new functionality.

    
https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md#releasing-simplified

I'm really excited about how this has come together; it feels like a
really solid improvement. However, it would be great to get some
feedback on the features and docs before releasing 2.4.0.

Please pull from master and use the new release task if you've got any
libraries you'd like to release to Clojars. Let us know how it goes and
if the docs are unclear in any way. If all goes well we could release
2.4.0 in the next week or so.

thanks!
Phil

Re: [leiningen] Release task

From:
Jim Crossley
Date:
2014-05-29 @ 02:50
Phil, I was unable to get 'lein release' working with this, per the DEPLOY
instructions:

{:repositories [["clojars" {:creds :gpg}]]
 :deploy-repositories [["releases" "clojars"]]}

I would always get this:

No credentials found for releases (did you mean `lein deploy clojars`?)
Password prompts are not supported when ran after other (potentially)
interactive tasks. Maybe setting up credentials may be an idea?

The only thing I got to work was this:

:deploy-repositories [["releases" {:url "https://clojars.org/repo/" :creds
:gpg}]]

Since 'lein release' is going to run 'lein deploy' without passing a repo,
I think the docs should be clearer about how to make that case work.

Thanks,
Jim


On Tue, May 27, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:

>
> Phil Hagelberg <phil@hagelb.org> writes:
>
> > But part of it has been that we wanted version 2.4.0 to include a
> > `lein release` task which could automate release processes which are
> > now done by hand.
>
> Thanks to a flurry of patches for a bunch of new contributors, the
> release task is landed and ready to go. The deploy guide has been
> updated with the new functionality.
>
>
> 
https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md#releasing-simplified
>
> I'm really excited about how this has come together; it feels like a
> really solid improvement. However, it would be great to get some
> feedback on the features and docs before releasing 2.4.0.
>
> Please pull from master and use the new release task if you've got any
> libraries you'd like to release to Clojars. Let us know how it goes and
> if the docs are unclear in any way. If all goes well we could release
> 2.4.0 in the next week or so.
>
> thanks!
> Phil
>

Re: [leiningen] Release task

From:
Jake McCrary
Date:
2014-05-29 @ 04:05
I'm extremely excited for lein release. Today I managed to mostly use the
default :release-tasks to push out a new release. Ended up changing the
["deploy"] step to ["deploy" "private"] since that is the repository we are
setup to deploy to. From reading the Jim's previous message it actually
looks like I should be able to change the project specification to make
that slight modification no longer required.

Another modification that I needed to perform was actually in Leiningen's
source. Temporarily hacked leiningen.vcs/which-vcs to always return :git
because of the way the project is structured.

The project is structured similarly to lein-test-refresh (
https://github.com/jakemcc/lein-test-refresh). The VCS root contains
multiple subdirectories. One of those subdirectores (in the
lein-test-refresh case the test-refresh directory) is the project that
needs to be released. This type of project structure doesn't play nicely
with which-vcs.

Jake


On Wed, May 28, 2014 at 9:50 PM, Jim Crossley <jim@crossleys.org> wrote:

> Phil, I was unable to get 'lein release' working with this, per the DEPLOY
> instructions:
>
> {:repositories [["clojars" {:creds :gpg}]]
>  :deploy-repositories [["releases" "clojars"]]}
>
> I would always get this:
>
> No credentials found for releases (did you mean `lein deploy clojars`?)
> Password prompts are not supported when ran after other (potentially)
> interactive tasks. Maybe setting up credentials may be an idea?
>
> The only thing I got to work was this:
>
> :deploy-repositories [["releases" {:url "https://clojars.org/repo/"
> :creds :gpg}]]
>
> Since 'lein release' is going to run 'lein deploy' without passing a repo,
> I think the docs should be clearer about how to make that case work.
>
> Thanks,
> Jim
>
>
> On Tue, May 27, 2014 at 5:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>
>>
>> Phil Hagelberg <phil@hagelb.org> writes:
>>
>> > But part of it has been that we wanted version 2.4.0 to include a
>> > `lein release` task which could automate release processes which are
>> > now done by hand.
>>
>> Thanks to a flurry of patches for a bunch of new contributors, the
>> release task is landed and ready to go. The deploy guide has been
>> updated with the new functionality.
>>
>>
>> 
https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md#releasing-simplified
>>
>> I'm really excited about how this has come together; it feels like a
>> really solid improvement. However, it would be great to get some
>> feedback on the features and docs before releasing 2.4.0.
>>
>> Please pull from master and use the new release task if you've got any
>> libraries you'd like to release to Clojars. Let us know how it goes and
>> if the docs are unclear in any way. If all goes well we could release
>> 2.4.0 in the next week or so.
>>
>> thanks!
>> Phil
>>
>
>

Re: [leiningen] Release task

From:
Phil Hagelberg
Date:
2014-05-31 @ 04:30
Jake McCrary <jake@jakemccrary.com> writes:

> The project is structured similarly to lein-test-refresh (
> https://github.com/jakemcc/lein-test-refresh). The VCS root contains
> multiple subdirectories. One of those subdirectores (in the
> lein-test-refresh case the test-refresh directory) is the project that
> needs to be released. This type of project structure doesn't play nicely
> with which-vcs.

Yeah. I'm a little hesitant to simply traverse up .. until a .git
directory is found for this though, because in this case tagging the
whole repo with a version that comes from one of its subdirectories
doesn't always make sense. You would probably want the tag for that
specific subproject to be prefixed with something. Not sure of the best
way to handle that.

-Phil

Re: [leiningen] Release task

From:
Jake McCrary
Date:
2014-06-04 @ 00:01
Yeah, the default tagging has been an issue. I'd be happy with the project
name becoming part of the commits and tag messages. I can imagine this
might not be satisfactory for others.

I think `lein release` is built from small enough parts that even if some
of the tagging and committing needs to be done in a shell script for
projects that have an unusual layout this is still an exciting addition to
leiningen. Can compose the pieces together and still make releasing much
easier.

Jake


On Fri, May 30, 2014 at 11:30 PM, Phil Hagelberg <phil@hagelb.org> wrote:

>
> Jake McCrary <jake@jakemccrary.com> writes:
>
> > The project is structured similarly to lein-test-refresh (
> > https://github.com/jakemcc/lein-test-refresh). The VCS root contains
> > multiple subdirectories. One of those subdirectores (in the
> > lein-test-refresh case the test-refresh directory) is the project that
> > needs to be released. This type of project structure doesn't play nicely
> > with which-vcs.
>
> Yeah. I'm a little hesitant to simply traverse up .. until a .git
> directory is found for this though, because in this case tagging the
> whole repo with a version that comes from one of its subdirectories
> doesn't always make sense. You would probably want the tag for that
> specific subproject to be prefixed with something. Not sure of the best
> way to handle that.
>
> -Phil
>

Re: [leiningen] Release task

From:
Sean Corfield
Date:
2014-05-27 @ 22:15
On May 27, 2014, at 2:33 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>    
https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md#releasing-simplified

Nice!

> I'm really excited about how this has come together; it feels like a
> really solid improvement. However, it would be great to get some
> feedback on the features and docs before releasing 2.4.0.

The DEPLOY.md has 'clojure' before (defproject... in bullets 1 & 4.

bump-version is shown as taking a stringified keyword ":release" which 
seems potentially error prone - would accepting a plain string "release" 
or a plain keyword :release also work there?

I'm not clear on a couple of points:

1. Does this add vcs and change tasks to Leiningen as well as release?

2. What exactly do :alpha, :beta, and :rc do?

3. If you have 2.4.0-SNAPSHOT and you decide to release an initial alpha 
(2.4.0-alpha1 I assume), what would be the sequence of Leiningen commands 
to do that? I would have initially assumed `lein release :alpha` but the 
docs suggest the $LEVEL affects the post-deploy change of version only.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)


Re: [leiningen] Release task

From:
Phil Hagelberg
Date:
2014-05-27 @ 23:05
Sean Corfield <sean@corfield.org> writes:

> The DEPLOY.md has 'clojure' before (defproject... in bullets 1 & 4.

Good catch; thanks.

> bump-version is shown as taking a stringified keyword ":release" which
> seems potentially error prone - would accepting a plain string
> "release" or a plain keyword :release also work there?

The args in `:release-tasks` are specified exactly as how the shell
would tokenize them and pass them off to Leiningen, and we need to pass
them through the reader before using them. However, for programmatic
access there is another non-task function you can use where the args
aren't assumed to be strings.

However, I've added an explicit check for this so you get a nice clear
message rather than a stack trace if you get this wrong.

Another tactic would be to assume the reader has already been called for
cases where you get a collection that's not uniformly strings, but I
feel like that's probably too clever and makes for weird edge cases if
your args are all strings that need to remain as strings.

> 1. Does this add vcs and change tasks to Leiningen as well as release?

Yep, these can be used outside the context of the release task of
course, but they're lower-level; I'd expect them usually to be invoked
either from the release task or an alias chain. The docstrings for these
tasks should be self-explanatory.

> 3. If you have 2.4.0-SNAPSHOT and you decide to release an initial
> alpha (2.4.0-alpha1 I assume), what would be the sequence of Leiningen
> commands to do that? I would have initially assumed `lein release
> :alpha` but the docs suggest the $LEVEL affects the post-deploy change
> of version only.

Yeah, the :alpha/:beta/:rc stuff was a late addition; I think it needs a
bit more work as the steps of removing the qualifier don't make sense in
that case. Tracking that here:

  https://github.com/technomancy/leiningen/issues/1539

-Phil

Re: [leiningen] Release task

From:
Sean Corfield
Date:
2014-05-27 @ 23:28
On May 27, 2014, at 4:05 PM, Phil Hagelberg <phil@hagelb.org> wrote:
> However, I've added an explicit check for this so you get a nice clear
> message rather than a stack trace if you get this wrong.

Cool.

> Another tactic would be to assume the reader has already been called for
> cases where you get a collection that's not uniformly strings, but I
> feel like that's probably too clever and makes for weird edge cases if
> your args are all strings that need to remain as strings.

Agreed. It does seem a little weird to specify ":release" instead of plain
"release" since you're already forced to deal with strings. Is there a 
precedent for this in Leiningen already? (:prep-tasks maybe?)

I remember this coming up as a point of confusion for lein-daemon which I 
believe was eventually changed to accept both :arg and "arg" and 
internally convert them to "arg" or :arg as needed for various things.

It's a small point but in terms of usability, it's a bit weird to specify 
"Clojure keywords" as command line arguments when they're going to be read
as strings anyway - and the code has to "artificially" convert them back 
to keywords internally. It's something I've always felt was a bit odd 
about things like `lein deps :tree` (rather than `lein deps tree` or `lein
deps --tree` or something more "normal" for a command line utility).

> Yep, these can be used outside the context of the release task of
> course, but they're lower-level; I'd expect them usually to be invoked
> either from the release task or an alias chain. The docstrings for these
> tasks should be self-explanatory.

Excellent!

>  https://github.com/technomancy/leiningen/issues/1539


Thanx. I like the idea of it but it seems tricky to get right...

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)


Re: [leiningen] Release task

From:
Phil Hagelberg
Date:
2014-05-27 @ 23:38
Sean Corfield <sean@corfield.org> writes:

> Agreed. It does seem a little weird to specify ":release" instead of
> plain "release" since you're already forced to deal with strings. Is
> there a precedent for this in Leiningen already? (:prep-tasks maybe?)
>
> It's a small point but in terms of usability, it's a bit weird to
> specify "Clojure keywords" as command line arguments when they're
> going to be read as strings anyway - and the code has to
> "artificially" convert them back to keywords internally. It's
> something I've always felt was a bit odd about things like `lein deps
> :tree` (rather than `lein deps tree` or `lein deps --tree` or
> something more "normal" for a command line utility).

Hm; actually I realized we accept symbols in addition to strings (sort
of by accident), but that lets us drop the colon in this particular
case. Of course, the danger is that it looks like you're passing a
string in when the task actually receives a symbol, which could lead to
confusion with other tasks that aren't as forgiving. But with decent
error reporting it shouldn't be a big deal.

-Phil