librelist archives

« back to archive

new template args require -- separator

new template args require -- separator

From:
Steve Miner
Date:
2013-12-23 @ 22:08
I recently upgraded to Leiningen 2.3.4.  I thought my template was broken,
but really there was a change to how template args are processed.  I was 
using optional "keyword" args (really strings starting with ":") with my 
template.  With 2.3.4, I stopped getting the extra args in my template 
function.  After a bit of investigation, it looks like you're now required
to separate template args from lein command args with a "--".  

Old way:
% lein new mytemplate myproj :foo 42

2.3.4 way:
% lein new mytemplate myproj -- :foo 42

Not a big deal.  I mention it only in case someone else is similarly surprised.

By the way, `lein new help` gave me a clue with its discussion of --snapshot:

If you use the `--snapshot` argument with template args you may need to use `--`
to prevent template args from being interpreted as arguments to `lein new`:

  lein new $TEMPLATE_NAME $PROJECT_NAME --snapshot -- template-arg-1 
template-arg-2

In case anyone cares, my actual template is:
https://github.com/miner/nocore

Steve Miner
steveminer@gmail.com

Re: [leiningen] new template args require -- separator

From:
Travis Vachon
Date:
2013-12-23 @ 22:50
Arg, shoot - the intent was for the args parsing change to be
backwards compatible, sorry about that. I actually think it's sort of
awkward to require -- in all cases, ideally it would just be required
in cases where there's ambiguity. I'll take a look at how hard this is
to fix.



On Mon, Dec 23, 2013 at 5:08 PM, Steve Miner <steveminer@gmail.com> wrote:
> I recently upgraded to Leiningen 2.3.4.  I thought my template was 
broken, but really there was a change to how template args are processed.
I was using optional "keyword" args (really strings starting with ":") 
with my template.  With 2.3.4, I stopped getting the extra args in my 
template function.  After a bit of investigation, it looks like you're now
required to separate template args from lein command args with a "--".
>
> Old way:
> % lein new mytemplate myproj :foo 42
>
> 2.3.4 way:
> % lein new mytemplate myproj -- :foo 42
>
> Not a big deal.  I mention it only in case someone else is similarly surprised.
>
> By the way, `lein new help` gave me a clue with its discussion of --snapshot:
>
> If you use the `--snapshot` argument with template args you may need to use `--`
> to prevent template args from being interpreted as arguments to `lein new`:
>
>   lein new $TEMPLATE_NAME $PROJECT_NAME --snapshot -- template-arg-1 
template-arg-2
>
> In case anyone cares, my actual template is:
> https://github.com/miner/nocore
>
> Steve Miner
> steveminer@gmail.com
>

Re: [leiningen] new template args require -- separator

From:
Phil Hagelberg
Date:
2013-12-23 @ 22:52
Steve Miner writes:

> I recently upgraded to Leiningen 2.3.4.  I thought my template was
> broken, but really there was a change to how template args are
> processed.  I was using optional "keyword" args (really strings
> starting with ":") with my template.  With 2.3.4, I stopped getting
> the extra args in my template function.  After a bit of investigation,
> it looks like you're now required to separate template args from lein
> command args with a "--".

Yeah, this was changed in order to support the --snapshot argument to
lein new. Adding new functionality to existing tasks that need to pass
arbitrary args on is tricky, and it looks like we may have missed a
spot. Happy to take a pull request that preserves the old behaviour as
long as --snapshot is still supported.

https://github.com/technomancy/leiningen/issues/1235

-Phil

Re: [leiningen] new template args require -- separator

From:
Travis Vachon
Date:
2013-12-24 @ 02:07
Oh I see - I think this actually changed after the work I did. In the
updated stuff, I see the following two tests:

---
  (is (= (parse-options [":fish" "salmon"])
         [{:fish "salmon"} []]))

  (is (= (parse-options ["salmon" "trout"])
         [{} ["salmon" "trout"]]))
---

The map in the first position is used by leiningen, the array is the
list of arguments passed to the template. It looks like non-keyword-y
strings are passed to the template as I'd expect, but keyword-y
arguments make lein interpret the arguments the way it would
interpret, eg, "--fish salmon"

Indeed,

---
lein new mytemplate myproj :foo 42
---

works the way you describe, but

---
lein new mytemplate myproj foo 42
---

passes "foo 42" to the template.


I'm hesitant to change this because I don't understand the current
test - what's the thinking behind this test:

---
  (is (= (parse-options [":fish" "salmon"])
         [{:fish "salmon"} []]))
---
?


If it's an accident I'll change the test to

----
  (is (= (parse-options [":fish" "salmon"])
         [{} [:fish "salmon"]]))
---

and figure out a fix.

Thanks!

Travis

On Mon, Dec 23, 2013 at 5:52 PM, Phil Hagelberg <phil@hagelb.org> wrote:
>
> Steve Miner writes:
>
>> I recently upgraded to Leiningen 2.3.4.  I thought my template was
>> broken, but really there was a change to how template args are
>> processed.  I was using optional "keyword" args (really strings
>> starting with ":") with my template.  With 2.3.4, I stopped getting
>> the extra args in my template function.  After a bit of investigation,
>> it looks like you're now required to separate template args from lein
>> command args with a "--".
>
> Yeah, this was changed in order to support the --snapshot argument to
> lein new. Adding new functionality to existing tasks that need to pass
> arbitrary args on is tricky, and it looks like we may have missed a
> spot. Happy to take a pull request that preserves the old behaviour as
> long as --snapshot is still supported.
>
> https://github.com/technomancy/leiningen/issues/1235
>
> -Phil

Re: [leiningen] new template args require -- separator

From:
Jean Niklas L'orange
Date:
2013-12-24 @ 08:52
Hello,

On 24 December 2013 03:07, Travis Vachon <travis.vachon@gmail.com> wrote:

> Oh I see - I think this actually changed after the work I did. In the
> updated stuff, I see the following two tests:
>
> ---
>   (is (= (parse-options [":fish" "salmon"])
>          [{:fish "salmon"} []]))
>
>   (is (= (parse-options ["salmon" "trout"])
>          [{} ["salmon" "trout"]]))
> ---
>
> The map in the first position is used by leiningen, the array is the
> list of arguments passed to the template. It looks like non-keyword-y
> strings are passed to the template as I'd expect, but keyword-y
> arguments make lein interpret the arguments the way it would
> interpret, eg, "--fish salmon"
>
> Indeed,
>
> ---
> lein new mytemplate myproj :foo 42
> ---
>
> works the way you describe, but
>
> ---
> lein new mytemplate myproj foo 42
> ---
>
> passes "foo 42" to the template.
>
>
> I'm hesitant to change this because I don't understand the current
> test - what's the thinking behind this test:
>
> ---
>   (is (= (parse-options [":fish" "salmon"])
>          [{:fish "salmon"} []]))
> ---
> ?
>
>
> If it's an accident I'll change the test to
>
> ----
>   (is (= (parse-options [":fish" "salmon"])
>          [{} [:fish "salmon"]]))
> ---
>

From what I can understand, this is how it is supposed to work -- it
provides plugin designers and other tasks general parsing options if
needed. The problem here is rather how other arguments are ignored in
new.clj. The `options` variable should be passed over to the template,
dissocing new-specific arguments and then serialised.

-- 
Regards,
Jean Niklas L'orange

Re: [leiningen] new template args require -- separator

From:
Travis Vachon
Date:
2013-12-24 @ 16:42
I'm not sure I follow - are you saying it's desirable to parse the ":foo 42" in

lein new mytemplate myproject :foo 42

into a map before it gets passed to mytemplate? I'm not really sure I
agree unless this is legacy behavior.

IMHO (based on very little real experience though) any arguments not
explicitly used by "lein new" should be passed along to the template
as a vector of strings. Templates can be responsible for turning that
vector into whatever they'd like.

Definitely interested in other opinions or thoughts on what this used
to do. Thanks!

Travis

On Tue, Dec 24, 2013 at 3:52 AM, Jean Niklas L'orange
<jeannikl@hypirion.com> wrote:
> Hello,
>
> On 24 December 2013 03:07, Travis Vachon <travis.vachon@gmail.com> wrote:
>>
>> Oh I see - I think this actually changed after the work I did. In the
>> updated stuff, I see the following two tests:
>>
>> ---
>>   (is (= (parse-options [":fish" "salmon"])
>>          [{:fish "salmon"} []]))
>>
>>   (is (= (parse-options ["salmon" "trout"])
>>          [{} ["salmon" "trout"]]))
>> ---
>>
>> The map in the first position is used by leiningen, the array is the
>> list of arguments passed to the template. It looks like non-keyword-y
>> strings are passed to the template as I'd expect, but keyword-y
>> arguments make lein interpret the arguments the way it would
>> interpret, eg, "--fish salmon"
>>
>> Indeed,
>>
>> ---
>> lein new mytemplate myproj :foo 42
>> ---
>>
>> works the way you describe, but
>>
>> ---
>> lein new mytemplate myproj foo 42
>> ---
>>
>> passes "foo 42" to the template.
>>
>>
>> I'm hesitant to change this because I don't understand the current
>> test - what's the thinking behind this test:
>>
>> ---
>>   (is (= (parse-options [":fish" "salmon"])
>>          [{:fish "salmon"} []]))
>> ---
>> ?
>>
>>
>> If it's an accident I'll change the test to
>>
>> ----
>>   (is (= (parse-options [":fish" "salmon"])
>>          [{} [:fish "salmon"]]))
>> ---
>
>
> From what I can understand, this is how it is supposed to work -- it
> provides plugin designers and other tasks general parsing options if needed.
> The problem here is rather how other arguments are ignored in new.clj. The
> `options` variable should be passed over to the template, dissocing
> new-specific arguments and then serialised.
>
> --
> Regards,
> Jean Niklas L'orange

Re: [leiningen] new template args require -- separator

From:
Jean Niklas L'orange
Date:
2013-12-26 @ 02:30
On 24 December 2013 17:42, Travis Vachon <travis.vachon@gmail.com> wrote:

> I'm not sure I follow - are you saying it's desirable to parse the ":foo
> 42" in
>
> lein new mytemplate myproject :foo 42
>
> into a map before it gets passed to mytemplate? I'm not really sure I
> agree unless this is legacy behavior.
>
> IMHO (based on very little real experience though) any arguments not
> explicitly used by "lein new" should be passed along to the template
> as a vector of strings. Templates can be responsible for turning that
> vector into whatever they'd like.
>

Ah, sorry for poor English. What I meant was that `parse-options` is
working as intended and should not be changed. The template task, on the
other hand, is buggy (because it is using `parse-options`) and should be
fixed.

-- 
Regards,
Jean Niklas L'orange

Re: [leiningen] new template args require -- separator

From:
Steve Miner
Date:
2013-12-26 @ 16:03
On Dec 24, 2013, at 11:42 AM, Travis Vachon <travis.vachon@gmail.com> wrote:

> I'm not sure I follow - are you saying it's desirable to parse the ":foo 42" in
> 
> lein new mytemplate myproject :foo 42
> 
> into a map before it gets passed to mytemplate? I'm not really sure I
> agree unless this is legacy behavior.

My understanding of the parsing code in Leiningen 2.3.4 is that it's 
trying to separate the "command args" from the "template args".  The 
command args are parsed into a map for internal use.  The template args 
are parsed into a vector that is basically applied to the template 
function (after the project name arg).

In my case, I had a bunch of optional template parameters that I wanted to
allow, but didn't want to use positional arguments so I faked a keyword 
style arg list.  In my template function, I use destructuring on the 
template args, which effectively pours the seq of fake keyword style args 
into a map.  So it's my code, not Leiningen, interpreting the args in a 
map-like way.

The issue is that Leiningen 2.3.4 is grabbing every keyword-like arg first
and giving them only to the `new` command, not my template anymore.  The 
magic marker "--" tells the parser that everything after it belongs to the
template args, so that's the work-around for my case.


On Dec 24, 2013, at 3:52 AM, Jean Niklas L'orange <jeannikl@hypirion.com> wrote:

> From what I can understand, this is how it is supposed to work -- it 
provides plugin designers and other tasks general parsing options if 
needed. The problem here is rather how other arguments are ignored in 
new.clj. The `options` variable should be passed over to the template, 
dissocing new-specific arguments and then serialised.

Yes, I think this the correct analysis, but I worry a bit about 
serializing the options map.  We have to be careful to maintain the exact 
order of the template args.  It might be better to make `parse-args` 
distinguish the official `new` arguments (--snapshot and --to-dir) and 
send anything else to the template.  I think it's reasonable to require 
that those two options always come before the template args.

As it stands, the "--" marker worked fine for me once I understood the issue.