librelist archives

« back to archive

error when trying to call parse

error when trying to call parse

From:
Jonathan Rochkind
Date:
2011-03-10 @ 17:01
I played around with parslet a couple months ago when I saw the 
announcement, and loved it. Very nicely done, well-structured, easy to 
use, thanks.

Today, I tried firing up parslet on a different machine, and am getting 
errors trying to use it. When calling #parse a class that subclasses  
Parslet::Parser, I get:

/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/re.rb:16: 
warning: regexp has invalid interval
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/re.rb:16: 
warning: regexp has `}' without escape
NoMethodError: undefined method `tap' for \\\" (SPACING? TOKEN 
SPACING?){1, } \\\":Parslet::Atoms::Re
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/entity.rb:25:in 
`parslet'


That last being a fatal error.

Any ideas? I have noticed that this machine has ruby 1.8.6 installed, is 
that the problem, do I need 1.8.7?

Thanks for any help.

Re: error when trying to call parse

From:
Kaspar Schiess
Date:
2011-03-14 @ 07:11
> Today, I tried firing up parslet on a different machine, and am getting
> errors trying to use it. When calling #parse a class that subclasses
> Parslet::Parser, I get:
>
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/re.rb:16:
> warning: regexp has invalid interval
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/re.rb:16:
> warning: regexp has `}' without escape
> NoMethodError: undefined method `tap' for \\\" (SPACING? TOKEN
> SPACING?){1, } \\\":Parslet::Atoms::Re
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/entity.rb:25:in
> `parslet'
>
>
> That last being a fatal error.
>
> Any ideas? I have noticed that this machine has ruby 1.8.6 installed, is
> that the problem, do I need 1.8.7?

It may very well be that 1.8.6 wont work. But judging from your stack 
trace, this might be related to input you're giving as well; 
Parslet::Atoms::Re constructs a regular expression from the range you're 
feeding it (match('...')) - and that seems to fail. Maybe if you give us 
something to work with (a piece of your grammar that reproduces the 
error), somebody here will be able to reproduce this and narrow it down?

kaspar

Re: error when trying to call parse

From:
Kaspar Schiess
Date:
2011-03-14 @ 07:16
> It may very well be that 1.8.6 wont work. But judging from your stack
> trace, this might be related to input you're giving as well;
> Parslet::Atoms::Re constructs a regular expression from the range you're
> feeding it (match('...')) - and that seems to fail. Maybe if you give us
> something to work with (a piece of your grammar that reproduces the
> error), somebody here will be able to reproduce this and narrow it down?

And a correction ;) .. Is #tap really missing from 1.8.6? I guess that 
would kill compatibility, because I am a fan ;)

kaspar

Re: [ruby.parslet] Re: error when trying to call parse

From:
Jonathan Rochkind
Date:
2011-03-14 @ 14:50
Investigating more, it looks like #tap really is missing from 1.8.6 (but 
present in 1.8.7), and it looks to me like that was the nature of the 
problem, not any input problems.

Fortunately, it also looks like this is really easy to fix. I added this 
code to the top of my file to monkey patch a pure ruby implementation of 
Object#tap back in, and Parslet seems to be working as expected now.  
(Looks like the actual 1.8.7 implementation is C, not pure ruby, so, you 
know, maybe there are performance implications, but I doubt it and don't 
care that much).

Again, thanks for Parslet, using it is hands-down the most delightful 
parsing experience I've ever had.

unless Object.method_defined?(:tap)
   class Object
     def tap
       yield(self)
       return self
     end
   end
end





On 3/14/2011 3:16 AM, Kaspar Schiess wrote:
>> It may very well be that 1.8.6 wont work. But judging from your stack
>> trace, this might be related to input you're giving as well;
>> Parslet::Atoms::Re constructs a regular expression from the range you're
>> feeding it (match('...')) - and that seems to fail. Maybe if you give us
>> something to work with (a piece of your grammar that reproduces the
>> error), somebody here will be able to reproduce this and narrow it down?
> And a correction ;) .. Is #tap really missing from 1.8.6? I guess that
> would kill compatibility, because I am a fan ;)
>
> kaspar
>

recognize this exception?

From:
Jonathan Rochkind
Date:
2011-03-14 @ 23:23
Okay, yeah, I'm trying to do complicated things. But without getting 
into my entire list of parse rules and asking you to do my work for me, 
have you ever seen an exception like this before?  When I try a certain 
variation of my parse rules, on certain input, I get this exception from 
parslet, which I was not expecting (just including the first part of the 
stack trace):

ArgumentError: wrong number of arguments (1 for 0)
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:184:in 
`flatten'
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:184:in 
`flatten_repetition'
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:115:in 
`flatten'
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:107:in 
`flatten'
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:106:in 
`map'
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:106:in 
`flatten'
         from 
/usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/named.rb:34:in 
`produce_return_value'

Re: [ruby.parslet] recognize this exception?

From:
Jonathan Rochkind
Date:
2011-03-14 @ 23:37
Hmm, weirdly that seemed to be caused not by the parse rules themselves, 
but by the as(:label)s I added to them. Adding an _additional_ 
.as(:label) in a certain place somehow solved this, I can't explain it. 
I could give you my entire complicated parser, but I don't expect anyone 
else to try and understand it.

On 3/14/2011 7:23 PM, Jonathan Rochkind wrote:
> Okay, yeah, I'm trying to do complicated things. But without getting
> into my entire list of parse rules and asking you to do my work for me,
> have you ever seen an exception like this before?  When I try a certain
> variation of my parse rules, on certain input, I get this exception from
> parslet, which I was not expecting (just including the first part of the
> stack trace):
>
> ArgumentError: wrong number of arguments (1 for 0)
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:184:in
> `flatten'
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:184:in
> `flatten_repetition'
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:115:in
> `flatten'
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:107:in
> `flatten'
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:106:in
> `map'
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/base.rb:106:in
> `flatten'
>           from
> /usr/lib64/ruby/gems/1.8/gems/parslet-1.2.0/lib/parslet/atoms/named.rb:34:in
> `produce_return_value'
>

Re: recognize this exception?

From:
Kaspar Schiess
Date:
2011-03-15 @ 09:11
On 15.03.11 00:37, Jonathan Rochkind wrote:
> Hmm, weirdly that seemed to be caused not by the parse rules themselves,
> but by the as(:label)s I added to them. Adding an _additional_
> .as(:label) in a certain place somehow solved this, I can't explain it.
> I could give you my entire complicated parser, but I don't expect anyone
> else to try and understand it.

Hei Jonathan,

I would sure like to try and understand it ;). And I guess we might find 
the mindshare of others as well. Can you post a sample that exposes this 
error? Even if it is not minimal, there might be a bug lurking in there!

kaspar


Re: [ruby.parslet] Re: recognize this exception?

From:
Jonathan Rochkind
Date:
2011-03-15 @ 15:14
Okay, sorry I don't have time/ability to create a minimal reproduction 
case, but here's my whole thing.

1) The whole grammar that reveals the problem. 2) the string input that 
will reveal the problem. 3) The rule change (adding an .as() ) that 
cures it.

As an aside, I have gotten the feeling writing this code that I'm not 
using .as() quite as you intended; but I couldn't quite wrap my head 
around how you intended with my use case, and the way I'm doing is 
working out well for my brain and use case.

And in fact I think the change I made _was_ neccesary for me to get 
output I could deal with properly; but perhaps the pre-change version 
still shouldn't have raised, or should have raised a more clear exception.

**
1. Original grammar revealing the problem
**
class AdvParser < Parslet::Parser
   root :query

   # query is actually a list of expressions.
   rule :query do
     (spacing? >>  (expression | paren_unit ) >> spacing?).repeat
   end

   rule :paren_list do
     (str('(') >> (spacing? >> unary_expression >> spacing?).repeat >> 
str(')'))
   end

   rule :paren_unit do
     (str('(') >> spacing? >> (expression ) >> spacing? >> str(')')) |
       paren_list
   end

   # Note well: It was tricky to parse the thing we want where you can
   # have a flat list with boolean operators, but where 'OR' takes 
precedence.
   # eg "A AND B OR C AND C" or "A OR B AND C OR D". Tricky to parse at all,
   # tricky to make precedence work. Important things that seem to make 
it work:
   # and_list comes BEFORE or_list in :expression.
   # and_list's operand can be an or_list, but NOT vice versa
   rule :expression do
     (and_list | or_list | unary_expression )
   end

   rule :and_list do
     ((or_list | unary_expression | paren_unit) >>
       (spacing >> str("AND") >> spacing >> (or_list | unary_expression 
| paren_unit)).repeat(1)).as(:and_list)
   end

   rule :or_list do
     ((unary_expression | paren_unit) >>
     (spacing >> str("OR") >> spacing >> (unary_expression | 
paren_unit)).repeat(1)).as(:or_list)
   end

   rule :unary_expression do
     (str('+') >> (phrase | token)).as(:mandatory) |
     (str('-') >> (phrase | token)).as(:excluded) |
     (str('NOT') >> spacing? >> (unary_expression | 
paren_unit)).as(:not_expression) |
     (phrase | token)
   end

   rule :token do
     match['^ ":)('].repeat(1).as(:token)
   end
   rule :phrase do
     match('"') >> match['^"'].repeat(1).as(:phrase) >> match('"')
   end


   rule :spacing do
     match[' '].repeat(1)
   end
   rule :spacing? do
     spacing.maybe
   end
end


**

2. Input string
**
(one two three) AND (four five six)
**

3. Change this rule adding an 'as', all is well again.
**
   rule :paren_list do
     (str('(') >> (spacing? >> unary_expression >> spacing?).repeat >> 
str(')')).as(:list)
   end
**


On 3/15/2011 5:11 AM, Kaspar Schiess wrote:
> On 15.03.11 00:37, Jonathan Rochkind wrote:
>> Hmm, weirdly that seemed to be caused not by the parse rules themselves,
>> but by the as(:label)s I added to them. Adding an _additional_
>> .as(:label) in a certain place somehow solved this, I can't explain it.
>> I could give you my entire complicated parser, but I don't expect anyone
>> else to try and understand it.
> Hei Jonathan,
>
> I would sure like to try and understand it ;). And I guess we might find
> the mindshare of others as well. Can you post a sample that exposes this
> error? Even if it is not minimal, there might be a bug lurking in there!
>
> kaspar
>
>
>

Re: recognize this exception?

From:
Kaspar Schiess
Date:
2011-03-17 @ 07:12
Hei Jonathan,

I've just created a small stub around the grammar you provided and ran 
it with Ruby 1.9.2, parslet 1.2 on the input you provided and got what 
looks like valid output to me (no exception):

[{:and_list=>[{:token=>"one"@1}, {:token=>"two"@5}, {:token=>"three"@9}, 
{:token=>"four"@21}, {:token=>"five"@26}, {:token=>"six"@31}]}]

Here's my setup:

   https://gist.github.com/873949

Can you try to be more specific? OS, Ruby and parslet version?

Thanks for your help on tracking this down!
kaspar

Re: [ruby.parslet] Re: recognize this exception?

From:
Jonathan Rochkind
Date:
2011-03-17 @ 14:48
ruby 1.8.6 (I know, I know!).   Parslet 1.2.0.   OS is linux  RHEL5: Red 
Hat Enterprise Linux Server release 5.5 (Tikanga).

It seems plausible that the problem does not occur in 1.9 but does in 
1.8.  It seems possible but less likely that it appears in 1.8.6 but not 
1.8.7, but I don't have time to test it on my 1.8.7 machine at the 
moment sorry. (I don't do any 1.9 yet).

It seems unlikely the OS would be relevant, yes?

[But in more positive news, I can't resist saying again I'm having so 
much fun and effectiveness with Parslet, it is so well done -- although 
like I said I'm not using the Transform stuff, I'm using my own 
different approach, but that's just a testament to the greatness of 
Parslet, that it doesn't lock you into just one way of doing things, 
it's flexible.]

Jonathan


On 3/17/2011 3:12 AM, Kaspar Schiess wrote:
> Hei Jonathan,
>
> I've just created a small stub around the grammar you provided and ran
> it with Ruby 1.9.2, parslet 1.2 on the input you provided and got what
> looks like valid output to me (no exception):
>
> [{:and_list=>[{:token=>"one"@1}, {:token=>"two"@5}, {:token=>"three"@9},
> {:token=>"four"@21}, {:token=>"five"@26}, {:token=>"six"@31}]}]
>
> Here's my setup:
>
>     https://gist.github.com/873949
>
> Can you try to be more specific? OS, Ruby and parslet version?
>
> Thanks for your help on tracking this down!
> kaspar
>

Re: recognize this exception?

From:
Kaspar Schiess
Date:
2011-03-18 @ 12:30
> ruby 1.8.6 (I know, I know!).   Parslet 1.2.0.   OS is linux  RHEL5: Red
> Hat Enterprise Linux Server release 5.5 (Tikanga).

I would suggest you clone parslet (kschiess/parslet.git) and try running 
the specs on 1.8.6 before going further into this. If all specs run, 
then indeed I will invest time. But I am guessing that parslet and 1.8.6 
are just not made for each other. I would sure accept well formed 
patches that introduce 1.8.6 compatibility - but won't provide those 
myself.

> [But in more positive news, I can't resist saying again I'm having so
> much fun and effectiveness with Parslet, it is so well done -- although
> like I said I'm not using the Transform stuff, I'm using my own
> different approach, but that's just a testament to the greatness of
> Parslet, that it doesn't lock you into just one way of doing things,
> it's flexible.]

Thank you for your kind words. As much as parslet is a library, it is 
also a testament to how I think Ruby libraries should be made; So your 
compliments mean that I am perhaps not alone in thinking so.

And on a more practical note: You have understood the decoupling between 
parser and transformer well and the fact that you chose not to use the 
latter might just be the best decision for your project!

best regards,
kaspar

Re: [ruby.parslet] Re: recognize this exception?

From:
Jonathan Rochkind
Date:
2011-03-21 @ 17:43
On 3/18/2011 8:30 AM, Kaspar Schiess wrote:
> I would suggest you clone parslet (kschiess/parslet.git) and try running
> the specs on 1.8.6 before going further into this. If all specs run,

Will do, makes sense. Not sure the 'right' way to run all the tests from 
the checked out code? Can you give me a hint?

We already know that Object#tap has to be added to a 1.8.6 env for 
parslet to work; I too am curious to see if all tests pass once this is 
done or not.

Either way, it is in fact _working_ (once I worked around that weird 
issue at the start of this thread) for me in 1.8.6.

Re: recognize this exception?

From:
Kaspar Schiess
Date:
2011-03-22 @ 07:59
Hi Jonathan,

> Will do, makes sense. Not sure the 'right' way to run all the tests from
> the checked out code? Can you give me a hint?
rake -T ;)

...
rake spec             # Run all tests: Exhaustive.
rake spec:unit        # Only run unit tests: Fast.
...

The integration tests in rake spec might provoke some failures, since 
the output of the examples is tested as well. That's 'normal'.

>
> We already know that Object#tap has to be added to a 1.8.6 env for
> parslet to work; I too am curious to see if all tests pass once this is
> done or not.
>
> Either way, it is in fact _working_ (once I worked around that weird
> issue at the start of this thread) for me in 1.8.6.

I guess I would try to make sure. But whatever fits your needs ;)

More assistance can be had on the IRC channel, btw. And we might ask you 
to explain what it is you're doing, out of curiosity ;)

kaspar





Re: [ruby.parslet] Re: recognize this exception?

From:
Jonathan Rochkind
Date:
2011-03-22 @ 14:02
Yeah, I actually tried running those rake tasks, but it did not work. 
Maybe because of a ruby 1.8.6 issue somehow?  Contrary to what the error 
messages imply, I do certainly have the rspec gem installed on this 
machine -- but it's rspec 1.3.0, maybe that's bad?

[rochkind@xs001 parslet]$ rake spec:unit --trace
(in /home/rochkind/parslet)
rake aborted!
no such file to load -- rspec/core/rake_task
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in 
`gem_original_require'

[Same thing from "rake spec". ]



I did try to stop in the IRC channel yesterday, but it seemed nobody was 
around when I was. I think our timezones may be too out of sync for 
synchronous communication.



What I am working on:   Parse somewhat google-like search queries, such 
as:  "one -two +three (a OR b OR c) NOT d" Translate them to a certain 
kind of query against Solr (there are some choices you could make there 
about how to translate, I have made some).

The code is actually in a Rails engine plugin (Rails2 at the moment, 
sadly) meant to work with a particular Rails2 application (called 
Blacklight), but is written in a reasonably de-coupled way so if someone 
was interested it could probably be extracted into it's own gem. Still a 
work in progress (and no documentation yet), but here's what it is so 
far, if you want to check it out. The first file is the Parslet grammar 
(which is pretty clean, I think).

The second file is my method of transforming into an Object tree and the 
definitions for those objects, including output translation methods 
(getting a bit sketchy, I might have been better off trying to do 
something with Parslet::Transformer, but just couldn't wrap my head 
around it), then three files of specs.


https://github.com/projectblacklight/blacklight_advanced_search/blob/parsing_nesting/lib/parsing_nesting/grammar.rb


https://github.com/projectblacklight/blacklight_advanced_search/blob/parsing_nesting/lib/parsing_nesting/tree.rb



https://github.com/projectblacklight/blacklight_advanced_search/blob/parsing_nesting/spec/parsing_nesting/consuming_spec.rb


https://github.com/projectblacklight/blacklight_advanced_search/blob/parsing_nesting/spec/parsing_nesting/build_tree_spec.rb


https://github.com/projectblacklight/blacklight_advanced_search/blob/parsing_nesting/spec/parsing_nesting/to_solr_spec.rb






On 3/22/2011 3:59 AM, Kaspar Schiess wrote:
> Hi Jonathan,
>
>> Will do, makes sense. Not sure the 'right' way to run all the tests from
>> the checked out code? Can you give me a hint?
> rake -T ;)
>
> ...
> rake spec             # Run all tests: Exhaustive.
> rake spec:unit        # Only run unit tests: Fast.
> ...
>
> The integration tests in rake spec might provoke some failures, since
> the output of the examples is tested as well. That's 'normal'.
>
>> We already know that Object#tap has to be added to a 1.8.6 env for
>> parslet to work; I too am curious to see if all tests pass once this is
>> done or not.
>>
>> Either way, it is in fact _working_ (once I worked around that weird
>> issue at the start of this thread) for me in 1.8.6.
> I guess I would try to make sure. But whatever fits your needs ;)
>
> More assistance can be had on the IRC channel, btw. And we might ask you
> to explain what it is you're doing, out of curiosity ;)
>
> kaspar
>
>
>
>
>
>