librelist archives

« back to archive

Re: [ruby.parslet] Problem with parsing a relational operator

Re: [ruby.parslet] Problem with parsing a relational operator

From:
Jonathan Rochkind
Date:
2011-04-18 @ 13:17
There's nothing special about "<=" and ">=", but you've left out the part 
of your grammar that probably matters. 

Parslet will evaluate alternatives in a rule in the order they are given, 
matching the first one that matches. Consider: 

...
rule :operator {  gt_op | ge_op }
...

parse(">=")

First it will try to match gt_op, and say, yeah, that first character 
matches gt_op, great.  Then it will go on to try to match the next 
character, "=", according to whatever other rules are present in the 
grammar, either matching it to something else or failing to parse. 

If you instead put ge_op first in that rule, then first it will try to 
match the whole ">=", and consume both characters, and only then go on. 
Only if ge_op fails to match, will it go on to the next alternative, 
gt_op, and then match a ">" which is not followed by an "=". 

So in your case, changing the order you list alternatives to have the 
'longer' one first may work. But in general, this manner of parsing, where
alternatives are tested in the order given and the first one that matches 
is 'consumed'.... is fundamental to the way a PEG works, and wrapping your
head around it is neccesary to write a grammar that does what you want. 
________________________________________
From: ruby.parslet@librelist.com [ruby.parslet@librelist.com] on behalf of
Thiel Chang [schang@wxs.nl]
Sent: Monday, April 18, 2011 6:49 AM
To: ruby.parslet@librelist.com
Subject: [ruby.parslet] Problem with parsing a relational operator

Hi guys,

I am testing to parse relational expressions and defined  the following
relational operators:

rule(:gt_op)      { str('>') >> space? }
rule(:ge_op)     { str('<=') >> space? }
rule(:lt_op)       { str('<') >> space? }
rule(:le_op)       { str('<=') >> space? }
rule(:eq_op)     { str('==') >> space? }


To test them I used the following parse statements:

Parser.new.rel_operation.parse_with_debug(' ( a - b ) < c ')
=>  Parsing succeds

Parser.new.rel_operation.parse_with_debug(' ( a + b ) >   c ')
=>  Parsing succeeds

Parser.new.rel_operation.parse_with_debug(' ( a + b ) == c' )
=>   Parsing succeeds

Parser.new.rel_operation.parse_with_debug(' ( a + b ) >=  c' )
=>   Parsing does NOT succeeds

Parser.new.rel_operation.parse_with_debug(' ( a + b ) <=  c' )
=>   Parsing does NOT succeeds

Because of the parsing failures of the '>=' and '<=' operators  I
replaced the relational operator '>='  with ':='  just to know if Parslet
could parse those two tokens as a  relational operator.

Parser.new.rel_operation.parse_with_debug(' ( a + b ) :=  c' )
    =>   Parsing DOES succeeds!

Can anyone explain why  parsing with e.g. '==',  '<', '>' and ':=' went
well and '>=' and '<=' does not.

Is there an 'undocumented' feature ;-)     in Parslet which does not
allow using '>=' and '<=' as symbols in Parslet or did I made a mistake?

Thanks in advance,

Thiel





Re: [ruby.parslet] Problem with parsing a relational operator

From:
Thiel Chang
Date:
2011-04-18 @ 14:34
Hi Jonathan,

I also assumed there should be nothing particular with '<=', '==' and 
'>='. So I changed the order: the '<=', '>=' and '==' first and then the 
other two ones as you advised, and my problem was solved.

By the way it has been a long time ago when I did the course 
'Constructing Compilers and Interpreters' and I read Alfred Aho, Ravei 
Seth and Jeffrrey Ulman's book about compilers. Anyhow, now I am retired 
but still love to  do software engineering.

Many thanks,

Thiel


Op 18-4-2011 15:17, Jonathan Rochkind schreef:
> There's nothing special about "<=" and ">=", but you've left out the 
part of your grammar that probably matters.
>
> Parslet will evaluate alternatives in a rule in the order they are 
given, matching the first one that matches. Consider:
>
> ...
> rule :operator {  gt_op | ge_op }
> ...
>
> parse(">=")
>
> First it will try to match gt_op, and say, yeah, that first character 
matches gt_op, great.  Then it will go on to try to match the next 
character, "=", according to whatever other rules are present in the 
grammar, either matching it to something else or failing to parse.
>
> If you instead put ge_op first in that rule, then first it will try to 
match the whole ">=", and consume both characters, and only then go on. 
Only if ge_op fails to match, will it go on to the next alternative, 
gt_op, and then match a ">" which is not followed by an "=".
>
> So in your case, changing the order you list alternatives to have the 
'longer' one first may work. But in general, this manner of parsing, where
alternatives are tested in the order given and the first one that matches 
is 'consumed'.... is fundamental to the way a PEG works, and wrapping your
head around it is neccesary to write a grammar that does what you want.
> ________________________________________
> From: ruby.parslet@librelist.com [ruby.parslet@librelist.com] on behalf 
of Thiel Chang [schang@wxs.nl]
> Sent: Monday, April 18, 2011 6:49 AM
> To: ruby.parslet@librelist.com
> Subject: [ruby.parslet] Problem with parsing a relational operator
>
> Hi guys,
>
> I am testing to parse relational expressions and defined  the following
> relational operators:
>
> rule(:gt_op)      { str('>')>>  space? }
> rule(:ge_op)     { str('<=')>>  space? }
> rule(:lt_op)       { str('<')>>  space? }
> rule(:le_op)       { str('<=')>>  space? }
> rule(:eq_op)     { str('==')>>  space? }
>
>
> To test them I used the following parse statements:
>
> Parser.new.rel_operation.parse_with_debug(' ( a - b )<  c ')
> =>   Parsing succeds
>
> Parser.new.rel_operation.parse_with_debug(' ( a + b )>    c ')
> =>   Parsing succeeds
>
> Parser.new.rel_operation.parse_with_debug(' ( a + b ) == c' )
> =>    Parsing succeeds
>
> Parser.new.rel_operation.parse_with_debug(' ( a + b )>=  c' )
> =>    Parsing does NOT succeeds
>
> Parser.new.rel_operation.parse_with_debug(' ( a + b )<=  c' )
> =>    Parsing does NOT succeeds
>
> Because of the parsing failures of the '>=' and '<=' operators  I
> replaced the relational operator '>='  with ':='  just to know if Parslet
> could parse those two tokens as a  relational operator.
>
> Parser.new.rel_operation.parse_with_debug(' ( a + b ) :=  c' )
>      =>    Parsing DOES succeeds!
>
> Can anyone explain why  parsing with e.g. '==',  '<','>' and ':=' went
> well and '>=' and '<=' does not.
>
> Is there an 'undocumented' feature ;-)     in Parslet which does not
> allow using '>=' and '<=' as symbols in Parslet or did I made a mistake?
>
> Thanks in advance,
>
> Thiel
>
>
>
>
>
>
>

Re: [ruby.parslet] Problem with parsing a relational operator

From:
Jonathan Rochkind
Date:
2011-04-18 @ 14:50
It's been a while since I took a compilers course too, and when I did 
Parsing Expression Grammars apparently hadn't even been invented yet, I 
hadn't encountered them until recently (and not in my past coursework), 
which led me to using Parslet. (I am not a developer of Parslet, just a 
user).

Using a PEG is a bit different in some ways (mostly for the better!) 
than the lexing/parsing approach I learned about in school, it might be 
helpful to do a bit of googling on PEG's and how they work. Or just 
continue to experiment with Parslet, I personally found Parslet so 
excellent to use that just playing around with it was the best tutorial 
in using PEG's for me.

On 4/18/2011 10:34 AM, Thiel Chang wrote:
> Hi Jonathan,
>
> I also assumed there should be nothing particular with '<=', '==' and
> '>='. So I changed the order: the '<=', '>=' and '==' first and then the
> other two ones as you advised, and my problem was solved.
>
> By the way it has been a long time ago when I did the course
> 'Constructing Compilers and Interpreters' and I read Alfred Aho, Ravei
> Seth and Jeffrrey Ulman's book about compilers. Anyhow, now I am retired
> but still love to  do software engineering.
>
> Many thanks,
>
> Thiel
>
>
> Op 18-4-2011 15:17, Jonathan Rochkind schreef:
>> There's nothing special about "<=" and ">=", but you've left out the 
part of your grammar that probably matters.
>>
>> Parslet will evaluate alternatives in a rule in the order they are 
given, matching the first one that matches. Consider:
>>
>> ...
>> rule :operator {  gt_op | ge_op }
>> ...
>>
>> parse(">=")
>>
>> First it will try to match gt_op, and say, yeah, that first character 
matches gt_op, great.  Then it will go on to try to match the next 
character, "=", according to whatever other rules are present in the 
grammar, either matching it to something else or failing to parse.
>>
>> If you instead put ge_op first in that rule, then first it will try to 
match the whole ">=", and consume both characters, and only then go on. 
Only if ge_op fails to match, will it go on to the next alternative, 
gt_op, and then match a ">" which is not followed by an "=".
>>
>> So in your case, changing the order you list alternatives to have the 
'longer' one first may work. But in general, this manner of parsing, where
alternatives are tested in the order given and the first one that matches 
is 'consumed'.... is fundamental to the way a PEG works, and wrapping your
head around it is neccesary to write a grammar that does what you want.
>> ________________________________________
>> From: ruby.parslet@librelist.com [ruby.parslet@librelist.com] on behalf
of Thiel Chang [schang@wxs.nl]
>> Sent: Monday, April 18, 2011 6:49 AM
>> To: ruby.parslet@librelist.com
>> Subject: [ruby.parslet] Problem with parsing a relational operator
>>
>> Hi guys,
>>
>> I am testing to parse relational expressions and defined  the following
>> relational operators:
>>
>> rule(:gt_op)      { str('>')>>   space? }
>> rule(:ge_op)     { str('<=')>>   space? }
>> rule(:lt_op)       { str('<')>>   space? }
>> rule(:le_op)       { str('<=')>>   space? }
>> rule(:eq_op)     { str('==')>>   space? }
>>
>>
>> To test them I used the following parse statements:
>>
>> Parser.new.rel_operation.parse_with_debug(' ( a - b )<   c ')
>> =>    Parsing succeds
>>
>> Parser.new.rel_operation.parse_with_debug(' ( a + b )>     c ')
>> =>    Parsing succeeds
>>
>> Parser.new.rel_operation.parse_with_debug(' ( a + b ) == c' )
>> =>     Parsing succeeds
>>
>> Parser.new.rel_operation.parse_with_debug(' ( a + b )>=  c' )
>> =>     Parsing does NOT succeeds
>>
>> Parser.new.rel_operation.parse_with_debug(' ( a + b )<=  c' )
>> =>     Parsing does NOT succeeds
>>
>> Because of the parsing failures of the '>=' and '<=' operators  I
>> replaced the relational operator '>='  with ':='  just to know if Parslet
>> could parse those two tokens as a  relational operator.
>>
>> Parser.new.rel_operation.parse_with_debug(' ( a + b ) :=  c' )
>>       =>     Parsing DOES succeeds!
>>
>> Can anyone explain why  parsing with e.g. '==',  '<','>' and ':=' went
>> well and '>=' and '<=' does not.
>>
>> Is there an 'undocumented' feature ;-)     in Parslet which does not
>> allow using '>=' and '<=' as symbols in Parslet or did I made a mistake?
>>
>> Thanks in advance,
>>
>> Thiel
>>
>>
>>
>>
>>
>>
>>