librelist archives

« back to archive

Strings matching and reserved words

Strings matching and reserved words

From:
Giovanni P
Date:
2014-10-31 @ 12:56
Hello, I'm totally new to this parser thing and I have a bizarre grammar
that is presenting me with various problems.

To abstract the most important one, I made this small grammar. See if you
can understand my drama (it works at http://zaach.github.io/jison/try/ if
you just paste it there, as this is what I was doing to test):

/* lexical grammar */
%lex
%%

\s+                                    /* skip whitespace */
"ice"                                  return 'ICE'
[-A-Za-z\u0080-\u00FF0-9 ]+            return 'STRING'

/lex

/* operator associations and precedence */

%start input

%% /* language grammar */

input
  : STRING { console.log($1) }
  ;

If I try to parse any string it succeeds, great, but if my string begins
with "ice" it fails, it says "Expecting 'STRING', got 'ICE'". I don't know
how to solve this. Isn't there a way for the parser to see that "ice" also
matches STRING, besides matching ICE?

So I tried to put the STRING matcher before the ICE matcher, now everything
works, but the problem is that I need "ice" as a special keyword in some
places, so I twisted the example a little bit:

/* lexical grammar */
%lex
%%\s+                                    /* skip whitespace */
[-A-Za-z\u0080-\u00FF0-9 ]+            return 'STRING'
"ice"                                  return 'ICE'
"."                                    return 'PT'
/lex/* operator associations and precedence */%start input%% /*
language grammar */input  : STRING { console.log($1) }
  | PT ICE iceexpr { console.log($$) }
  ;iceexpr  : STRING { $$ = "ice expr" }
  ;


now it succeeds with both "ice cream" and any other string, but it fails
with ". ice cream" (which should match with the "PT ICE iceexpr" rule). How
can I make this work? No way?

Thank you very much.

Re: [jison] Strings matching and reserved words

From:
ComFreek
Date:
2014-10-31 @ 16:43
Hello,

start conditions should be able to solve this.
See http://zaach.github.io/jison/docs/ (search for 'start conditions'), 
https://github.com/zaach/jison/issues/249

(I hope this helps, the last time I tried start conditions I couldn't 
get them to work, so I can't send you a sample code.)

Am 31.10.2014 um 13:56 schrieb Giovanni P:
> Hello, I'm totally new to this parser thing and I have a bizarre 
> grammar that is presenting me with various problems.
>
> To abstract the most important one, I made this small grammar. See if 
> you can understand my drama (it works at 
> http://zaach.github.io/jison/try/ if you just paste it there, as this 
> is what I was doing to test):
>
> /* lexical grammar */
> %lex
> %%
>
> \s+                                    /* skip whitespace */
> "ice"                                  return 'ICE'
> [-A-Za-z\u0080-\u00FF0-9 ]+        � �   return 'STRING'
>
> /lex
>
> /* operator associations and precedence */
>
> %start input
>
> %% /* language grammar */
>
> input
>   : STRING { console.log($1) }
>   ;
>
> If I try to parse any string it succeeds, great, but if my string 
> begins with "ice" it fails, it says "Expecting 'STRING', got 'ICE'". I 
> don't know how to solve this. Isn't there a way for the parser to see 
> that "ice" also matches STRING, besides matching ICE?
>
> So I tried to put the STRING matcher before the ICE matcher, now 
> everything works, but the problem is that I need "ice" as a special 
> keyword in some places, so I twisted the example a little bit:
>
> /* lexical grammar */
> %lex
> %%
>
> \s+                                    /* skip whitespace */
> [-A-Za-z\u0080-\u00FF0-9 ]+            return 'STRING'
> "ice"                                  return 'ICE'
> "."                                    return 'PT'
>
>
> /lex
>
> /* operator associations and precedence */
>
> %start input
>
> %% /* language grammar */
>
> input
>    : STRING { console.log($1) }
>    | PT ICE iceexpr { console.log($$) }
>    ;
>
> iceexpr
>    : STRING { $$ = "ice expr" }
>    ;
>
> now it succeeds with both "ice cream" and any other string, but it 
> fails with ". ice cream" (which should match with the "PT ICE iceexpr" 
> rule). How can I make this work? No way?
>
> Thank you very much.
>
>

Re: [jison] Strings matching and reserved words

From:
Giovanni P
Date:
2014-10-31 @ 19:53
Thank you, that was the solution I was looking for.

Now, just for the records, there is another alternative for doing similar
things:

http://librelist.com/browser//jison/2010/11/10/how-to-do-start-conditions/#1e75c37e66872c5b22b78c2f5dda63cc
(I only found this because I looked for "start conditions").

On Fri, Oct 31, 2014 at 2:43 PM, ComFreek <comfreek@outlook.com> wrote:

>  Hello,
>
> start conditions should be able to solve this.
> See http://zaach.github.io/jison/docs/ (search for 'start conditions'),
> https://github.com/zaach/jison/issues/249
>
> (I hope this helps, the last time I tried start conditions I couldn't get
> them to work, so I can't send you a sample code.)
>
> Am 31.10.2014 um 13:56 schrieb Giovanni P:
>
>   Hello, I'm totally new to this parser thing and I have a bizarre
> grammar that is presenting me with various problems.
>
>  To abstract the most important one, I made this small grammar. See if
> you can understand my drama (it works at http://zaach.github.io/jison/try/
> if you just paste it there, as this is what I was doing to test):
>
> /* lexical grammar */
> %lex
> %%
>
> \s+                                    /* skip whitespace */
> "ice"                                  return 'ICE'
> [-A-Za-z\u0080-\u00FF0-9 ]+        � �   return 'STRING'
>
> /lex
>
> /* operator associations and precedence */
>
> %start input
>
> %% /* language grammar */
>
> input
>   : STRING { console.log($1) }
>   ;
>
>  If I try to parse any string it succeeds, great, but if my string begins
> with "ice" it fails, it says "Expecting 'STRING', got 'ICE'". I don't
> know how to solve this. Isn't there a way for the parser to see that "ice"
> also matches STRING, besides matching ICE?
>
>  So I tried to put the STRING matcher before the ICE matcher, now
> everything works, but the problem is that I need "ice" as a special keyword
> in some places, so I twisted the example a little bit:
>
> /* lexical grammar */
> %lex
> %%\s+                                    /* skip whitespace */
> [-A-Za-z\u0080-\u00FF0-9 ]+            return 'STRING'
> "ice"                                  return 'ICE'
> "."                                    return 'PT'
> /lex/* operator associations and precedence */%start input%% /* language
grammar */input  : STRING { console.log($1) }
>   | PT ICE iceexpr { console.log($$) }
>   ;iceexpr  : STRING { $$ = "ice expr" }
>   ;
>
>
> now it succeeds with both "ice cream" and any other string, but it fails
> with ". ice cream" (which should match with the "PT ICE iceexpr" rule). How
> can I make this work? No way?
>
>  Thank you very much.
>
>
>
>

Re: [jison] Strings matching and reserved words

From:
Giovanni P
Date:
2014-10-31 @ 21:24
Ok, I'm not getting this. I've changed a lot of things in my grammar and
lexer, and it seems to be almost right (now with a sounder foundation,
using various different start conditions), but I can't get them to work
exactly, because the state never changes, my tokens that should trigger a
"this.begin" never get matched. Can someone look at this and tell me if
they spot some obvious error?

the grammar:

https://raw.githubusercontent.com/fiatjaf/vendasalva-app/586572a114da8aaa9f8542a6094645dd9a88007a/parser/day-parser.y
some test input (wrapped in some coffeescript code that uses jison to
compile the grammar, but not important):

https://github.com/fiatjaf/vendasalva-app/blob/586572a114da8aaa9f8542a6094645dd9a88007a/parser/test1.coffee


On Fri, Oct 31, 2014 at 5:53 PM, Giovanni P <fiatjaf@gmail.com> wrote:

> Thank you, that was the solution I was looking for.
>
> Now, just for the records, there is another alternative for doing similar
> things:
> 
http://librelist.com/browser//jison/2010/11/10/how-to-do-start-conditions/#1e75c37e66872c5b22b78c2f5dda63cc
> (I only found this because I looked for "start conditions").
>
> On Fri, Oct 31, 2014 at 2:43 PM, ComFreek <comfreek@outlook.com> wrote:
>
>>  Hello,
>>
>> start conditions should be able to solve this.
>> See http://zaach.github.io/jison/docs/ (search for 'start conditions'),
>> https://github.com/zaach/jison/issues/249
>>
>> (I hope this helps, the last time I tried start conditions I couldn't get
>> them to work, so I can't send you a sample code.)
>>
>> Am 31.10.2014 um 13:56 schrieb Giovanni P:
>>
>>   Hello, I'm totally new to this parser thing and I have a bizarre
>> grammar that is presenting me with various problems.
>>
>>  To abstract the most important one, I made this small grammar. See if
>> you can understand my drama (it works at
>> http://zaach.github.io/jison/try/ if you just paste it there, as this is
>> what I was doing to test):
>>
>> /* lexical grammar */
>> %lex
>> %%
>>
>> \s+                                    /* skip whitespace */
>> "ice"                                  return 'ICE'
>> [-A-Za-z\u0080-\u00FF0-9 ]+        � �   return 'STRING'
>>
>> /lex
>>
>> /* operator associations and precedence */
>>
>> %start input
>>
>> %% /* language grammar */
>>
>> input
>>   : STRING { console.log($1) }
>>   ;
>>
>>  If I try to parse any string it succeeds, great, but if my string
>> begins with "ice" it fails, it says "Expecting 'STRING', got 'ICE'". I
>> don't know how to solve this. Isn't there a way for the parser to see that
>> "ice" also matches STRING, besides matching ICE?
>>
>>  So I tried to put the STRING matcher before the ICE matcher, now
>> everything works, but the problem is that I need "ice" as a special keyword
>> in some places, so I twisted the example a little bit:
>>
>> /* lexical grammar */
>> %lex
>> %%\s+                                    /* skip whitespace */
>> [-A-Za-z\u0080-\u00FF0-9 ]+            return 'STRING'
>> "ice"                                  return 'ICE'
>> "."                                    return 'PT'
>> /lex/* operator associations and precedence */%start input%% /* 
language grammar */input  : STRING { console.log($1) }
>>   | PT ICE iceexpr { console.log($$) }
>>   ;iceexpr  : STRING { $$ = "ice expr" }
>>   ;
>>
>>
>> now it succeeds with both "ice cream" and any other string, but it fails
>> with ". ice cream" (which should match with the "PT ICE iceexpr" rule). How
>> can I make this work? No way?
>>
>>  Thank you very much.
>>
>>
>>
>>
>