librelist archives

« back to archive

couple of questions about parslet

couple of questions about parslet

From:
Dotan N.
Date:
2011-04-26 @ 11:05
Hi guys,
Many thanks for a great tool. I was able to complete (a previously written
treetop parser) in about 15 minutes!

I wanted to ask a couple of questions that naturally stem from my move from
treetop to parslet

Firstly, is there a benefit to treetop's generating code in a separate file
? or even - is there a benefit for parslet keeping it 'in memory' ? (in
terms of runtime use, not development)

Secondly, is there _any_ way to use treetop grammars with parslet? what I
turned to do was a mechanic search + replace of my treetop grammar plus
minor hand tweaks. immediately I was wondering about that.

When I finished, I realized I had two versions of the parser written with
both treetop and parslet. As a typical engineer I ran a quick benchmark --
and parslet (1.2.0) was faster by a considerable margin (perhaps i'll post
the results further down the week).


Thanks guys!

--
Dotan, @jondot

Re: [ruby.parslet] couple of questions about parslet

From:
Jason Garber
Date:
2011-04-26 @ 13:31
Very cool, Dotan. Be sure to show us the complete parser if it's open
source! I learn lots from other people's code.

I was rewriting RedCloth in Treetop but hit a wall before Parslet came out.
I wanted to do recursion with exclusion neatly, but couldn't extend Treetop
well enough to do what I wanted to.  Also, speed was horrendous. I was
seeing all these parsers, like RedCloth, Gherkin, or Hpricot, that were
written in C/Java for speed but had problems on account of being native
extensions and had to solve the same problems—cross-compilation, character
encodings, etc.—over and over. What if they could share a common native
library but themselves be just a parser definition in pure Ruby? With
generated code, that's not an option.

I was considering forking Treetop and changing it to be in-memory when Citrus
came 
out<http://groups.google.com/group/treetop-dev/browse_thread/thread/af8d889f6c6100bd>.
I didn't make time to explore it before Parslet came out and really lit my
fire. The recursion with

exclusions<https://github.com/jgarber/redcloth-parslet/blob/master/lib/redcloth-parslet/parser/inline.rb>thing
I spoke of earlier was
a 
snap<https://github.com/jgarber/redcloth-parslet/blob/master/lib/redcloth-parslet/parser/exclusions.rb>since
I mostly based it on one of Kaspar's examples. We'll see how fast or
slow Parslet is in the end, but so far it's looking good.

Yes, you can use Treetop grammars with Parslet. See

lib/parslet/expression.rb<https://github.com/kschiess/parslet/blob/master/lib/parslet/expression.rb>.
It's experimental and I'm sure there are plenty of gotchas.

Interested to see your benchmarks when you're ready to share!

Cheers,
Jason

On Tue, Apr 26, 2011 at 5:05 AM, Dotan N. <dipidi@gmail.com> wrote:
> Hi guys,
> Many thanks for a great tool. I was able to complete (a previously written
> treetop parser) in about 15 minutes!
> I wanted to ask a couple of questions that naturally stem from my move
from
> treetop to parslet
> Firstly, is there a benefit to treetop's generating code in a separate
file
> ? or even - is there a benefit for parslet keeping it 'in memory' ? (in
> terms of runtime use, not development)
> Secondly, is there _any_ way to use treetop grammars with parslet? what I
> turned to do was a mechanic search + replace of my treetop grammar plus
> minor hand tweaks. immediately I was wondering about that.
> When I finished, I realized I had two versions of the parser written with
> both treetop and parslet. As a typical engineer I ran a quick benchmark --
> and parslet (1.2.0) was faster by a considerable margin (perhaps i'll post
> the results further down the week).
>
> Thanks guys!
> --
> Dotan, @jondot
>

Re: couple of questions about parslet

From:
Kaspar Schiess
Date:
2011-04-26 @ 13:56
Hi Dotan,

Glad to hear that parslet is working fine for you!

> Secondly, is there _any_ way to use treetop grammars with parslet? what I
> turned to do was a mechanic search + replace of my treetop grammar plus
> minor hand tweaks. immediately I was wondering about that.
>
> When I finished, I realized I had two versions of the parser written with
> both treetop and parslet. As a typical engineer I ran a quick benchmark --
> and parslet (1.2.0) was faster by a considerable margin (perhaps i'll post
> the results further down the week).

I would love to see the benchmark and the results. Here are mine ;) 
http://blog.absurd.li/2011/02/02/parslet_and_its_friends.html (Somewhat 
old, but still valid. Misleading X axis, all my fault)

Like Jason mentioned, Parslet has the experimental foreign expression 
parse feature; one of the things that didn't convince me completely 
after I had built it. It's still in there, although somewhat unfinished; 
have a look at expression_spec.rb and the 'exp' method. For example:

    exp("'a' 'b')

will be equal to

   str('a') >> str('b')

I mostly use it to round-trip test the #inspect method of all parslet 
atoms. It doesn't do a lot of things Treetop grammars do, like 
predicates and .. of course action blocks. But its a start.

Parslet exports (on the other hand) neatly to citrus and treetop - 
although it beats me why you would ever want to leave us, we're so 
friendly ;)

Likewise, I think it would probably be easier to extend Treetop with 
parslet output than it is to built another treetop parser. And there is 
the matter of the small incompatibilities/ the opinionated bits about 
parslet that don't fit Treetop exactly.

So no, no direct conversion now, and probably never. Its just too easy 
to rebuild parsers; plus - most of the treetop code I've seen isn't 
tested, something I heavily recommend.

best regards,
kaspar

Re: [ruby.parslet] Re: couple of questions about parslet

From:
Dotan N.
Date:
2011-04-26 @ 14:34
Thanks for the answers guys!

I think the fact that Parslet can understand Treetop grammar is excellent
(which is my first question).

it may have been a language barrier - but i wasn't looking for a way to
convert Parslet to Treetop :)
what i was intending to ask is that i was wondering about what are the
benefit of generating code (treetop) vs keeping it in memory (parslet)

regarding the code for the parser - unfortunately i can't put it out, but it
is a parallel of a log / machine output parser - very simple.

I'll try to abstract out the details and publish something about it and
benchmarks towards weekend

thanks again! :)

On Tue, Apr 26, 2011 at 4:56 PM, Kaspar Schiess <eule@space.ch> wrote:

> Hi Dotan,
>
> Glad to hear that parslet is working fine for you!
>
> > Secondly, is there _any_ way to use treetop grammars with parslet? what I
> > turned to do was a mechanic search + replace of my treetop grammar plus
> > minor hand tweaks. immediately I was wondering about that.
> >
> > When I finished, I realized I had two versions of the parser written with
> > both treetop and parslet. As a typical engineer I ran a quick benchmark
> --
> > and parslet (1.2.0) was faster by a considerable margin (perhaps i'll
> post
> > the results further down the week).
>
> I would love to see the benchmark and the results. Here are mine ;)
> http://blog.absurd.li/2011/02/02/parslet_and_its_friends.html (Somewhat
> old, but still valid. Misleading X axis, all my fault)
>
> Like Jason mentioned, Parslet has the experimental foreign expression
> parse feature; one of the things that didn't convince me completely
> after I had built it. It's still in there, although somewhat unfinished;
> have a look at expression_spec.rb and the 'exp' method. For example:
>
>    exp("'a' 'b')
>
> will be equal to
>
>   str('a') >> str('b')
>
> I mostly use it to round-trip test the #inspect method of all parslet
> atoms. It doesn't do a lot of things Treetop grammars do, like
> predicates and .. of course action blocks. But its a start.
>
> Parslet exports (on the other hand) neatly to citrus and treetop -
> although it beats me why you would ever want to leave us, we're so
> friendly ;)
>
> Likewise, I think it would probably be easier to extend Treetop with
> parslet output than it is to built another treetop parser. And there is
> the matter of the small incompatibilities/ the opinionated bits about
> parslet that don't fit Treetop exactly.
>
> So no, no direct conversion now, and probably never. Its just too easy
> to rebuild parsers; plus - most of the treetop code I've seen isn't
> tested, something I heavily recommend.
>
> best regards,
> kaspar
>
>
>

Re: [ruby.parslet] Re: couple of questions about parslet

From:
Jonathan Rochkind
Date:
2011-04-26 @ 14:45
As a Parslet user (not developer of Parslet), I find the Parslet 
approach to be a lot easier to debug and work with than the treetop code 
generation approach.  I generally don't like generated code though.

I'm not sure I'd describe the Parlset approach as "keeping it in memory" 
exactly. Rather, Parslet is just ruby classes that parse, configured by 
a grammar.  Is the way I think of it at least.

This leads to a lot more flexibility working with Parslet, everything is 
just an ordinary ruby object that you can interact with in ordinary 
ways, including parsing on individual rules as 'root' (very useful for 
development), adding rules with methods or module inclusion, etc.   The 
Parslet approach seems like a lot less 'magic' to me working on a 
parser, there's less indirection going on.

So to me as a user of Parslet, that's the main advantage; I find the 
Parslet approach easier to work with than the treetop approach.

Apparently some benchmarks say Parslet is also faster than treetop; I 
couldn't say if this is related to the "code generation" vs "not" 
approaches or not.

On 4/26/2011 10:34 AM, Dotan N. wrote:
> Thanks for the answers guys!
>
> I think the fact that Parslet can understand Treetop grammar is 
> excellent (which is my first question).
>
> it may have been a language barrier - but i wasn't looking for a way 
> to convert Parslet to Treetop :)
> what i was intending to ask is that i was wondering about what are the 
> benefit of generating code (treetop) vs keeping it in memory (parslet)
>
> regarding the code for the parser - unfortunately i can't put it out, 
> but it is a parallel of a log / machine output parser - very simple.
>
> I'll try to abstract out the details and publish something about it 
> and benchmarks towards weekend
>
> thanks again! :)
>
> On Tue, Apr 26, 2011 at 4:56 PM, Kaspar Schiess <eule@space.ch 
> <mailto:eule@space.ch>> wrote:
>
>     Hi Dotan,
>
>     Glad to hear that parslet is working fine for you!
>
>     > Secondly, is there _any_ way to use treetop grammars with
>     parslet? what I
>     > turned to do was a mechanic search + replace of my treetop
>     grammar plus
>     > minor hand tweaks. immediately I was wondering about that.
>     >
>     > When I finished, I realized I had two versions of the parser
>     written with
>     > both treetop and parslet. As a typical engineer I ran a quick
>     benchmark --
>     > and parslet (1.2.0) was faster by a considerable margin (perhaps
>     i'll post
>     > the results further down the week).
>
>     I would love to see the benchmark and the results. Here are mine ;)
>     http://blog.absurd.li/2011/02/02/parslet_and_its_friends.html
>     (Somewhat
>     old, but still valid. Misleading X axis, all my fault)
>
>     Like Jason mentioned, Parslet has the experimental foreign expression
>     parse feature; one of the things that didn't convince me completely
>     after I had built it. It's still in there, although somewhat
>     unfinished;
>     have a look at expression_spec.rb and the 'exp' method. For example:
>
>        exp("'a' 'b')
>
>     will be equal to
>
>       str('a') >> str('b')
>
>     I mostly use it to round-trip test the #inspect method of all parslet
>     atoms. It doesn't do a lot of things Treetop grammars do, like
>     predicates and .. of course action blocks. But its a start.
>
>     Parslet exports (on the other hand) neatly to citrus and treetop -
>     although it beats me why you would ever want to leave us, we're so
>     friendly ;)
>
>     Likewise, I think it would probably be easier to extend Treetop with
>     parslet output than it is to built another treetop parser. And
>     there is
>     the matter of the small incompatibilities/ the opinionated bits about
>     parslet that don't fit Treetop exactly.
>
>     So no, no direct conversion now, and probably never. Its just too easy
>     to rebuild parsers; plus - most of the treetop code I've seen isn't
>     tested, something I heavily recommend.
>
>     best regards,
>     kaspar
>
>
>

Re: couple of questions about parslet

From:
Kaspar Schiess
Date:
2011-04-27 @ 07:21
> it may have been a language barrier - but i wasn't looking for a way to
> convert Parslet to Treetop:)
> what i was intending to ask is that i was wondering about what are the
> benefit of generating code (treetop) vs keeping it in memory (parslet)

To be honest, I was hoping someone else would try to answer this. The 
question seems innocuous enough, but it is really hard to answer.

The question really touches on the philosophical issues of compiler 
design. In terms of functionality, the two approaches are aequivalent 
(or can be made). In terms of speed, parslet may just be doing more 
work, and that means being slower. But apparently there are benchmarks 
which claim the opposite.

The easy answer would be: It just is that way. If I had to cite one 
point where the two libraries differ: Perhaps it is easier to hack 
parslet to do new things, something that a lot of people do and that I 
encourage in the experiments and examples folders of the project.

But as I am writing this, I am thinking of ways to disprove it, so let's 
stick with: there is no difference at all, its just Ruby code...

regards,
kaspar