Pluggable Backends

[dbrgn]

I searched for a parser generator library that can parse my pest grammar file with few modifications and output C++{11,17} code. Unfortunately I did not find anything.

If pest had pluggable backends and could output parser source code in various languages, that would be really fantastic and would give it yet another advantage over other parsers.

Is anything like that planned / possible?

Probably depends on #158.

[dragostis]

This is a very tricky issue. One of the points of pest is to not need an extra tool & step to generate your parser. My solution here would be to generate an optional .h file which will make pest easy to integrate with any other language as needed.

However, if need does arise and pest is constantly used in other build environments, maybe it would make sense to crate a tool that can statically compile the parser and export the necessary glue to multiple PLs (C++ headers, Ruby gems?)

[dragostis]

A good first step here would probably be to have an attribute #[export(grammar = "my_grammar.pest")] that would generate a header my_grammar.h in src/.. and its implementation. Maybe a trait the with extern "C" fn parse(...) method would be in order not unlike the current Parser trait.

@dbrgn Willing to give this a try?

[dbrgn]

I'm quite busy at the moment, so not in the next weeks. Maybe in summer, but don't wait for me on this issue :)

[Victor-Savu]

If I am not mistaken, pest already has pluggable backends. You can see pest_derive as the backend which generates rust code. It just so happens that for Rust that code doesn't go to a file, but it is hooked into the procedural macros framework, where it is picked up by the compiler.

One can always write another crate which generates code for another language. However, at the moment, pest_derive does a bit more than just generate rust code: it also parses the pest code, validates it and optimizes it. The first two steps are, in fact, going to be extracted into a different crate in #159. @dragostis Should we consider extracting the optimizer out as well so that pest_derive would only handle the code generation, which is Rust specific? I think we considered it as part of #158 but decided to leave it in pest_derive for now. We can revisit that decision since this issue provides good motivation to do so.

[dragostis]

@Victor-Savu Yes. I agree that it makes sense to separate the optimizer in its own crate. I'll work on it once #159 is merged.

[jstnlef]

On master, we've moved the optimizer into pest_meta as well. So presumably, at this point, writing another crate which generates some other output should be completely feasible. I'd like to remove the 2.0 milestone from this since creating that crate is a separate project.

[dragostis]

@jstnlef I agree. This was tagged 2.0 tentatively.