todo: d: push and token ctors are done

This commit is contained in:
Akim Demaille
2021-08-04 21:42:04 +02:00
parent 93f9e527ca
commit b7d2b854be

96
TODO
View File

@@ -304,102 +304,6 @@ maintenance *simple* by avoiding any gratuitous difference.
** CI ** CI
Check when gdc and ldc. Check when gdc and ldc.
** Token Constructors
It is possible to mix incorrectly kinds and values, and for instance:
return parser.Symbol (TokenKind.NUM, "Hello, World!\n");
attaches a string value to NUM kind (wrong, of course). When
api.token.constructor is set, in C++, Bison generated "token constructors":
parser.make_NUM. parser.make_PLUS, parser.make_STRING, etc. The previous
example becomes
return parser.make_NUM ("Hello, World!\n");
which would easily be caught by the type checker.
** Push Parser
Add support for push parser. Do not start a nice skeleton, just enhance the
current one to support push parsers. This is going to be a tougher nut to
crack.
First, you need to understand well how the push parser is expected to work.
To this end:
- read the doc
- look at examples/c/pushcalc
- create an example of a Java push parser.
- have a look at the generated parser in Java, which has the advantage of
being already based on a parser object, instead of just a function.
The C case is harder to read, but it may help too. Keep in mind that
because there's no object to maintain state, the C push parser uses some
struct (yypstate) to preserve this state. We don't need this in D, the
parser object will suffice.
I think working directly on the skeleton to add push-parser support is not
the simplest path. I suggest that you (1) transform a generated parser into
a push parser by hand, and then (2) transform lalr1.d to generate such a
parser.
Use `git commit` frequently to make sure you keep track of your progress.
*** (1.a) Prepare pull parser by hand
Copy again one of the D examples into say examples/d/pushcalc. Also
check-in the generated parser to facilitate experimentation.
- find local variables of yyparse should become members of the parser object
(so that we preserve state from one call to the next).
- do it in your generated D parser. We don't need an equivalent for
yypstate, because we already have it: that the parser object itself.
- have your *pull*-parser (i.e., the good old yy::parser::parse()) work
properly this way. Write and run tests. That's one of the reasons I
suggest using examples/d/calc as a starting point: it already has tests,
you can/should add more.
At this point you have a pull-parser which you prepared to turn into a
push-parser.
*** (1.b) Turn pull parser into push parser by hand
- look again at how push parsers are implemented in Java/C to see what needs
to change in yyparse so that the control is inverted: parse() will
be *given* the tokens, instead of having to call yylex itself. When I say
"look at C", I think your best option are (i) yacc.c (look for b4_push_if)
and (ii) examples/c/pushcalc.
- rename parse() as push_parse(Symbol yyla) (or push_parse(TokenKind, Value,
Location)) that takes the symbol as argument. That's the push parser we
are looking for.
- define a new parse() function which has the same signature as the usual
pull-parser, that repeatedly calls the push_parse function. Something
like this:
int parse ()
{
int status = 0;
do {
status = this->push_parse (yylex());
} while (status == YYPUSH_MORE);
return status;
}
- show me that parser, so that we can validate the approach.
*** (2) Port that into the skeleton
- once we agree on the API of the push parser, implement it into lalr1.d.
You will probaby need help on this regard, but imitation, again, should
help.
- have example/d/pushcalc work properly and pass tests
- add tests in the "real" test suite. Do that in tests/calc.at. I can
help.
- document
** GLR Parser ** GLR Parser
This is very ambitious. That's the final boss. There are currently no This is very ambitious. That's the final boss. There are currently no
"clean" implementation to get inspiration from. "clean" implementation to get inspiration from.