todo: d: push and token ctors are done

This commit is contained in:
Akim Demaille
2021-08-04 21:42:04 +02:00
parent 93f9e527ca
commit b7d2b854be

96
TODO
View File

@@ -304,102 +304,6 @@ maintenance *simple* by avoiding any gratuitous difference.
** CI
Check when gdc and ldc.
** Token Constructors
It is possible to mix incorrectly kinds and values, and for instance:
return parser.Symbol (TokenKind.NUM, "Hello, World!\n");
attaches a string value to NUM kind (wrong, of course). When
api.token.constructor is set, in C++, Bison generated "token constructors":
parser.make_NUM. parser.make_PLUS, parser.make_STRING, etc. The previous
example becomes
return parser.make_NUM ("Hello, World!\n");
which would easily be caught by the type checker.
** Push Parser
Add support for push parser. Do not start a nice skeleton, just enhance the
current one to support push parsers. This is going to be a tougher nut to
crack.
First, you need to understand well how the push parser is expected to work.
To this end:
- read the doc
- look at examples/c/pushcalc
- create an example of a Java push parser.
- have a look at the generated parser in Java, which has the advantage of
being already based on a parser object, instead of just a function.
The C case is harder to read, but it may help too. Keep in mind that
because there's no object to maintain state, the C push parser uses some
struct (yypstate) to preserve this state. We don't need this in D, the
parser object will suffice.
I think working directly on the skeleton to add push-parser support is not
the simplest path. I suggest that you (1) transform a generated parser into
a push parser by hand, and then (2) transform lalr1.d to generate such a
parser.
Use `git commit` frequently to make sure you keep track of your progress.
*** (1.a) Prepare pull parser by hand
Copy again one of the D examples into say examples/d/pushcalc. Also
check-in the generated parser to facilitate experimentation.
- find local variables of yyparse should become members of the parser object
(so that we preserve state from one call to the next).
- do it in your generated D parser. We don't need an equivalent for
yypstate, because we already have it: that the parser object itself.
- have your *pull*-parser (i.e., the good old yy::parser::parse()) work
properly this way. Write and run tests. That's one of the reasons I
suggest using examples/d/calc as a starting point: it already has tests,
you can/should add more.
At this point you have a pull-parser which you prepared to turn into a
push-parser.
*** (1.b) Turn pull parser into push parser by hand
- look again at how push parsers are implemented in Java/C to see what needs
to change in yyparse so that the control is inverted: parse() will
be *given* the tokens, instead of having to call yylex itself. When I say
"look at C", I think your best option are (i) yacc.c (look for b4_push_if)
and (ii) examples/c/pushcalc.
- rename parse() as push_parse(Symbol yyla) (or push_parse(TokenKind, Value,
Location)) that takes the symbol as argument. That's the push parser we
are looking for.
- define a new parse() function which has the same signature as the usual
pull-parser, that repeatedly calls the push_parse function. Something
like this:
int parse ()
{
int status = 0;
do {
status = this->push_parse (yylex());
} while (status == YYPUSH_MORE);
return status;
}
- show me that parser, so that we can validate the approach.
*** (2) Port that into the skeleton
- once we agree on the API of the push parser, implement it into lalr1.d.
You will probaby need help on this regard, but imitation, again, should
help.
- have example/d/pushcalc work properly and pass tests
- add tests in the "real" test suite. Do that in tests/calc.at. I can
help.
- document
** GLR Parser
This is very ambitious. That's the final boss. There are currently no
"clean" implementation to get inspiration from.