See "glr.c: log the execution of deferred actions".
* data/skeletons/glr2.cc (yyuserAction): Take yyk as a new argument.
Rename argument yyn as yyrule for clarity.
Log before and after the user action.
Adjust callers to not call YY_REDUCE_PRINT and YY_SYMBOL_PRINT.
Currently deferred reductions are not "verbose" at all: only immediate
reductions are displayed in the YYDEBUG traces. I don't understand
why. Besides it seems actually simpler the install the reduction
traces right around the user action inside yyuserAction rather that
around calls to yyuserAction.
This only trouble is that yyuserAction does not know the stack number
it works on, so we have to pass it. And pass -1 when we are actually
running on a temporary stack.
The glr example, on "T(x) + y;" as input, adds these logs, which
allow to see when the `<cast>` is built:
Stack 0 Entering state 26
Reduced stack 0 by rule 7 (line 108); action deferred. Now in state 7.
Stack 0 Entering state 7
Reading a token
Next token is token '+' (1.6: )
Stack 1 Entering state 27
Reduced stack 1 by rule 13 (line 123); action deferred. Now in state 12.
Stack 1 Entering state 12
Next token is token '+' (1.6: )
Stack 1 dies.
Removing dead stacks.
On stack 0, shifting token '+' (1.6: )
Stack 0 now in state #14
+Reducing stack -1 by rule 6 (line 107):
+ $1 = token identifier (1.3: x)
+-> $$ = nterm expr (1.3: x)
+Reducing stack -1 by rule 7 (line 108):
+ $1 = token typename (1.0: T)
+ $2 = token '(' (1.2: )
+ $3 = nterm expr (1.3: x)
+ $4 = token ')' (1.4: )
+-> $$ = nterm expr (1.0-3: <cast>(x,T))
Returning to deterministic operation.
* data/skeletons/glr.c (yyuserAction): Take yyk as a new argument.
Rename argument yyn as yyrule for clarity.
Log before and after the user action.
Adjust callers to not call YY_REDUCE_PRINT and YY_SYMBOL_PRINT.
The next commit wants to use YY_REDUCE_PRINT above its current
definition. Move it higher.
* data/skeletons/glr.c (yylhsNonterm, YY_REDUCE_PRINT): Make available
earlier.
* examples/c/glr/c++-types.y (node_print): New.
Use YY_LOCATION_PRINT instead of duplicating it.
And actually use it in the action instead of badly duplicating it.
(main): Add proper option support.
* examples/c/glr/c++-types.test: Adjust expectations on locations.
* examples/c++/glr/c++-types.yy: Fix bad iteration.
Currently each time we meet %merge we record this location as the
defining location (and symbol). Instead, record the first definition.
In the generated code we go from
yy0->A = merge (*yy0, *yy1);
to
yy0->S = merge (*yy0, *yy1);
where S was indeed the first symbol, and in the diagnostics we go from
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type2>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:30.18-24: note: previous declaration
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
to
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type1>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
where both duplicates are reported against definition 1, rather than
using definition 1 as a reference when diagnosing about definition 2,
and then 2 as a reference for 3.
* src/reader.c (record_merge_function_type): Keep the first definition.
* tests/glr-regression.at: Adjust.
Reported by Jot Dot.
https://lists.gnu.org/r/help-bison/2020-12/msg00014.html
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): Use
the symbol's slot, not its type.
* examples/c/glr/c++-types.y: Use explicit per-symbol typing together
with api.value.type=union.
(yylex): Use yytoken_kind_t.
Don't generate C code from bison, leave that to the skeletons.
* src/output.c (merger_output): Emit invocations to b4_call_merger.
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): New.
Symbols are richer than types, and in M4 it is my simpler (and more
common) to deal with symbols rather than types. So let's associate
mergers to a symbol rather than a type name.
* src/reader.h (merger_list): Replace the 'type' member by a symbol
member.
* src/reader.c (record_merge_function_type): Take a symbol as
argument, rather than a type name.
* src/output.c (merger_output): Adjust.
Now that the lookahead macros (that used yystackp) are out of the way,
there is no reason to continue using a pointer.
* data/skeletons/glr2.cc: Use yystack, a reference, rather that
yystackp, a pointer.
Fix tons of const-correctness issues.
Now that we no longer play dangerous games with macros, we can give
the lookahead's token kind its proper name. The content of yychar
_is_ raw (as opposed to yytoken), there's no reason to pleonasmicate
it (and thus to neologize).
* data/skeletons/glr2.cc (glr_stack::yyrawchar): Rename as...
(glr_stack::yychar): this.
In glr.c, the macros yychar, yylval and yylloc allow to deal with
api.pure: sometimes they point to global variables (impure), sometimes
they point to the member variables (pure).
There's no room for globals in glr2.cc. Besides, they map yychar to
yyrawchar, yylval to yyval, etc. which obfuscates what is actually
going on.
* data/skeletons/glr2.cc (glr_stack::yyval, glr_stack::yyloc): Rename
as...
(glr_stack::yylval, glr_stack::yylloc): these, for clarity.
(yynerrs, yychar, yylval, yylloc, yystackp): Remove these macros.
(b4_yygetToken_call): Remove.
Restore a more natural order: first define the macros and then use
them. Currently, some macros were defined between the moment the
header is issued, and then the implementation file. As a result, it
was possible for the header and the implementation to not use the same
versions of the macros.
* data/skeletons/glr2.cc: Define the macros first, then use them.
* data/skeletons/lalr1.cc: Minor comment and quoting changes.
* data/skeletons/glr2.cc: Define value_type and location_type where
needed, and use them only.
(yyuserMerge): Make it a member function of the glr_state class.
We always refer to the triplet "kind, value, location". All of them
are nouns, and we support api.value.type and api.location.type. On
this regard, "semantic_type" was a poor choice. Make it "value_type".
The test suite was not updated to use value_type, on purpose, to
enforce backward compatibility.
* data/skeletons/c++.m4, data/skeletons/glr.cc, data/skeletons/glr2.cc,
* data/skeletons/variant.hh, doc/bison.texi: Define value_type rather
than semantic_type.
Add a backward compatibility typedef.
* examples/c++/glr/c++-types.yy: Migrate.
* data/skeletons/glr2.cc (glr_state_set::yyremoveDeletes): Use
vector::resize rather than vector::erase.
(glr_state::copyFrom): Merge into...
(glr_state::operator=): here.
Valentin wanted each assignment to be explicit, hence copyFrom rather
that operator=. But in 0a82316e54
(glr2.cc: example: use objects (not pointers) to represent the AST),
in order to get real objects to be processed correctly, we had to
introduce the assignment operator. Afterward, we also introduced a
full implementation of the copy-ctor, independent of copyFrom. As a
result, today the only invocation of copyFrom is from the assignment
operator. Simplify this.
With GCC10, the CI shows tons of warnings such as
(327. actions.at:374: testing Initial location: glr2.cc):
input.cc: In member function 'YYRESULTTAG glr_stack::yyglrReduce(state_set_index, rule_num, bool)':
input.cc:1357:11: error: '<anonymous>.glr_state::yyloc' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1357 | yyloc = other.yyloc;
| ~~~~~~^~~~~~~~~~~~~
This is because we don't have the constructors for locations. But we
should have them! That's only because of glr.cc that ctors were not
enabled by default. In glr2.cc, they should.
That fixes all the warnings when Bison's locations are used. However,
when user-defined locations without constructor are used, we still
have:
550. calc.at:1409: testing Calculator glr2.cc %locations api.location.type={Span} ...
calc.cc: In member function 'YYRESULTTAG glr_stack::yyglrReduce(state_set_index, rule_num, bool)':
calc.cc:1261:11: error: '<anonymous>.glr_state::yyloc' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1261 | yyloc = other.yyloc;
| ~~~~~~^~~~~~~~~~~~~
To address this case, we need glr_state to explicily initialize its
yyloc member.
* data/skeletons/glr2.cc: Use genuine objects, with ctors, for position
and location.
(glr_state): Explicitly initialize yyloc in the constructors.
The copy constructor was (lazily) implemented by a call to copyFrom.
Unfortunately copyFrom reads yyresolved from the destination (and
source), and in the case of the copy-ctor this is random garbagge,
which UBSAN catches:
glr-regr2a.cc:1072:10: runtime error: load of value 7, which is not a valid value for type 'bool'
Rather than defining yyresolved before calling copyFrom, let's just
provide a genuine cpy-ctor for glr_state.
* data/skeletons/glr2.cc (glr_state::glr_state): Implement properly.
In yycompressStack:
while (yyr != YY_NULLPTR)
{
nextFreeItem->check_ ();
yyr->check_();
nextFreeItem->setState(*yyr);
glr_state& nextFreeState = nextFreeItem->getState();
yyr = yyr->pred();
nextFreeState.setPred(&(nextFreeItem - 1)->getState());
setFirstTop(&nextFreeState);
++nextFreeItem;
}
it is possible that nextFreeItem and yyr are actually the same state.
In which case `nextFreeItem->setState(*yyr)` does really bad things.
* data/skeletons/glr2.cc (glr_stack_item::setState): Beware of
self-assignment.
* data/skeletons/glr.c: A bit more doc.
(yypstates): Rename yyst (only occurrence) to yys (commonly used for
yyGLRState).
* data/skeletons/glr2.cc: Ditto.
Prefer '\n' to "\n".
The complete symbol approach in yylex removes the need for the methods
semanticVal, startPos and endPos, which were used when the values were
reported separately.
* data/skeletons/lalr1.d: Here.
* doc/bison.texi: Remove sections about the three methods.
* examples/d/calc/calc.y, examples/d/simple/calc.y: Remove the unused methods.
* tests/calc.at, tests/d.at, tests/scanner.at: Test it.
* data/skeletons/d.m4 (b4_public_types_declare): Here.
* data/skeletons/lalr1.d: Adjust.
* doc/bison.texi: Document it.
* examples/d/calc/calc.y: Use it.
* tests/calc.at: Test it.
The yychar variable was keeping the external form of the token (the
TokenKind). As the D parser translates the token to its internal
form (the SymbolKind) inside the struct Symbol, there is no need for
yychar anymore.
* data/examples/lalr1.d (yychar): Remove.
Use only yytoken.
* examples/d/calc/calc.y (start, end): Replace by this...
(location): new member variable in the Lexer class.
Use it.
* tests/calc.at: Use the defined location variable.
