The union of the values is handled by the backend.
In D, unions can hold classes, structs, etc., so this is more similar
to the C++ api.value.type variant.
* data/skeletons/d.m4, data/skeletons/lalr1.d: Here.
* tests/calc.at, tests/local.at: Test it.
When a pstate is used for multiple successive parses, some state may
leak from one run into the following one. That was introduced in
330552ea49 "yacc.c: push: don't clear
the parser state when accepting/rejecting".
Reported by Ryan <[email protected]>
https://lists.gnu.org/r/bug-bison/2021-03/msg00000.html
* data/skeletons/yacc.c (yypush_parse): We reusing a pstate from a
previous run, do behave as if it were the first run.
* tests/push.at (Pstate reuse): Check this.
The right-shift added in c22902e360
("tables: fix handling for useless tokens") is incorrect. In
particular, we need to reset the "new" bits.
Reported by Balázs Scheidler.
https://github.com/akimd/bison/issues/74
* src/tables.c (pos_set_set): Fix the right-shift.
Don't repeatedly call malloc/free for each call to map_file_name.
* bootstrap.conf: We need hash-map.
* src/files.h, src/files.c (map_file_name): The caller must not free
the result.
Adjust callers.
(mapped_dir_prefix, spec_mapped_header_file): Remove.
* src/files.c
(map_file_name): Rename as...
(map_file_name_alloc): this.
(mapped_files, map_file_name, string_equals, string_hash, string_free):
New.
* examples/c/bistromathic/parse.y, examples/c/glr/c++-types.y,
* examples/c/lexcalc/parse.y: Use const where appropriate.
Avoid `yy` prefixes where it does not make sense.
Avoid the `p` prefix for pointers.
* examples/c/bistromathic/parse.y, examples/c/glr/c++-types.y,
* examples/c/lexcalc/parse.y: Don't use the private internal detail
`YY_LOCATION_PRINT`, use `YYLOCATION_PRINT`.
It is very helpful to be able to look at the locations in the debug
traces, let's provide the users with (i) a means to do that for their
location types, and (ii) a public macro to print locations when debug
traces are enabled.
* data/skeletons/c.m4 (b4_yy_location_print_define): Rename as...
(b4_yylocation_print_define): this.
Define YYLOCATION_PRINT instead of YY_LOCATION_PRINT.
Ensure backward compatibility for those who might have defined/used
YY_LOCATION_PRINT in spite the warnings.
Adjust dependencies.
* data/skeletons/glr2.cc: We don't use YYLOCATION_PRINT here.
That change was started in Bison 3.4. The announcement for 3.7 stated
that in Bison 3.8 we would use *.gv by default.
* src/files.c (compute_output_file_names): spec_graph_file defaults
too *.gv.
* doc/bison.texi, examples/c++/calc++/local.mk, tests/output.at:
Adjust.
Its removal was annonced several times in NEWS (for Bison 3.5, 3.6, 3.7).
* data/skeletons/c.m4, data/skeletons/yacc.c: Remove support for YYPRINT.
* NEWS: Fix the mess introduced by the merge.
Document the removal of YYPRINT.
* doc/bison.texi (The YYPRINT Macro): Remove.
Using #define YYSTYPE has always been strongly discouraged in C++.
Macros are dangerous and can result in subtle bugs.
https://lists.gnu.org/r/bug-bison/2020-12/msg00007.html
Maybe some people are currently using #define YYSTYPE. Instead of
dropping support right now, first issue a warning. Bison can "see" if
YYDEBUG is defined (it could even be on the command line), only the
compiler knows. Unfortunately `#warning` is non-portable, and
actually GCC even dies on it when `-pedantic` is enabled. So we need
to use `#pragma message`. We must make it conditional as some
compilers might not support it, but it doesn't matter if only _some_
compilers emit the warning: it should be enough to catch the attention
of the developers.
* data/skeletons/c++.m4: Issue a warning when the user defined
YYSTYPE.
* tests/actions.at: Don't #define YYSTYPE.
* tests/headers.at (Several parsers): Ignore the YYSTYPE in the
warning.
We have been accepting this for years, but it is deprecated: people
are expecting to define api.value.type instead.
* doc/bison.texi: Make it clear that YYSTYPE and YYLTYPE are for C
only.
* data/skeletons/glr2.cc, data/skeletons/lalr1.cc (b4_lex): Don't
refer to YYSTYPE, but to value_type.
Note that this is "symbolic" in the sense that these values are never
used by b4_function_call.
* data/skeletons/bison.m4 (b4_union_if): New.
Remove support for "%define variant", which was deprecated long ago.
* tests/local.at (AT_UNION_IF): New.
* src/main.c (main): When traces are enabled, display the Bison
version.
* tests/conflicts.at, tests/report.at, tests/sets.at:
Use AT_PACKAGE_VERSION (for package.m4) instead of post-processing the
output.
Automake appears to not recognize "check-local" when it is not a
isolated target. Fixes 77a8b84fc8.
* tests/local.mk (check-local): Make it visible to Automake.
Currently glr2.cc does not use 'symbol's everywhere, in various places
it also uses yykind, yyval and yyloc "by hand". So we need two
different calls for user-defined constructors: once for ~symbol,
another for yy_destroy_. Both need to call the user destructors with
different calling conventions.
* data/skeletons/glr2.cc (b4_symbol_action): Rename as...
(b4_symbol_action_for_yyval): this.
(b4_symbol_action): New, taken from lalr1.cc.
(yy_destroy_, yy_symbol_value_print_): Use b4_symbol_action_for_yyval.
Three new commits:
commit 8358090292
Author: Paul Eggert <[email protected]>
Date: Wed Jan 20 18:30:16 2021 -0800
c: port to HP-UX 11.23
commit 2c294c1325
Author: Vincent Imbimbo <[email protected]>
Date: Sat Jan 23 13:25:18 2021 -0500
cex: fix state-item pruning
commit c22902e360
Author: Akim Demaille <[email protected]>
Date: Sat Jan 23 18:40:15 2021 +0100
tables: fix handling for useless tokens
In some rare conditions, the generated parser can be wrong when there
are useless tokens.
Reported by Balázs Scheidler.
https://github.com/akimd/bison/issues/72
Balázs managed to prove that the bug was introduced in
commit af1c6f973a
Author: Theophile Ranquet <[email protected]>
Date: Tue Nov 13 10:38:49 2012 +0000
tables: use bitsets for a performance boost
Suggested by Yuri at
<http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00000.html>.
The improvement is marginal for most grammars, but notable for large
grammars (e.g., PosgreSQL's postgre.y), and very large for the
sample.y grammar submitted by Yuri in
http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00012.html.
Measured with --trace=time -fsyntax-only.
parser action tables postgre.y sample.y
Before 0,129 (44%) 37,095 (99%)
After 0,117 (42%) 5,046 (93%)
* src/tables.c (pos): Replace this set of integer coded as an unsorted
array of integers with...
(pos_set): this bitset.
which was implemented long ago, but that I installed only recently
(March 2019), first published in v3.3.90.
That patch introduces a bitset to represent a set of integers. It
managed negative integers by using a (fixed) base (the smallest
integer to represent). It avoided negative accesses into the bitset
by ignoring integers smaller than the base, under the asumption that
these cases correspond to useless tokens that are ignored anyway.
While it turns out to be true for all the test cases in the test suite
(!), Balázs' use case demonstrates that it is not always the case.
So we need to be able to accept negative integers that are smaller
than the current base.
"Amusingly" enough, the aforementioned patch was visibly unsure about
itself:
/* Store PLACE into POS_SET. PLACE might not belong to the set
of possible values for instance with useless tokens. It
would be more satisfying to eliminate the need for this
'if'. */
This commit needs several improvements in the future:
- support from bitset for bit assignment and shifts
- amortized resizing of pos_set
- test cases
* src/tables.c (pos_set_base, pos_set_dump, pos_set_set, pos_set_test):
New.
Use them instead of using bitset_set and bitset_test directly.
Currently each time we meet %merge we record this location as the
defining location (and symbol). Instead, record the first definition.
In the generated code we go from
yy0->A = merge (*yy0, *yy1);
to
yy0->S = merge (*yy0, *yy1);
where S was indeed the first symbol, and in the diagnostics we go from
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type2>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:30.18-24: note: previous declaration
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
to
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type1>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
where both duplicates are reported against definition 1, rather than
using definition 1 as a reference when diagnosing about definition 2,
and then 2 as a reference for 3.
* src/reader.c (record_merge_function_type): Keep the first definition.
* tests/glr-regression.at: Adjust.
Reported by Jot Dot.
https://lists.gnu.org/r/help-bison/2020-12/msg00014.html
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): Use
the symbol's slot, not its type.
* examples/c/glr/c++-types.y: Use explicit per-symbol typing together
with api.value.type=union.
(yylex): Use yytoken_kind_t.
Don't generate C code from bison, leave that to the skeletons.
* src/output.c (merger_output): Emit invocations to b4_call_merger.
* data/skeletons/glr.c (b4_call_merger): New.
Symbols are richer than types, and in M4 it is my simpler (and more
common) to deal with symbols rather than types. So let's associate
mergers to a symbol rather than a type name.
* src/reader.h (merger_list): Replace the 'type' member by a symbol
member.
* src/reader.c (record_merge_function_type): Take a symbol as
argument, rather than a type name.
* src/output.c (merger_output): Adjust.
This macro is not exposed to users, make start it with 'YY_'.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/lalr1.cc,
* src/parse-gram.c, tests/actions.at, tests/c++.at, tests/headers.at,
* tests/local.at (YYUSE): Rename as...
(YY_USE): this.
When comparing traces from different machines, the mixture of
stdout/stderr in the output are making things uselessly difficult.
* src/lssi.c, src/state-item.c: Output debug traces on stderr.
When using glr.cc, the C function yyparse is an internal detail that
should not be exposed. Users might call it by accident (I did).
* data/skeletons/glr.c (yyparse): When used for glr.cc, rename as yy_parse_impl.
* data/skeletons/glr.cc: Adjust.
The yydefgoto table uses -1 as an invalid for an impossible case (we
never use yydefgoto[0], since it corresponds to the reduction to
$accept, which never happens). Since yydefgoto is a table of state
numbers, this -1 forces a signed type uselessly, which (1) might
trigger compiler warnings when storing a value from yydefgoto into a
state number (nonnegative), and (2) wastes bits which might result in
using a int16 where a uint8 suffices.
Reported by Jot Dot <[email protected]>.
https://lists.gnu.org/r/bug-bison/2020-11/msg00027.html
* src/tables.c (default_goto): Use 0 rather than -1 as invalid value.
* tests/regression.at: Adjust.
* data/skeletons/lalr1.d (YYLocation.step): New.
* examples/d/calc/calc.y (Lexer): Reduce scopes to avoid uninitialized
varibles.
Factor the handling of locations.
We don't need lenChars.
Suggested by Joe Nelson <[email protected]>.
https://lists.gnu.org/r/help-bison/2020-12/msg00020.html
* data/skeletons/glr.c, data/skeletons/yacc.c (YYNOMEM): New.
Use it.
(yyexhaustedlab): Rename as...
(yynomemlab): this.
* tests/calc.at: Check it.
* doc/bison.texi: Document it.
Fix incorrect statements about non-existing constants for YYERROR etc.
There are some tests that cover them, but nothing for all the
skeletons. Let's do that in the calculator tests.
* tests/calc.at: Check YYACCEPT and YYABORT.
I am not aware of people subclassing the parser class, and I fail to
see how this could be useful. Rather than leaving a badly baked
feature (as in glr.cc currently), let's not support it at all, until
someone comes and explains why and how it would be useful.
* data/skeletons/glr2.cc (parser): We need no virtual function members.
It's on purpose that I keep the `this->` now. We'll see later if we
want to remove them.
* data/skeletons/glr2.cc (yygetToken): Move into...
(glr_stack::yyget_token): this.
(b4_lex): Adjust.
Currently we have two classes that actually should be fused together:
parser and glr_stack. Both carry part of the parsing: (i) parser
contains `parse`, which is the top-level of the parsing process, it
uses yygetToken, etc., and (ii) glr_stack takes care of all the
details (dealing with the stack), and also calls yygetToken.
However if we fuse them together, we would get a large parser class,
in the header file. So it is probably better to split this large
class using the pimpl idiom. But then it appears that glr_stack is
very close from being the impl of parser.
Let's improve this.
For a start...
* data/skeletons/glr2.cc (parser::parse): Move to...
(glr_stack::parse): here.
(parser::parse): Use it.
When comparing traces from different machines, the mixture of
stdout/stderr in the output are making things uselessly difficult.
* src/lssi.c, src/state-item.c: Output debug traces on stderr.
* data/skeletons/glr2.cc: We no longer play dirty tricks with
parse-params, remove the now useless dance around them.
We now have genuine objects for locations, leave the initial action
alone.
* data/skeletons/glr2.cc: Add support for api.token.constructor.
* examples/c++/glr/c++-types.yy: Use it.
* examples/c++/glr/c++-types.test: Adjust expectations for error
messages.
* data/skeletons/c++.m4 (b4_yytranslate_define): Use static_cast
rather than the YY_CAST macro.
Avoids the need to define YY_CAST in the header.
* data/skeletons/glr2.cc: Fix calls to b4_shared_declarations: pass
the type of file we are in.
Don't define YYTRANSLATE.
(parser::yytranslate_): New, as in lalr1.cc.
Adjust to use it.
* tests/glr-regression.at: Adjust.
Instead of tracking the lookahead with yychar, use yytoken. This is
consistent with lalr1.cc, saves us from calls to YYTRANSLATE (except
when calling yylex), and makes it easier to migrate to using
symbol_type.
* data/skeletons/glr2.cc: Replace all uses of `int yychar` with
`symbol_kind_type yytoken`.
(yygetToken): Don't take/return the lookahead's token-kind and
symbol-kind, just work directly on yystack's `yytoken` member.
* tests/glr-regression.at (AT_PRINT_LOOKAHEAD_DECLARE)
(AT_PRINT_LOOKAHEAD_DEFINE): New.
Adjust to the fact that we have yytoken, not yychar, in glr2.cc.
* data/skeletons/glr2.cc (b4_symbol_action): New, so that we use
`yyval` and `yyloc` rather than the `yyvaluep` and `yylocationp`
pointers.
(yy_symbol_value_print_, yy_symbol_print_, yy_destroy_): Use
references rather than pointers.
Drop support for the undocumented, obsolete, `yyoutput` variable.
Adjust callers.
(glr_state::destroy): Don't use yy_symbol_print_ when we don't have a
symbol. Rather, write dedicated code.
Akim Demaille