Problem reported by James K. Lowden in:
https://lists.gnu.org/r/bison-patches/2026-07/msg00000.html
* data/skeletons/yacc.c: In C++, check that YYLTYPE is
trivially copyable if this is not already known.
But skip the check in older compilers.
* bootstrap.conf (gnulib_modules): Add mcel-prefer.
(gnulib_tool_option_extras): Avoid mbiterf.
This simplifies Bison a bit, which does not need
the more-complex mbiterf.
* src/muscle-tab.c (muscle_grow, muscle_syncline_grow)
(muscle_pair_list_grow, muscle_boundary_grow, string_decode)
(define_directive): Use char *, not char const *, for
pointers that are passed to obstack_free or are stored through.
This avoids undefined behavior in C.
* src/print-xml.c (print_html):
* src/reader.c (prepare_percent_define_front_end_variables):
Drop undocumented support for lines like ‘%define tool.xsltproc
"whatever"’, as this can cause more trouble than it cures.
Fix suggested by Paul Eggert.
* tests/atlocal.in (lf): New variable.
* tests/local.at (AT_JAVA_PARSER_CHECK): Pass a Windows-agnostic value
of line.separator to the Java runtime.
Violation of C standard detected by clang -fsanitize=undefined
with clang 19.1.7 on Fedora 41 x86-64.
* src/counterexample.c (si_bfs_free):
* src/files.c (prefix_map_free, add_prefix_map):
* src/fixits.c (fixit_cmp, fixit_free, fixits_register):
(expand_to_conflict, nonunifying_shift_path)
(search_state_free_children, search_state_free, ssb_free)
(ssb_hasher, ssb_comp, ssb_equals, visited_hasher)
(visited_comparator, ssb_append, unifying_example):
* src/lssi.c (lssi_free, lssi_hasher, lssi_comparator)
(shortest_path_from_start):
* src/parse-simulation.c (free_parse_state)
(parse_state_list_new, parser_pop):
* src/state-item.c (hash_pair_hasher, hash_pair_comparator)
(hash_pair_free, hash_pair_table_create):
Avoid undefined behavior in C, which does not allow you to cast a
function pointer to some other function type and then call it via
that other type. Instead, use functions with correct types
according to the C standard, and cast their parameters.
* src/getargs.c (xargmatch_fn): Return int const, not int, to
match what ARGMATCH_DEFINE_GROUP does. In all uses of
ARGMATCH_DEFINE_GROUP, say that they return int, to match
xargmatch_fn.
(FLAGS_ARGMATCH): Do not cast function pointer.
* src/parse-simulation.c (vc_derivation_list_append): New function.
* src/system.h (deconst): New static function.
* data/skeletons/lalr1.cc:
* data/skeletons/yacc.c:
Add YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN and
YY_IGNORE_MAYBE_UNINITIALIZED_END to pacify GCC 14,
apparently after inlining.
* src/parse-gram.y, src/scan-gram.l: Do not cast c-ctype.h function
args to unsigned char unnecessarily.
* src/parse-gram.c: Regenerate.
Copyright-paperwork-exempt: yes
Fix syntax error regarding interface inheritance of the Lexer. It
appeared when the `%code lexer` option was used.
Reported by ledaniel2.
<https://github.com/akimd/bison/issues/84>
* data/skeletons/lalr1.d: Fix syntax.
* tests/d.at: Test it.
Reported by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2022-07/msg00011.html
* bootstrap.conf: Use c_strtod, so that even in French locales "1.5"
is accepted, instead of "1,5".
* src/counterexample.c, src/state-item.c: Use xtime_t instead of
time_t, so that accuracy goes from seconds to nanoseconds.
( counterexample_init): Depend on cex.timeout rather than
$TIME_LIMIT.
* doc/bison.texi (%define Summary): Document cex.timeout.
We crashed when rules were given on YYEOF.
Reported by Han Zheng.
Fixes https://github.com/akimd/bison/issues/92.
* src/reader.c (check_and_convert_grammar): Make sure YYEOF is not
an nterm.
* tests/input.at (Rule for YYEOF): New.
When we use `b4_` or `m4_` somewhere in the input, it is escaped as
`b4@'_`/`m4@'_` so that the warning about unexpanded b4_foo/m4_foo
macros does not fire.
But in the case of muscles, the `@'` escape was not recognized, and an
assertion was triggered.
Reported by Han Zheng.
<https://github.com/akimd/bison/issues/91>
* src/muscle-tab.c (COMMON_DECODE): Handle `@'`.
* tests/skeletons.at (Suspicious sequences): Check that case.
Most often yynerrs is not used. Clang 15 now complains about such
variables. Bison itself does not compile:
```
src/parse-gram.c:1797:9: error: variable 'gram_nerrs' set but not used [-Werror,-Wunused-but-set-variable]
int yynerrs = 0;
^
src/parse-gram.c:79:25: note: expanded from macro 'yynerrs'
^
1 error generated.
```
Reported by Nikita Popov.
Fixes https://github.com/akimd/bison/issues/89.
* data/skeletons/yacc.c (yynerrs): Flag with YY_ATTRIBUTE_UNUSED.
* data/skeletons/lalr1.cc (yynerrs_): Likewise.
440-453 and 685 failed with GCC 12.1:
```
test.cc:1814:21: required from here
/opt/local/include/gcc12/c++/bits/stl_construct.h:95:14: error: noexcept-expression evaluates to 'false' because of a call to '{anonymous}::glr_stack_item::glr_stack_item(bool)' [-Werror=noexcept]
95 | noexcept(noexcept(::new((void*)0) _Tp(std::declval<_Args>()...)))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test.cc:1486:5: note: but '{anonymous}::glr_stack_item::glr_stack_item(bool)' does not throw; perhaps it should be declared 'noexcept'
1486 | glr_stack_item (bool state = true)
| ^~~~~~~~~~~~~~
```
Reported by Paul Eggert.
<https://lists.gnu.org/r/bison-patches/2022-07/msg00008.html>
* data/skeletons/glr2.cc (glr_state, glr_stack_item): Declare the
default ctors noexcept.
Many C++ tests (400-25, 440-53, 664-5, 667-76, 679-80, 682-3, 685-9,
692) failed with GCC 12.1:
```
test.cc:843:28: required from here
/opt/local/include/gcc12/c++/bits/stl_construct.h:95:14: error: noexcept-expression evaluates to 'false' because of a call to 'yy::parser::stack_symbol_type::stack_symbol_type()' [-Werror=noexcept]
95 | noexcept(noexcept(::new((void*)0) _Tp(std::declval<_Args>()...)))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test.cc:990:3: note: but 'yy::parser::stack_symbol_type::stack_symbol_type()' does not throw; perhaps it should be declared 'noexcept'
990 | parser::stack_symbol_type::stack_symbol_type ()
| ^~~~~~
```
Reported by Paul Eggert.
<https://lists.gnu.org/r/bison-patches/2022-07/msg00008.html>
* data/skeletons/lalr1.cc (stack_symbol_type::stack_symbol_type()):
Declare noexcept.
* .travis.yml (jobs):
* doc/local.mk ($(top_srcdir)/%D%/bison.help):
* tests/regression.at (Web2c Actions, Useless Tokens):
* tests/scanner.at (Token numbers: $1):
Use plain ], not \], in a POSIX regular expression,
as POSIX says the interpretation of \] is undefined.
This reverses the 2012-01-13T09:39:[email protected] patch,
as Gnulib’s fdl.texi module now says “Don't use this module!”
* bootstrap.conf (gnulib_modules): Remove fdl.
* doc/.gitignore: Don’t ignore fdl.texi.
* doc/fdl.texi: New file, copied from Gnulib.
Suuggested by Frank Heckenbach.
<https://lists.gnu.org/r/bug-bison/2022-01/msg00000.html>
* doc/bison.texi (Location Type): Explain why location's "column" are
defined vaguely.
Show tab handling in ltcalc and calc++.
* examples/c/bistromathic/parse.y: Show tab handling.
* examples/c++/calc++/calc++.test,
* examples/c/bistromathic/bistromathic.test:
Check tab handling.
Currently, occurrences of these identifiers in the user's input yield
spurious warnings.
To tell the difference between a legitimate m4_foo from the user, and
a bad m4_foo coming from a non-evaluated macro of a skeleton, escape
the user's identifiers as m4@'_foo. We already use @' as a special
sequence to be stripped from the skeleton's output.
See <https://lists.gnu.org/r/bug-bison/2021-10/msg00026.html> and
previous commit ("warnings: be less picky about occurrences of m4_/b4_
in the output").
* src/flex-scanner.h (OBSTACK_SGROW): New.
* src/output.c (output_escaped): Escape m4_ and b4_.
* src/scan-code.l: Likewise.
* src/system.h (obstack_escape): Likewise.
And rewrite as a function.
* tests/skeletons.at (Suspicious sequences): Make sure the user can
use m4_foo/b4_foo without spurious warnings.
$ cat /tmp/foo.cc
using foo = int;
foo f;
$ clang++ -Wc++11-extensions -c /tmp/foo.cc
/tmp/foo.cc:1:13: warning: alias declarations are a C++11 extension [-Wc++11-extensions]
using foo = int;
^
1 warning generated.
$ clang++ --version
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
* tests/local.at (AT_COMPILE_CXX): Make sure -std=c++11 is passed when
running glr2.cc. It may be overridden by another flag in CXXFLAGS
afterwards.
* cfg.mk: Disable sc_indent as auto indent is too invasive for now.
Enable sc_prohibit_atoi_atof, except where we don't care.
* src/location.c, src/muscle-tab.c: Use strtol instead of atoi.
To get
commit 7818455627c5e54813ac89924b8b67d0bc869146
Author: Bruno Haible <[email protected]>
Date: Fri Sep 17 22:22:50 2021 +0200
threadlib: Avoid crashes in thread-related functions on Cygwin 3.2.0.
Reported by Brian Inglis via Akim Demaille in
<https://lists.gnu.org/archive/html/bug-gnulib/2021-09/msg00063.html>.
* m4/threadlib.m4 (gl_WEAK_SYMBOLS): Force a "guessing no" result on
Cygwin.
Suggested by Don Macpherson.
<https://github.com/akimd/bison/issues/80>
* data/skeletons/c++.m4, data/skeletons/glr2.cc,
* data/skeletons/lalr1.cc, data/skeletons/stack.hh: Use YY_NOEXCEPT
where it helps constructors.
These operators were introduced in "c++: add move assignments to the
symbol type" (fdaedc780a) for glr2.cc.
* data/skeletons/c++.m4: Define them for glr2.cc only.
* data/skeletons/glr2.cc (semantic_option): Simplify the rule-based
constructor. This shows that it should be easy to use a symbol_kind,
instead of the kind/value/location triple.
I regret that %destructor and %printer give access to the %parse-param
in lalr1.cc, since that prevents them from being implemented as a
simple destructor and operator<<. Let's not repeat the mistake in
glr2.cc. In addition, fixes a name conflict we have currently in
tests 566 568 570 657:
calc.cc:1395:85: error: declaration shadows a field of 'calc::parser' [-Werror,-Wshadow]
void glr_state::destroy (char const* yymsg, calc::parser& yyparser, semantic_value *result, int *count, int *nerrs)
^
calc.hh:441:21: note: previous declaration is here
semantic_value *result;
^
With this commit, the whole test suite passes for glr2.cc.
* data/skeletons/glr2.cc (glr_state::destroy): Don't take the user
arguments.
Fixes several calc tests.
Tests 566 568 570 657 still fail because of a name clash when using
%parse-param:
calc.cc:1395:85: error: declaration shadows a field of 'calc::parser' [-Werror,-Wshadow]
void glr_state::destroy (char const* yymsg, calc::parser& yyparser, semantic_value *result, int *count, int *nerrs)
^
calc.hh:441:21: note: previous declaration is here
semantic_value *result;
^
* data/skeletons/glr2.cc: Fix indentation/trailing spaces.
In order to be able to link several glr2.cc parser together, we cannot
have glr_stack and glr_state be in no namespace. Putting them in the
unnamed namespace is not doable, since we need to fwd declare them in
the parser. Putting them in the specified namespace is not enough
either, since some users would like to be able to put several parsers
in the same name, only differing by the class name.
* data/skeletons/glr2.cc (glr_state, glr_stack): Move into yy::parser.
If we link several glr2.cc parsers together, we get linking failures
because of duplicate symbols.
* data/skeletons/glr2.cc (semantic_option::indexIn)
(semantic_option::next): Remove the useless overloads.
We don't need them in the header file.
* data/skeletons/glr2.cc (YY_EXCEPTIONS): Define only in the
implementation file.
* tests/headers.at (Several Parsers): Also check glr2.cc.
Let's use c++/glr to demonstrate custom error messages in C++ (not
just in glr2.cc).
* examples/c++/glr/c++-types.yy (report_syntax_error): New.
* examples/c++/glr/c++-types.test: Adjust.
* examples/c/bistromathic/parse.y: Comment changes.
* tests/local.at (AT_YYERROR_DEFINE(c++)): Use a nicer way to print
the lookakead's name.
Currently glr2.cc uses three variables/struct members to denote the
symbols' kind (or state), value and location. lalr1.cc has two types
for "complete" symbols: symbol_type and stack_symbol_type. Let's use
that model in glr2.cc too.
For a start use yyla (a symbol_type) to denote the lookahead, instead
of the triple yytoken, yylval and yylloc. This will make easier the
introduction of the "context" subclass, used in parse.error=custom.
It simplifies the code in several places. For instance from:
symbol_kind_type yytoken_current = this->yytoken;]b4_variant_if([[
value_type yylval_current;
]b4_symbol_variant([this->yytoken],
[yylval_current], [move], [this->yylval])], [[
value_type yylval_current = this->yylval;]])[]b4_locations_if([
location_type yylloc_current = this->yylloc;])[
to:
symbol_type yyla_current = std::move (this->yyla);
* data/skeletons/glr2.cc (yytoken, yylval, yylloc): Replace by...
(yyla): this.
Adjust all dependencies.
(yyloc_default): Remove, unused.
* tests/c++.at, tests/glr-regression.at, tests/types.at: C++11 is
required for glr2.cc.
Adjust to changes in glr2.cc.
* data/skeletons/c++.m4: Don't define obsolete identifiers in the case
of glr2.cc. Let's not start with technical debt.
* data/skeletons/glr2.cc, data/skeletons/lalr1.cc,
* data/skeletons/variant.hh: Use token_kind_type, not token_type.
* tests/c++.at, tests/local.at: Use value_type, not semantic_type.
It is not available on HP-UX 11i.
Reported by Larkin Nickle <[email protected]>.
<https://lists.gnu.org/r/bug-bison/2021-09/msg00012.html>
Gnulib provides no replacement, but anyway we should be using doubles,
since difftime uses doubles.
* bootstrap.conf: We want portability on stdtod.
* src/counterexample.c: Use double, not float, for time measurements.
The recent changes to comply with POSIX are breaking Automake's test
suite.
Reported by Kiyoshi Kanazawa.
<https://lists.gnu.org/r/bug-bison/2021-09/msg00005.html>
To limit the impact of POSIX changes, bind them to $POSIXLY_CORRECT.
Suggested by Karl Berry.
<https://lists.gnu.org/r/bug-bison/2021-09/msg00009.html>
The existing `maintainer-check-posix` Make target checks these
changes.
* src/getargs.h, src/getargs.c (set_yacc): New.
Use it.
* data/skeletons/bison.m4 (b4_posix_if): New.
* data/skeletons/yacc.c (b4_declare_yyerror_and_yylex): Use it.
* doc/bison.texi, tests/local.at: Adjust.
For some reason this test fails on Solaris/x86. But multistart is not
part of 3.8, so we can postpone the debugging of this issue.
Reported by Dagobert Michelsen.
<https://lists.gnu.org/r/bug-bison/2021-08/msg00027.html>
* tests/report.at (Multistart reports): Comment out.
Contrary to what commit d8cc6b073e "c++:
shorten the assertions that check whether tokens are correct"
believed, MS Visual C++'s preprocessor limitation is not on the input
line length, but on the size of the line holding the full C++
statement.
Reported by Vince Huffaker <[email protected]>
<https://lists.gnu.org/r/help-bison/2021-08/msg00003.html>
* data/skeletons/variant.hh (_b4_symbol_constructor_define): Hide the
assertion from Visual C++.
On Solaris, sed throws away the NUL bytes from the stream, even in C
locale. So instead of postprocessing bison's stderr to neutralize
changes in value of `argv[0]`, use an envvar to actually neutralize
variations of `argv[0]` during tests.
Reported by Dagobert Michelsen.
<https://lists.gnu.org/r/bug-bison/2021-08/msg00025.html>
* src/main.c (main): Change `argv[0]` if BISON_PROGRAM_NAME is
defined.
* tests/bison.in: No longer mess with stderr, just pass
the expected BISON_PROGRAM_NAME value.
Besides, for mysterious reasons, this fails on some environment.
Reported by Dagobert Michelsen.
<https://lists.gnu.org/archive/html/bug-bison/2021-08/msg00008.html>
* examples/c/lexcalc/lexcalc.test, examples/c/lexcalc/parse.y,
* examples/c/lexcalc/scan.l: Revert to a single-start example.
Reported by Dagobert Michelsen.
https://lists.gnu.org/r/bug-bison/2021-08/msg00006.html
* m4/bison-cxx-std.m4 (_BISON_CXXSTD_98_snippet): We don't need
vector::data, it was only for glr2.cc, which is C++11 anyway.
(_BISON_CXXSTD_11_snippet): We need vector::data and std::swap on
arrays.
* m4/cxx.m4 (BISON_TEST_FOR_WORKING_CXX_COMPILER): We don't need
vector::data.
* tests/local.at (AT_COMPILE_CXX): Skip when glr2.cc and no support
for C++11.
In c2ba260487 ("glr.c: fix signature
when using custom error messages"), I meant to add a test case for C,
not C++. It does not work in C++.
* tests/calc.at: Run for glr.c, not glr.cc.
GCC 4.2 on macOS claims to support C++98, but does not feature it.
input.cc: In member function 'void state_stack::yycompressStack()':
input.cc:1774: error: 'class std::vector<glr_stack_item, std::allocator<glr_stack_item> >' has no member named 'data'
Reported by Christopher Nielsen <[email protected]>.
<https://trac.macports.org/raw-attachment/ticket/59927/bison-test-results-20210811-95b72.log.xz>.
* m4/bison-cxx-std.m4 (_BISON_CXXSTD_98_snippet): Check for it.
* m4/cxx.m4 (BISON_TEST_FOR_WORKING_CXX_COMPILER): Ditto.
* tests/atlocal.in: It was an error for tests to depend on gnulib:
they must not, as gnulib would hide portability issues that we want to
catch. So this piece of code is no longer useful, and must not be
useful.
In POSIX Yacc mode, declare yyerror and yylex unless already #defined,
or if YYERROR_IS_DECLARED/YYLEX_IS_DECLARED are defined (for
consistency with Bison's YYSTYPE_IS_DECLARED/YYLTYPE_IS_DECLARED).
See <https://austingroupbugs.net/view.php?id=1388#c5220>.
* data/skeletons/c.m4 (b4_function_declare): Resurect.
(b4_lex_formals): Since we will possibly expose this prototype
in the header, take the prefix into account.
* data/skeletons/yacc.c (b4_declare_yyerror_and_yylex): New.
(b4_shared_declarations): Use it.
* tests/local.at (AT_YACC_IF): New.
When in Yacc mode, set the `yacc` Autotest keyword.
(AT_YYERROR_DECLARE(c)): Don't declare in Yacc mode,
to avoid clashes (since this signature is static).
(AT_YYERROR_DEFINE(c)): Don't define as static in Yacc mode.
* tests/regression.at (Early token definitions with --yacc): Specify
that we are in Yacc mode.
Unfortunately it seems to be quite difficult to have "LAC: Exploratory
stack" run for D.
* data/skeletons/lalr1.d: We need File when traces are enabled.
* tests/local.at (AT_YYLEX_DEFINE(d)): New.
* tests/regression.at: Prepare for D, but don't run it, it does not
work.
* tests/local.at (AT_FOR_EACH_SKEL): New.
Use where appropriate.
* data/skeletons/lalr1.d: Reject -d.
* tests/input.at, tests/scanner.at: Also check D.
Check in m4's output if there are sequences such as m4_foo or b4_foo,
which are probably resulting from incorrect m4 processing.
It actually already is useful:
- it caught a leaking b4_lac_if leaking from glr.c, where LAC is not
supported, hence b4_lac_if is not defined.
- it also caught references to location.hh in position.hh when
location.hh does not exist.
- while making "Code injection" robust to these new warnings (it is
its very purpose to let b4_canary pass unevaluated), I saw that it
did not check lalr1.d, and when adding lalr1.d, it revealed it did
underquote ocurrences of token value types.
* src/scan-skel.l (macro): New abbreviation.
Use it.
* data/skeletons/glr.c: Don't use b4_lac_if, we don't have it.
* data/skeletons/location.cc: Don't generate position.hh when we don't
generate location.hh.
* data/skeletons/d.m4 (b4_basic_symbol_constructor_define): Fix
underquotation.
* data/skeletons/bison.m4 (b4_canary): New.
* tests/input.at (Code injection): Use it, and check lalr1.d too.
* examples/c++/Makefile, examples/c++/calc++/Makefile,
* examples/c++/glr/Makefile, examples/c/bistromathic/Makefile,
* examples/c/calc/Makefile, examples/c/glr/Makefile,
* examples/c/lexcalc/Makefile, examples/c/mfcalc/Makefile,
* examples/c/pushcalc/Makefile, examples/c/reccalc/Makefile,
* examples/c/rpcalc/Makefile, examples/d/calc/Makefile,
* examples/d/simple/Makefile, examples/java/calc/Makefile,
* examples/java/simple/Makefile:
Use --html to generate *.html directly.
No longer demonstrate --xml.
No longer show rules for xml to html.
Use --header, not --defines.
Use --graph without specifying the output file now that we
generate *.gv by default.
We had:
```
-mbchar ...|\xF0[\x\90-\xBF]([\x80-\xBF]{2})|...
+mbchar ...|\xF0[\x90-\xBF]([\x80-\xBF]{2})|...
```
so a precise sequence that matches the incorrect regex can let NUL
bytes pass through, which triggers an assertion violation downstream.
It is a pity that Flex does not report an error for such input.
Reported by Ahcheong Lee <[email protected]>.
<https://lists.gnu.org/r/bug-bison/2021-04/msg00003.html>
* src/scan-gram.l (mbchar): Fix the bad regex.
* tests/input.at (Invalid inputs): Check that case.
* doc/bison.texi: Use @samp{...}, not "..." for pieces of code.
Use @samp{...}, not @command{...} for command lines.
Promote %header/--header over %defines/--defines.
Spellcheck.
Support the push-pull directive with the options pull, push and both.
Pull remains the default option.
* data/skeletons/d.m4: Add user aliases for the push parser's return
values: PUSH_MORE, ABORT, ACCEPT.
* data/skeletons/lalr1.d: Add push parser support.
* tests/calc.at: Test it.
The user can return from yylex() by calling the Symbol method of the
same name as the TokenKind reported, and adding the parameters for
value and location if necessary. These methods generate compile-time
errors if the parameters are not correlated. Token constructors work
with both %union and api.value.type union.
* data/skeletons/d.m4: Here.
* tests/calc.at: Test it.
The union of the values is handled by the backend.
In D, unions can hold classes, structs, etc., so this is more similar
to the C++ api.value.type variant.
* data/skeletons/d.m4, data/skeletons/lalr1.d: Here.
* tests/calc.at, tests/local.at: Test it.
When a pstate is used for multiple successive parses, some state may
leak from one run into the following one. That was introduced in
330552ea49 "yacc.c: push: don't clear
the parser state when accepting/rejecting".
Reported by Ryan <[email protected]>
https://lists.gnu.org/r/bug-bison/2021-03/msg00000.html
* data/skeletons/yacc.c (yypush_parse): We reusing a pstate from a
previous run, do behave as if it were the first run.
* tests/push.at (Pstate reuse): Check this.
The right-shift added in c22902e360
("tables: fix handling for useless tokens") is incorrect. In
particular, we need to reset the "new" bits.
Reported by Balázs Scheidler.
https://github.com/akimd/bison/issues/74
* src/tables.c (pos_set_set): Fix the right-shift.
Don't repeatedly call malloc/free for each call to map_file_name.
* bootstrap.conf: We need hash-map.
* src/files.h, src/files.c (map_file_name): The caller must not free
the result.
Adjust callers.
(mapped_dir_prefix, spec_mapped_header_file): Remove.
* src/files.c
(map_file_name): Rename as...
(map_file_name_alloc): this.
(mapped_files, map_file_name, string_equals, string_hash, string_free):
New.
Applies the file name mapping before exporting it as a symbol. This
allows the symbols to correctly respect the --file-prefix-map command
line option.
Signed-off-by: Joshua Watt <[email protected]>
* examples/c/bistromathic/parse.y, examples/c/glr/c++-types.y,
* examples/c/lexcalc/parse.y: Use const where appropriate.
Avoid `yy` prefixes where it does not make sense.
Avoid the `p` prefix for pointers.
* examples/c/bistromathic/parse.y, examples/c/glr/c++-types.y,
* examples/c/lexcalc/parse.y: Don't use the private internal detail
`YY_LOCATION_PRINT`, use `YYLOCATION_PRINT`.
It is very helpful to be able to look at the locations in the debug
traces, let's provide the users with (i) a means to do that for their
location types, and (ii) a public macro to print locations when debug
traces are enabled.
* data/skeletons/c.m4 (b4_yy_location_print_define): Rename as...
(b4_yylocation_print_define): this.
Define YYLOCATION_PRINT instead of YY_LOCATION_PRINT.
Ensure backward compatibility for those who might have defined/used
YY_LOCATION_PRINT in spite the warnings.
Adjust dependencies.
* data/skeletons/glr2.cc: We don't use YYLOCATION_PRINT here.
That change was started in Bison 3.4. The announcement for 3.7 stated
that in Bison 3.8 we would use *.gv by default.
* src/files.c (compute_output_file_names): spec_graph_file defaults
too *.gv.
* doc/bison.texi, examples/c++/calc++/local.mk, tests/output.at:
Adjust.
Its removal was annonced several times in NEWS (for Bison 3.5, 3.6, 3.7).
* data/skeletons/c.m4, data/skeletons/yacc.c: Remove support for YYPRINT.
* NEWS: Fix the mess introduced by the merge.
Document the removal of YYPRINT.
* doc/bison.texi (The YYPRINT Macro): Remove.
Using #define YYSTYPE has always been strongly discouraged in C++.
Macros are dangerous and can result in subtle bugs.
https://lists.gnu.org/r/bug-bison/2020-12/msg00007.html
Maybe some people are currently using #define YYSTYPE. Instead of
dropping support right now, first issue a warning. Bison can "see" if
YYDEBUG is defined (it could even be on the command line), only the
compiler knows. Unfortunately `#warning` is non-portable, and
actually GCC even dies on it when `-pedantic` is enabled. So we need
to use `#pragma message`. We must make it conditional as some
compilers might not support it, but it doesn't matter if only _some_
compilers emit the warning: it should be enough to catch the attention
of the developers.
* data/skeletons/c++.m4: Issue a warning when the user defined
YYSTYPE.
* tests/actions.at: Don't #define YYSTYPE.
* tests/headers.at (Several parsers): Ignore the YYSTYPE in the
warning.
We have been accepting this for years, but it is deprecated: people
are expecting to define api.value.type instead.
* doc/bison.texi: Make it clear that YYSTYPE and YYLTYPE are for C
only.
* data/skeletons/glr2.cc, data/skeletons/lalr1.cc (b4_lex): Don't
refer to YYSTYPE, but to value_type.
Note that this is "symbolic" in the sense that these values are never
used by b4_function_call.
* data/skeletons/bison.m4 (b4_union_if): New.
Remove support for "%define variant", which was deprecated long ago.
* tests/local.at (AT_UNION_IF): New.
* src/main.c (main): When traces are enabled, display the Bison
version.
* tests/conflicts.at, tests/report.at, tests/sets.at:
Use AT_PACKAGE_VERSION (for package.m4) instead of post-processing the
output.
Automake appears to not recognize "check-local" when it is not a
isolated target. Fixes 77a8b84fc8.
* tests/local.mk (check-local): Make it visible to Automake.
Currently glr2.cc does not use 'symbol's everywhere, in various places
it also uses yykind, yyval and yyloc "by hand". So we need two
different calls for user-defined constructors: once for ~symbol,
another for yy_destroy_. Both need to call the user destructors with
different calling conventions.
* data/skeletons/glr2.cc (b4_symbol_action): Rename as...
(b4_symbol_action_for_yyval): this.
(b4_symbol_action): New, taken from lalr1.cc.
(yy_destroy_, yy_symbol_value_print_): Use b4_symbol_action_for_yyval.
Three new commits:
commit 8358090292
Author: Paul Eggert <[email protected]>
Date: Wed Jan 20 18:30:16 2021 -0800
c: port to HP-UX 11.23
commit 2c294c1325
Author: Vincent Imbimbo <[email protected]>
Date: Sat Jan 23 13:25:18 2021 -0500
cex: fix state-item pruning
commit c22902e360
Author: Akim Demaille <[email protected]>
Date: Sat Jan 23 18:40:15 2021 +0100
tables: fix handling for useless tokens
In some rare conditions, the generated parser can be wrong when there
are useless tokens.
Reported by Balázs Scheidler.
https://github.com/akimd/bison/issues/72
Balázs managed to prove that the bug was introduced in
commit af1c6f973a
Author: Theophile Ranquet <[email protected]>
Date: Tue Nov 13 10:38:49 2012 +0000
tables: use bitsets for a performance boost
Suggested by Yuri at
<http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00000.html>.
The improvement is marginal for most grammars, but notable for large
grammars (e.g., PosgreSQL's postgre.y), and very large for the
sample.y grammar submitted by Yuri in
http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00012.html.
Measured with --trace=time -fsyntax-only.
parser action tables postgre.y sample.y
Before 0,129 (44%) 37,095 (99%)
After 0,117 (42%) 5,046 (93%)
* src/tables.c (pos): Replace this set of integer coded as an unsorted
array of integers with...
(pos_set): this bitset.
which was implemented long ago, but that I installed only recently
(March 2019), first published in v3.3.90.
That patch introduces a bitset to represent a set of integers. It
managed negative integers by using a (fixed) base (the smallest
integer to represent). It avoided negative accesses into the bitset
by ignoring integers smaller than the base, under the asumption that
these cases correspond to useless tokens that are ignored anyway.
While it turns out to be true for all the test cases in the test suite
(!), Balázs' use case demonstrates that it is not always the case.
So we need to be able to accept negative integers that are smaller
than the current base.
"Amusingly" enough, the aforementioned patch was visibly unsure about
itself:
/* Store PLACE into POS_SET. PLACE might not belong to the set
of possible values for instance with useless tokens. It
would be more satisfying to eliminate the need for this
'if'. */
This commit needs several improvements in the future:
- support from bitset for bit assignment and shifts
- amortized resizing of pos_set
- test cases
* src/tables.c (pos_set_base, pos_set_dump, pos_set_set, pos_set_test):
New.
Use them instead of using bitset_set and bitset_test directly.
There were several bugs in pruning that would leave the state-item
graph in an inconsistent state which could cause crashes later on:
- Pruning now happens in one pass instead of two.
- Disabled state-items no longer prune the state-items they transition
to if that state-item has other states that transition to it.
- State-items that transition to disabled state-items are always
pruned even if they have productions.
Reported by Michal Bartkowiak <[email protected]>
https://lists.gnu.org/r/bug-bison/2021-01/msg00000.html
and Zartaj Majeed
https://github.com/akimd/bison/issues/71
* src/state-item.c (prune_forward, prune_backward): Fuse into...
(prune_state_item): this.
Adjust callers.
Currently each time we meet %merge we record this location as the
defining location (and symbol). Instead, record the first definition.
In the generated code we go from
yy0->A = merge (*yy0, *yy1);
to
yy0->S = merge (*yy0, *yy1);
where S was indeed the first symbol, and in the diagnostics we go from
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type2>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:30.18-24: note: previous declaration
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
to
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type1>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
where both duplicates are reported against definition 1, rather than
using definition 1 as a reference when diagnosing about definition 2,
and then 2 as a reference for 3.
* src/reader.c (record_merge_function_type): Keep the first definition.
* tests/glr-regression.at: Adjust.
Reported by Jot Dot.
https://lists.gnu.org/r/help-bison/2020-12/msg00014.html
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): Use
the symbol's slot, not its type.
* examples/c/glr/c++-types.y: Use explicit per-symbol typing together
with api.value.type=union.
(yylex): Use yytoken_kind_t.
Don't generate C code from bison, leave that to the skeletons.
* src/output.c (merger_output): Emit invocations to b4_call_merger.
* data/skeletons/glr.c (b4_call_merger): New.
Symbols are richer than types, and in M4 it is my simpler (and more
common) to deal with symbols rather than types. So let's associate
mergers to a symbol rather than a type name.
* src/reader.h (merger_list): Replace the 'type' member by a symbol
member.
* src/reader.c (record_merge_function_type): Take a symbol as
argument, rather than a type name.
* src/output.c (merger_output): Adjust.
This macro is not exposed to users, make start it with 'YY_'.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/lalr1.cc,
* src/parse-gram.c, tests/actions.at, tests/c++.at, tests/headers.at,
* tests/local.at (YYUSE): Rename as...
(YY_USE): this.
When comparing traces from different machines, the mixture of
stdout/stderr in the output are making things uselessly difficult.
* src/lssi.c, src/state-item.c: Output debug traces on stderr.
When using glr.cc, the C function yyparse is an internal detail that
should not be exposed. Users might call it by accident (I did).
* data/skeletons/glr.c (yyparse): When used for glr.cc, rename as yy_parse_impl.
* data/skeletons/glr.cc: Adjust.
The yydefgoto table uses -1 as an invalid for an impossible case (we
never use yydefgoto[0], since it corresponds to the reduction to
$accept, which never happens). Since yydefgoto is a table of state
numbers, this -1 forces a signed type uselessly, which (1) might
trigger compiler warnings when storing a value from yydefgoto into a
state number (nonnegative), and (2) wastes bits which might result in
using a int16 where a uint8 suffices.
Reported by Jot Dot <[email protected]>.
https://lists.gnu.org/r/bug-bison/2020-11/msg00027.html
* src/tables.c (default_goto): Use 0 rather than -1 as invalid value.
* tests/regression.at: Adjust.
* data/skeletons/lalr1.d (YYLocation.step): New.
* examples/d/calc/calc.y (Lexer): Reduce scopes to avoid uninitialized
varibles.
Factor the handling of locations.
We don't need lenChars.
Suggested by Joe Nelson <[email protected]>.
https://lists.gnu.org/r/help-bison/2020-12/msg00020.html
* data/skeletons/glr.c, data/skeletons/yacc.c (YYNOMEM): New.
Use it.
(yyexhaustedlab): Rename as...
(yynomemlab): this.
* tests/calc.at: Check it.
* doc/bison.texi: Document it.
Fix incorrect statements about non-existing constants for YYERROR etc.
There are some tests that cover them, but nothing for all the
skeletons. Let's do that in the calculator tests.
* tests/calc.at: Check YYACCEPT and YYABORT.
I am not aware of people subclassing the parser class, and I fail to
see how this could be useful. Rather than leaving a badly baked
feature (as in glr.cc currently), let's not support it at all, until
someone comes and explains why and how it would be useful.
* data/skeletons/glr2.cc (parser): We need no virtual function members.
It's on purpose that I keep the `this->` now. We'll see later if we
want to remove them.
* data/skeletons/glr2.cc (yygetToken): Move into...
(glr_stack::yyget_token): this.
(b4_lex): Adjust.
Currently we have two classes that actually should be fused together:
parser and glr_stack. Both carry part of the parsing: (i) parser
contains `parse`, which is the top-level of the parsing process, it
uses yygetToken, etc., and (ii) glr_stack takes care of all the
details (dealing with the stack), and also calls yygetToken.
However if we fuse them together, we would get a large parser class,
in the header file. So it is probably better to split this large
class using the pimpl idiom. But then it appears that glr_stack is
very close from being the impl of parser.
Let's improve this.
For a start...
* data/skeletons/glr2.cc (parser::parse): Move to...
(glr_stack::parse): here.
(parser::parse): Use it.
When comparing traces from different machines, the mixture of
stdout/stderr in the output are making things uselessly difficult.
* src/lssi.c, src/state-item.c: Output debug traces on stderr.
* data/skeletons/glr2.cc: We no longer play dirty tricks with
parse-params, remove the now useless dance around them.
We now have genuine objects for locations, leave the initial action
alone.
* data/skeletons/glr2.cc: Add support for api.token.constructor.
* examples/c++/glr/c++-types.yy: Use it.
* examples/c++/glr/c++-types.test: Adjust expectations for error
messages.
* data/skeletons/c++.m4 (b4_yytranslate_define): Use static_cast
rather than the YY_CAST macro.
Avoids the need to define YY_CAST in the header.
* data/skeletons/glr2.cc: Fix calls to b4_shared_declarations: pass
the type of file we are in.
Don't define YYTRANSLATE.
(parser::yytranslate_): New, as in lalr1.cc.
Adjust to use it.
* tests/glr-regression.at: Adjust.
Instead of tracking the lookahead with yychar, use yytoken. This is
consistent with lalr1.cc, saves us from calls to YYTRANSLATE (except
when calling yylex), and makes it easier to migrate to using
symbol_type.
* data/skeletons/glr2.cc: Replace all uses of `int yychar` with
`symbol_kind_type yytoken`.
(yygetToken): Don't take/return the lookahead's token-kind and
symbol-kind, just work directly on yystack's `yytoken` member.
* tests/glr-regression.at (AT_PRINT_LOOKAHEAD_DECLARE)
(AT_PRINT_LOOKAHEAD_DEFINE): New.
Adjust to the fact that we have yytoken, not yychar, in glr2.cc.
* data/skeletons/glr2.cc (b4_symbol_action): New, so that we use
`yyval` and `yyloc` rather than the `yyvaluep` and `yylocationp`
pointers.
(yy_symbol_value_print_, yy_symbol_print_, yy_destroy_): Use
references rather than pointers.
Drop support for the undocumented, obsolete, `yyoutput` variable.
Adjust callers.
(glr_state::destroy): Don't use yy_symbol_print_ when we don't have a
symbol. Rather, write dedicated code.
The locations are actually false: they should include the location of
the semicolon (we print statements), but they don't.
* examples/c++/glr/c++-types.test, examples/c++/glr/c++-types.yy,
* examples/c/glr/c++-types.test, examples/c/glr/c++-types.y,
* tests/cxx-type.at:
Fix locations and adjust expectations.
This will be useful to support changes in glr2.cc.
tests/glr-regression.at
(Incorrect lookahead during deterministic GLR)
(Incorrect lookahead during nondeterministc GLR):
Introduce and use PRINT_LOOKAHEAD.
Without the history, D should not support this option. Before the
removal, 'detailed' and 'verbose' options generated the same code.
* data/skeletons/lalr1.d: Here.
* doc/bison.texi: Adapt tests to use 'detailed' instead of 'verbose'.
* tests/calc.at: Document it.
Currently we display the addresses of the semantic values. Instead,
print the values.
Add support for YY_USE across languages.
* data/skeletons/c.m4, data/skeletons/d.m4 (b4_use): New.
* data/skeletons/bison.m4 (b4_symbol_actions): Use b4_use to be
portable to D.
Add support for %printer, and use it.
* data/skeletons/d.m4: Instead of duplicating what's already in
c-like.m4, include it.
(b4_symbol_action): New.
Differs from the one in bison.m4 in that it uses yyval/yyloc instead
of *yyvaluep and *yylocationp.
* data/skeletons/lalr1.d (yy_symbol_print): Avoid calls to formatting,
just call write directly.
Use the %printer.
* examples/d/calc/calc.y: Specify a printer.
Enable traces when $YYDEBUG is set.
* tests/calc.at: Fix the use of %printer with D.
The D parser implements this feature similarly to the C parser,
by using Gettext. Functions gettext() and dgettext() are
imported using extern(C). The internationalisation uses yysymbol_name
to report the name of the SymbolKinds.
* data/skeletons/d.m4 (SymbolKind.toString.yytranslatable,
SymbolKind.toString.yysymbol_name: New), data/skeletons/lalr1.d: Here.
* doc/bison.texi: Document it.
* tests/calc.at: Test it.
(Bison) Variants are extremely picky, which makes them both
annoying (lots of micro-details must be taken care of) and
precious (all the micro-details must be taken care of, in particular
object lifetime).
So (i) each time a semantic value is stored, it must be stored in a
place that exists, and (ii) each time a semantic value is discarded,
its place must have been emptied.
Example of (i)
- new (&yys.value ()) value_type (s->value ());
+ {]b4_variant_if([[
+ new (&yys.value ()) value_type ();
+ ]b4_symbol_variant([yy_accessing_symbol (s->yylrState)],
+ [yys.value ()], [copy], [s->value ()])], [[
+ new (&yys.value ()) value_type (s->value ());]])[
+ }
Example of (ii)
yyparser.yy_destroy_ ("Error: discarding",
- yytoken, &yylval]b4_locations_if([, &yylloc])[);
+ yytoken, &yylval]b4_locations_if([, &yylloc])[);]b4_variant_if([[
+ // Value type destructor.
+ ]b4_symbol_variant([[YYTRANSLATE (this->yychar)]], [[yylval]], [[template destroy]])])[
this->yychar = ]b4_namespace_ref[::]b4_parser_class[::token::]b4_symbol(empty, id)[;
However, in some places we must not be "pure". In particular:
glr_stack_item (const glr_stack_item& other) YY_NOEXCEPT YY_NOTHROW
: is_state_ (other.is_state_)
{
std::memcpy (raw_, other.raw_, union_size);
}
still must use memcpy, because the constructor would change pred, and
it must not. This constructor is used only when resizing the stack,
in which case pred (which is relative) must not be "adjusted".
The result works, but is messy. Its verbosity comes from at least two
factors:
- we don't have support for complete symbols (binding kind, value and
location), and we should at least try to have it. That simplified
lalr1.cc a lot.
- I have not tried to be smart and use 'move' when possible. As a
consequence many places have 'copy' and then 'destroy'. That kind
of clean up can be done once everything appears to be solid.
* data/skeletons/glr2.cc: Be more rigorous in object lifetime.
In particular, don't forget to discard the lookahead when we're done
with it.
Call variant routines where needed.
Deal with plenty of details.
(b4_call_merger): Add support for variants.
Use references in mergers, rather than pointers.
* examples/c++/glr/c++-types.yy: Exercise variants.
On some experimentation I was running, the test suite was passing, yet
the example crashed when run in verbose mode. Let's add this case to
the test suite.
* tests/cxx-type.at: Run all these tests in verbose mode too.
* examples/c/glr/c++-types.y: Flush stdout so that the logs (on
stderr) and the effective output (on stdout) mix correctly.
While at it, be a bit more const-correct.
This macro is not exposed to users, make start it with 'YY_'.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/glr2.cc, data/skeletons/lalr1.cc,
* src/parse-gram.c, tests/actions.at, tests/c++.at, tests/headers.at,
* tests/local.at (YYUSE): Rename as...
(YY_USE): this.
See "glr.c: log the execution of deferred actions".
* data/skeletons/glr2.cc (yyuserAction): Take yyk as a new argument.
Rename argument yyn as yyrule for clarity.
Log before and after the user action.
Adjust callers to not call YY_REDUCE_PRINT and YY_SYMBOL_PRINT.
Currently deferred reductions are not "verbose" at all: only immediate
reductions are displayed in the YYDEBUG traces. I don't understand
why. Besides it seems actually simpler the install the reduction
traces right around the user action inside yyuserAction rather that
around calls to yyuserAction.
This only trouble is that yyuserAction does not know the stack number
it works on, so we have to pass it. And pass -1 when we are actually
running on a temporary stack.
The glr example, on "T(x) + y;" as input, adds these logs, which
allow to see when the `<cast>` is built:
Stack 0 Entering state 26
Reduced stack 0 by rule 7 (line 108); action deferred. Now in state 7.
Stack 0 Entering state 7
Reading a token
Next token is token '+' (1.6: )
Stack 1 Entering state 27
Reduced stack 1 by rule 13 (line 123); action deferred. Now in state 12.
Stack 1 Entering state 12
Next token is token '+' (1.6: )
Stack 1 dies.
Removing dead stacks.
On stack 0, shifting token '+' (1.6: )
Stack 0 now in state #14
+Reducing stack -1 by rule 6 (line 107):
+ $1 = token identifier (1.3: x)
+-> $$ = nterm expr (1.3: x)
+Reducing stack -1 by rule 7 (line 108):
+ $1 = token typename (1.0: T)
+ $2 = token '(' (1.2: )
+ $3 = nterm expr (1.3: x)
+ $4 = token ')' (1.4: )
+-> $$ = nterm expr (1.0-3: <cast>(x,T))
Returning to deterministic operation.
* data/skeletons/glr.c (yyuserAction): Take yyk as a new argument.
Rename argument yyn as yyrule for clarity.
Log before and after the user action.
Adjust callers to not call YY_REDUCE_PRINT and YY_SYMBOL_PRINT.
The next commit wants to use YY_REDUCE_PRINT above its current
definition. Move it higher.
* data/skeletons/glr.c (yylhsNonterm, YY_REDUCE_PRINT): Make available
earlier.
* examples/c/glr/c++-types.y (node_print): New.
Use YY_LOCATION_PRINT instead of duplicating it.
And actually use it in the action instead of badly duplicating it.
(main): Add proper option support.
* examples/c/glr/c++-types.test: Adjust expectations on locations.
* examples/c++/glr/c++-types.yy: Fix bad iteration.
Currently each time we meet %merge we record this location as the
defining location (and symbol). Instead, record the first definition.
In the generated code we go from
yy0->A = merge (*yy0, *yy1);
to
yy0->S = merge (*yy0, *yy1);
where S was indeed the first symbol, and in the diagnostics we go from
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type2>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:30.18-24: note: previous declaration
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
to
glr-regr18.y:30.18-24: error: result type clash on merge function 'merge': <type2> != <type1>
30 | sym2: sym3 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
glr-regr18.y:31.13-19: error: result type clash on merge function 'merge': <type3> != <type1>
31 | sym3: %merge<merge> { $$ = 0; } ;
| ^~~~~~~
glr-regr18.y:29.18-24: note: previous declaration
29 | sym1: sym2 %merge<merge> { $$ = $1; } ;
| ^~~~~~~
where both duplicates are reported against definition 1, rather than
using definition 1 as a reference when diagnosing about definition 2,
and then 2 as a reference for 3.
* src/reader.c (record_merge_function_type): Keep the first definition.
* tests/glr-regression.at: Adjust.
Reported by Jot Dot.
https://lists.gnu.org/r/help-bison/2020-12/msg00014.html
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): Use
the symbol's slot, not its type.
* examples/c/glr/c++-types.y: Use explicit per-symbol typing together
with api.value.type=union.
(yylex): Use yytoken_kind_t.
Don't generate C code from bison, leave that to the skeletons.
* src/output.c (merger_output): Emit invocations to b4_call_merger.
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): New.
Symbols are richer than types, and in M4 it is my simpler (and more
common) to deal with symbols rather than types. So let's associate
mergers to a symbol rather than a type name.
* src/reader.h (merger_list): Replace the 'type' member by a symbol
member.
* src/reader.c (record_merge_function_type): Take a symbol as
argument, rather than a type name.
* src/output.c (merger_output): Adjust.
Now that the lookahead macros (that used yystackp) are out of the way,
there is no reason to continue using a pointer.
* data/skeletons/glr2.cc: Use yystack, a reference, rather that
yystackp, a pointer.
Fix tons of const-correctness issues.
Now that we no longer play dangerous games with macros, we can give
the lookahead's token kind its proper name. The content of yychar
_is_ raw (as opposed to yytoken), there's no reason to pleonasmicate
it (and thus to neologize).
* data/skeletons/glr2.cc (glr_stack::yyrawchar): Rename as...
(glr_stack::yychar): this.
In glr.c, the macros yychar, yylval and yylloc allow to deal with
api.pure: sometimes they point to global variables (impure), sometimes
they point to the member variables (pure).
There's no room for globals in glr2.cc. Besides, they map yychar to
yyrawchar, yylval to yyval, etc. which obfuscates what is actually
going on.
* data/skeletons/glr2.cc (glr_stack::yyval, glr_stack::yyloc): Rename
as...
(glr_stack::yylval, glr_stack::yylloc): these, for clarity.
(yynerrs, yychar, yylval, yylloc, yystackp): Remove these macros.
(b4_yygetToken_call): Remove.
Restore a more natural order: first define the macros and then use
them. Currently, some macros were defined between the moment the
header is issued, and then the implementation file. As a result, it
was possible for the header and the implementation to not use the same
versions of the macros.
* data/skeletons/glr2.cc: Define the macros first, then use them.
* data/skeletons/lalr1.cc: Minor comment and quoting changes.
* data/skeletons/glr2.cc: Define value_type and location_type where
needed, and use them only.
(yyuserMerge): Make it a member function of the glr_state class.
We always refer to the triplet "kind, value, location". All of them
are nouns, and we support api.value.type and api.location.type. On
this regard, "semantic_type" was a poor choice. Make it "value_type".
The test suite was not updated to use value_type, on purpose, to
enforce backward compatibility.
* data/skeletons/c++.m4, data/skeletons/glr.cc, data/skeletons/glr2.cc,
* data/skeletons/variant.hh, doc/bison.texi: Define value_type rather
than semantic_type.
Add a backward compatibility typedef.
* examples/c++/glr/c++-types.yy: Migrate.
* data/skeletons/glr2.cc (glr_state_set::yyremoveDeletes): Use
vector::resize rather than vector::erase.
(glr_state::copyFrom): Merge into...
(glr_state::operator=): here.
Valentin wanted each assignment to be explicit, hence copyFrom rather
that operator=. But in 0a82316e54
(glr2.cc: example: use objects (not pointers) to represent the AST),
in order to get real objects to be processed correctly, we had to
introduce the assignment operator. Afterward, we also introduced a
full implementation of the copy-ctor, independent of copyFrom. As a
result, today the only invocation of copyFrom is from the assignment
operator. Simplify this.
With GCC10, the CI shows tons of warnings such as
(327. actions.at:374: testing Initial location: glr2.cc):
input.cc: In member function 'YYRESULTTAG glr_stack::yyglrReduce(state_set_index, rule_num, bool)':
input.cc:1357:11: error: '<anonymous>.glr_state::yyloc' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1357 | yyloc = other.yyloc;
| ~~~~~~^~~~~~~~~~~~~
This is because we don't have the constructors for locations. But we
should have them! That's only because of glr.cc that ctors were not
enabled by default. In glr2.cc, they should.
That fixes all the warnings when Bison's locations are used. However,
when user-defined locations without constructor are used, we still
have:
550. calc.at:1409: testing Calculator glr2.cc %locations api.location.type={Span} ...
calc.cc: In member function 'YYRESULTTAG glr_stack::yyglrReduce(state_set_index, rule_num, bool)':
calc.cc:1261:11: error: '<anonymous>.glr_state::yyloc' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1261 | yyloc = other.yyloc;
| ~~~~~~^~~~~~~~~~~~~
To address this case, we need glr_state to explicily initialize its
yyloc member.
* data/skeletons/glr2.cc: Use genuine objects, with ctors, for position
and location.
(glr_state): Explicitly initialize yyloc in the constructors.
The copy constructor was (lazily) implemented by a call to copyFrom.
Unfortunately copyFrom reads yyresolved from the destination (and
source), and in the case of the copy-ctor this is random garbagge,
which UBSAN catches:
glr-regr2a.cc:1072:10: runtime error: load of value 7, which is not a valid value for type 'bool'
Rather than defining yyresolved before calling copyFrom, let's just
provide a genuine cpy-ctor for glr_state.
* data/skeletons/glr2.cc (glr_state::glr_state): Implement properly.
In yycompressStack:
while (yyr != YY_NULLPTR)
{
nextFreeItem->check_ ();
yyr->check_();
nextFreeItem->setState(*yyr);
glr_state& nextFreeState = nextFreeItem->getState();
yyr = yyr->pred();
nextFreeState.setPred(&(nextFreeItem - 1)->getState());
setFirstTop(&nextFreeState);
++nextFreeItem;
}
it is possible that nextFreeItem and yyr are actually the same state.
In which case `nextFreeItem->setState(*yyr)` does really bad things.
* data/skeletons/glr2.cc (glr_stack_item::setState): Beware of
self-assignment.
* data/skeletons/glr.c: A bit more doc.
(yypstates): Rename yyst (only occurrence) to yys (commonly used for
yyGLRState).
* data/skeletons/glr2.cc: Ditto.
Prefer '\n' to "\n".
The complete symbol approach in yylex removes the need for the methods
semanticVal, startPos and endPos, which were used when the values were
reported separately.
* data/skeletons/lalr1.d: Here.
* doc/bison.texi: Remove sections about the three methods.
* examples/d/calc/calc.y, examples/d/simple/calc.y: Remove the unused methods.
* tests/calc.at, tests/d.at, tests/scanner.at: Test it.
* data/skeletons/d.m4 (b4_public_types_declare): Here.
* data/skeletons/lalr1.d: Adjust.
* doc/bison.texi: Document it.
* examples/d/calc/calc.y: Use it.
* tests/calc.at: Test it.
The yychar variable was keeping the external form of the token (the
TokenKind). As the D parser translates the token to its internal
form (the SymbolKind) inside the struct Symbol, there is no need for
yychar anymore.
* data/examples/lalr1.d (yychar): Remove.
Use only yytoken.
* examples/d/calc/calc.y (start, end): Replace by this...
(location): new member variable in the Lexer class.
Use it.
* tests/calc.at: Use the defined location variable.
examples/c++/glr/c++-types.cc:721:24: error:
expected the class name after '~' to name a destructor
yysval.YYSTYPE::~semantic_type ();
^
Using a local typedef, for some reaon, result in clang complaining
about a useless local typedef. Since anyway we don't want to keep on
using YYSTYPE and YYLTYPE, it is time to introduce proper typedefs to
reach these guys. And to be slightly in advance of the other
skeletons: use value_type, not semantic_type. This is much more
consistent with our use of the (kind, value, location) triplet.
* data/skeletons/glr2.cc (glr_state::value_type)
(glr_state::location_type): New.
(glr_state::~glr_state): Use value_type to name the dtor.
Currently we are using pointers. The whole point of
glr2.cc (vs. glr.cc) is precisely to allow genuine C++ objects to be
semantic values. Let's make that work.
* data/skeletons/glr2.cc (glr_state::glr_state): Be sure to initialize
yysval.
(glr_state): Add copy-ctor, assignment and dtor.
(glr_state::copyFrom): Be sure to initialize the destination if it was
not.
(glr_state::~glr_state): Destroy the semantic value.
* examples/c++/glr/ast.hh: Rewrite so that we use genuine objects,
rather than a traditional OOP hierarchy that requires to deal with
pointers.
With help from Bruno Belanyi <[email protected]>.
* examples/c++/glr/c++-types.yy: Remove memory management.
Use true objects.
(main): Don't reach yydebug directly.
* examples/c++/glr/local.mk: We need C++11.
When expanding the GLR stack, none of the pointers were updated to
reflect the new location of the displaced objects.
This fixes
748: Incorrect lookahead during nondeterministic GLR: glr2.cc
* data/skeletons/glr2.cc (yyexpandGLRStack): Update the split point
and the stack tops.
(reduceToOneStack): Factor a bit.
This test fails:
748: Incorrect lookahead during nondeterministic GLR: glr2.cc
It consumes lots of stack space, so at some point we need to expand
it. Because of Boolean logic mistakes, we then claim
memory-exhausted (first error). Hence we jump to cleaning the
stack (popall_), calling all the destructors, and at some point we
crash with heap-use-after-free (second error).
This commit fixes the first error. Unfortunately, even though we now
do expand the stack, we crash again with (another)
heap-use-after-free, not addressed here.
Eventually, we should make sure popall_() properly works.
* data/skeletons/glr2.cc (yyexpandGLRStackIfNeeded): Return true iff
success (i.e., memory not exhausted).
From a debugger, it is easier to pass a file name than working on
stdin.
* examples/c++/glr/c++-types.yy: Reduce scopes.
Avoid YYSTYPE/YYLTYPE: use the C++ types.
(input, process): New.
(main): Use them.
And also, remove the incorrect indentation of these comments:
- /* YYR2[YYN] -- Number of symbols on the right hand side of rule YYN. */
+/* YYR2[RULE-NUM] -- Number of symbols on the right-hand side of rule RULE-NUM. */
static const yytype_int8 yyr2[] =
{
0, 2, 4, 0, 2, 1, 1, 1, 3, 2,
I don't remember why this indentation was added (in
0991e29b75), but it seems wrong,
at least for yacc.c. I suspect this was done with lalr1.cc (where
this is embeded in the class definition, so it should be indented),
but today lalr1.cc uses other routines to output these comments.
* data/skeletons/bison.m4 (b4_integral_parser_tables_map): Improve the
wording of the comments of some tables.
* data/skeletons/c.m4 (b4_integral_parser_table_define): Remove
indentation.
A glr_stack_item has "raw" memory to store either a glr_state or a
semantic_option. glr_stack_item::setState stores a state using a copy
assignment. However, this is more like a construction: we are
starting from "raw" memory, so use the placement new operator instead.
While it probably makes no difference when parse.assert is disabled,
it does make one when it is: the constructor properly initialize the
magic number, the assignment does not. So without these changes, the
next commit (which stores genuine objects in semantic values) fails
tests 712 and 730 because of incorrect magic numbers.
* data/skeletons/glr2.cc (glr_stack_item::setState): Build the state,
don't just copy it.
ast.hh:24:7: error: 'Node' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
class Node
^
ast.hh:57:7: error: 'Nterm' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
class Nterm : public Node
^
ast.hh:102:7: error: 'Term' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
class Term : public Node
^
* examples/c++/glr/ast.hh: Define the destructors out of the class
definition.
This does not change anything, it is still in the header, but that
does pacify clang.
When debugging these parsers, we really need debug traces.
Enable them, and bind them to $YYDEBUG.
* tests/glr-regression.at: Support the YYDEBUG envvar.
As a consequence, now that syntactic ambiguities are reported, adjust
the expected output.
(No users destructors if stack 0 deleted): Don't return 0 on memory
exhaustion, really return the parser's status, and adust expectations.
Currently the example really looks like C. Instead of a union of
structs to implement the AST, use a hierarchy. It would be nice to
feature a C++17 version with std variants.
* examples/c++/glr/c++-types.yy (Node, free_node, new_nterm)
(new_term): Move into...
* examples/c++/glr/ast.hh: here, a proper C++ hierarchy.
Amusingly enough, glr2.cc still had its core function, yyparse, being
a free function instead of a member function.
* data/skeletons/glr2.cc (yyparse): Remove this free function called
from yyparser::parse. Inline its body into...
(yyparser::parse): this member function.
This requires moving a bit the yychar, etc. macros.
Access to token can be simplified (the
b4_namespace_ref::b4_parser_class prefix is no longer needed).
Remove the useless conditional b4_pure_if: the skeleton is, of course,
pure (no global variables). Make glr2.cc intrinsically pure.
* data/skeletons/glr2.cc (b4_pure_if): Remove definition and uses.
(b4_lex): New.
Stolen from lalr1.cc to avoid needing to use the one from c.m4.
Currently, yycompressStack expects the free items to be states only.
That's not the case.
Fixes 712 and 730 pass. 748 still fails, but later and
differently (heap-use-after-free).
* data/skeletons/glr2.cc (glr_stack_item::setState): New.
(glr_stack_item::yycompressStack): Use it.
* tests/glr-regression.at: Adjust.
A glr_state keeps tracks of its predecessor using an offset relative
to itself (i.e., pointer subtraction). Unfortunately we sometimes
have to compute offsets for pointers that live in different
containers, in particular in yyfillin. In that case there is no
reason for the distance between the two objects to be a multiple of
the object size (0x40 on my machine), and the resulting ptrdiff_t may
be "wrong", i.e., it does allow to recover one from the other. We
cannot use "typed" pointer arithmetics here, the Euclidean division
has it wrong. So use "plain" char* pointers.
Fixes 718 (Duplicate representation of merged trees: glr2.cc) and
examples/c++/glr/c++-types.
Still XFAIL:
712: Improper handling of embedded actions and dollar(-N) in GLR parsers: glr2.cc
730: Incorrectly initialized location for empty right-hand side in GLR: glr2.cc
748: Incorrect lookahead during nondeterministic GLR: glr2.cc
* data/skeletons/glr2.cc (glr_state::as_pointer_): New.
(glr_state::pred): Use it.
* examples/c++/glr/c++-types.test: The test passes.
* tests/glr-regression.at (Duplicate representation of merged trees:
glr2.cc): Passes.
The use of YY_IGNORE_NULL_DEREFERENCE_BEGIN/END in `check_` is to
please GCC 10:
glr-regr8.cc: In member function 'YYRESULTTAG glr_stack::yyresolveValue(glr_state&)':
glr-regr8.cc:1433:21: error: potential null pointer dereference [-Werror=null-dereference]
1433 | YYASSERT (this->magic_ == MAGIC);
| ~~~~~~^~~~~~
glr-regr8.cc:905:40: note: in definition of macro 'YYASSERT'
905 | # define YYASSERT(Condition) ((void) ((Condition) || (abort (), 0)))
| ^~~~~~~~~
* data/skeletons/glr2.cc (glr_state::check_): New.
Use it in the member functions.
We obviously have broken pointer arithmetics that hands us
glr_stack_items that are not glr_stack_items. Have a simple check for
this, to have earlier failures.
* data/skeletons/glr2.cc (glr_stack_item::check_): New.
Use it.
(glr_stack_item::contents): Avoid the useless struct.
Fix minor stylistic issues.
When installed on master as of 2020-12-05 (on top of "glr2.cc: fix
when the stack is not expandable", almost all the GLR regression tests
fail (with a SEGV):
709: Badly Collapsed GLR States: glr2.cc FAILED (glr-regression.at:130)
712: Improper handling of embedded actions and dollar(-N) in GLR parsers: glr2.cc FAILED (glr-regression.at:275)
715: Improper merging of GLR delayed action sets: glr2.cc FAILED (glr-regression.at:404)
718: Duplicate representation of merged trees: glr2.cc FAILED (glr-regression.at:502)
721: User destructor for unresolved GLR semantic value: glr2.cc FAILED (glr-regression.at:566)
724: User destructor after an error during a split parse: glr2.cc FAILED (glr-regression.at:624)
727: Duplicated user destructor for lookahead: glr2.cc FAILED (glr-regression.at:724)
730: Incorrectly initialized location for empty right-hand side in GLR: glr2.cc FAILED (glr-regression.at:823)
733: No users destructors if stack 0 deleted: glr2.cc FAILED (glr-regression.at:911)
736: Corrupted semantic options if user action cuts parse: glr2.cc FAILED (glr-regression.at:974)
739: Undesirable destructors if user action cuts parse: glr2.cc FAILED (glr-regression.at:1042)
742: Leaked semantic values if user action cuts parse: glr2.cc FAILED (glr-regression.at:1173)
748: Incorrect lookahead during nondeterministic GLR: glr2.cc FAILED (glr-regression.at:1546)
751: Leaked semantic values when reporting ambiguity: glr2.cc FAILED (glr-regression.at:1639)
754: Leaked lookahead after nondeterministic parse syntax error: glr2.cc FAILED (glr-regression.at:1710)
757: Uninitialized location when reporting ambiguity: glr2.cc FAILED (glr-regression.at:1794)
766: Predicates: glr2.cc FAILED (glr-regression.at:2045)
These pass:
745: Incorrect lookahead during deterministic GLR: glr2.cc ok
760: Missed %merge type warnings when LHS type is declared later: glr2.cc ok
763: Ambiguity reports: glr2.cc ok
With Valentin Tolmer's "glr2.cc: Fix memory corruption bug" commit,
these test fail "gracefully":
712: Improper handling of embedded actions and dollar(-N) in GLR parsers: glr2.cc FAILED (glr-regression.at:268)
730: Incorrectly initialized location for empty right-hand side in GLR: glr2.cc FAILED (glr-regression.at:816)
748: Incorrect lookahead during nondeterministic GLR: glr2.cc FAILED (glr-regression.at:1539)
And these do not end:
709: Badly Collapsed GLR States: glr2.cc FAILED (glr-regression.at:123)
715: Improper merging of GLR delayed action sets: glr2.cc FAILED (glr-regression.at:397)
718: Duplicate representation of merged trees: glr2.cc FAILED (glr-regression.at:495)
751: Leaked semantic values when reporting ambiguity: glr2.cc FAILED (glr-regression.at:1632)
With "tests: glr2.cc: run the glr-regression tests", none loop, and
709, 715, and 751 pass. Only 718 still fails.
* tests/glr-regression.at: Run all the tests with glr2.cc.
* tests/local.at (AT_GLR2_CC_IF): New.
When "tests: glr2.cc: run the glr-regression tests" tests are run,
before this commit the following tests used to loop endlessly:
709: Badly Collapsed GLR States: glr2.cc FAILED (glr-regression.at:123)
715: Improper merging of GLR delayed action sets: glr2.cc FAILED (glr-regression.at:397)
718: Duplicate representation of merged trees: glr2.cc FAILED (glr-regression.at:495)
751: Leaked semantic values when reporting ambiguity: glr2.cc FAILED (glr-regression.at:1632)
After this commit, no test loops and 709, 715, and 751 pass. Only 718
still fails.
* data/skeletons/glr2.cc (yyresolveValue): Add missing incrementation
of the iteration variable.
* data/skeletons/glr2.cc: Use 'const' on variables and applicable
member functions.
Improve comments.
Use references where applicable.
Enforce names_like_this, notLikeThis.
Reduce scopes.
* tests/glr-regression.at: Adjust the tests to be more independent of
the language, and run them with glr.cc.
Some tests relied on yychar, yylval and yylloc being global variables:
pass arguments instead.
* tests/glr-regression.at: Instead of using AT_BISON_CHECK and
AT_COMPILE, use AT_FULL_COMPILE. This is shorter, and makes it easier
to add support for other programming languages.
* tests/glr-regression.at: Use %expect and %expect-rr in the grammar
files, rather than accepting diagnostics.
This will make it easier to support other programming languages.
When using glr.cc, the C function yyparse is an internal detail that
should not be exposed. Users might call it by accident (I did).
* data/skeletons/glr.c (yyparse): When used for glr.cc, rename as yy_parse_impl.
* data/skeletons/glr.cc: Adjust.
src/reader.c: In function 'grammar_start_symbols_add':
src/reader.c:67:24: error: declaration of 'dup' shadows a global declaration [-Werror=shadow]
* src/reader.c (grammar_start_symbols_add): Rename dup as dupl.
The yydefgoto table uses -1 as an invalid for an impossible case (we
never use yydefgoto[0], since it corresponds to the reduction to
$accept, which never happens). Since yydefgoto is a table of state
numbers, this -1 forces a signed type uselessly, which (1) might
trigger compiler warnings when storing a value from yydefgoto into a
state number (nonnegative), and (2) wastes bits which might result in
using a int16 where a uint8 suffices.
Reported by Jot Dot <[email protected]>.
https://lists.gnu.org/r/bug-bison/2020-11/msg00027.html
* src/tables.c (default_goto): Use 0 rather than -1 as invalid value.
* tests/regression.at: Adjust.
We were declaring foo_parse_expr but implementing yyparse_expr.
* data/skeletons/yacc.c (_b4_define_sub_yyparse): Use api.prefix for
the parse function name.
This avoids heap allocation and gives minimal costs for the
creation and destruction of the YYParser.Symbol struct if
the location tracking is active.
Suggested by H. S. Teoh.
* data/skeletons/lalr1.d: Here.
* doc/bison.texi: Document it.
* examples/d/calc/calc.y: Adjust.
* tests/calc.at: Test it.
The complete symbol approach was deemed to be the right approach for Dlang.
Now, the user can return from yylex() an instance of YYParser.Symbol structure,
which binds together the TokenKind, the semantic value and the location. Before,
the last two were reported separately to the parser.
Only the user API is changed, Bisons's internal structure is kept the same.
* data/skeletons/d.m4 (struct YYParser.Symbol): New.
* data/skeletons/lalr1.d: Change the return value.
* doc/bison.texi: Document it.
* examples/d/calc/calc.y, examples/d/simple/calc.y: Demonstrate it.
* tests/calc.at, tests/scanner.at: Test it.
When using lookahead correction, the method YYParser.Context.getExpectedTokens
is not annotated with const, because the method calls yylacCheck, which is not
const. Also, because of yylacStack and yylacEstablished, yylacCheck needs to
be called from the context of the parser class, which is sent as parameter to
the Context's constructor.
* data/skeletons/lalr1.d (yylacCheck, yylacEstablish, yylacDiscard,
yylacStack, yylacEstablished): New.
(Context): Use it.
* doc/bison.texi: Document it.
* tests/calc.at: Check it.
Changed from syntax_error to reportSyntaxError to be similar to the Java parser.
* data/skeletons/lalr1.d: Change the function name.
* doc/bison.texi: Document it.
* tests/local.at: Adjust.
* maint:
c++: shorten the assertions that check whether tokens are correct
c++: don't glue functions together
lalr1.cc: YY_ASSERT should use api.prefix
c++: don't use YY_ASSERT at all if parse.assert is disabled
c++: style: follow the Bison m4 quoting pattern
yacc.c: provide the Bison version as an integral macro
regen
style: make conversion of version string to int public
%require: accept version numbers with three parts ("3.7.4")
yacc.c: fix #definition of YYEMPTY
gnulib: update
doc: fix incorrect section title
doc: minor grammar fixes in counterexamples section
Before:
YY_ASSERT (tok == token::YYEOF || tok == token::YYerror || tok == token::YYUNDEF || tok == 120 || tok == 49 || tok == 50 || tok == 51 || tok == 52 || tok == 53 || tok == 54 || tok == 55 || tok == 56 || tok == 57 || tok == 97 || tok == 98);
After:
YY_ASSERT (tok == token::YYEOF
|| (token::YYerror <= tok && tok <= token::YYUNDEF)
|| tok == 120
|| (49 <= tok && tok <= 57)
|| (97 <= tok && tok <= 98));
Clauses are now also wrapped on several lines. This is nicer to read
and diff, but also avoids pushing Visual C++ to its arbitrary
limits (640K and lines of 16380 bytes ought to be enough for anybody,
otherwise make an C2026 error).
The useless parens are there for the dummy warnings about
precedence (in the future, will we also have to put parens in
`1+2*3`?).
* data/skeletons/variant.hh (_b4_filter_tokens, b4_tok_in, b4_tok_in):
New.
(_b4_token_constructor_define): Use them.
Working on the previous commit I realized that YY_ASSERT was used in
the generated headers, so must follow api.prefix to avoid clashes when
multiple C++ parser with variants are used.
Actually many more macros should obey api.prefix (YY_CPLUSPLUS,
YY_COPY, etc.). There was no complaint so far, so it's not urgent
enough for 3.7.4, but it should be addressed in 3.8.
* data/skeletons/variant.hh (b4_assert): New.
Use it.
* tests/local.at (AT_YYLEX_RETURN): Fix.
* tests/headers.at: Make sure variant-based C++ parsers are checked
too.
This test did find that YY_ASSERT escaped renaming (before the fix in
this commit).
In some extreme situations (about 800 tokens), we generate a
single-line assertion long enough for Visual C++ to discard the end of
the line, thus falling into parse ends for the missing `);`. On a
shorter example:
YY_ASSERT (tok == token::TOK_YYEOF || tok == token::TOK_YYerror || tok == token::TOK_YYUNDEF || tok == token::TOK_ASSIGN || tok == token::TOK_MINUS || tok == token::TOK_PLUS || tok == token::TOK_STAR || tok == token::TOK_SLASH || tok == token::TOK_LPAREN || tok == token::TOK_RPAREN);
Whether NDEBUG is used or not is irrelevant, the parser dies anyway.
Reported by Jot Dot <[email protected]>.
https://lists.gnu.org/r/bug-bison/2020-11/msg00002.html
We should avoid emitting lines so long.
We probably should also use a range-based assertion (with extraneous
parens to pacify fascist compilers):
YY_ASSERT ((token::TOK_YYEOF <= tok && tok <= token::TOK_YYUNDEF)
|| (token::TOK_ASSIGN <= tok && ...)
But anyway, we should simply not emit this assertion at all when not
asked for.
* data/skeletons/variant.hh: Do not define, nor use, YY_ASSERT when it
is not enabled.
When generating a C parser, YYEMPTY is present in enum yytokentype but
there is no corresponding #define like there is for the other values.
There is a special case for YYEMPTY in b4_token_enums but no
corresponding case in b4_token_defines.
* data/skeletons/c.m4 (b4_token_defines): Do define YYEMPTY.
IELR needs to rule out the successors of the kernel items of the
initial state (`$accept: input • $end`). In the case of multistart,
this condition must be expressed differently: the mere item index does
not suffice.
* src/ielr.c (ielr_item_has_lookahead, ielr_compute_lookaheads): Don't
rely on the item index to check whether is_successor_of_initial_item.
It is certainly more costly than just checking the item index, but (i)
we need to compute the rule anyway, so it's not very much more costly,
and (ii) in ielr_item_has_lookahead, this situation is actually
impossible, so an optimizing compiler reading the assertions should
actually avoid this computation.
Checking that an item index is > 1 means ruling out `$accept: • input
$end` and `$accept: input • $end`. But actually only the latter is
possible there, i.e., we're checking whether this item is about a
successor of a (kernel) item of the initial state ($accept: input •
$end).
* src/ielr.c (is_successor_of_initial_item): Use a variable to name
this condition.
* src/gram.h (rule_is_initial): New.
* src/graphviz.c, src/print-xml.c, src/print.c, src/lalr.c: Use it.
Some of these occurrences were incorrect (checking whether this is
rule 0), and not behaving properly in the case of multistart.
* src/lalr.c: Remove incorrect comment (subsumed anyway by the
(correct) one in the header.
(set_goto_map): More debug traces.
(map_goto): Add an assertion.
Parser.Context class returns a const YYLocation, so Lexer's method
yyerror() needs to receive the location as a const parameter.
Internal error reporting flow is changed to be similar to that of
the other skeletons. Before, case YYERRLAB was calling yyerror()
with the result of yysyntax_error() as the string parameter. As the
custom error message lets the user decide if they want to use
yyerror() or not, this flow needed to be changed. Now, case YYERRLAB
calls yyreportSyntaxError(), that builds the error message using
yysyntaxErrorArguments(). Then yyreportSyntaxError() passes the
error message to the user defined syntax_error() in case of a custom
message, or to yyerror() otherwise.
In the tests in tests/calc.at, the order of the tokens needs to be
changed in order of precedence, so that the D program outputs the
expected tokens in the same order as the other parsers.
* data/skeletons/lalr1.d: Add the custom error message feature.
* doc/bison.texi: Document it.
* examples/d/calc/calc.y: Adjust.
* tests/calc.at, tests/local.at: Test it.
This should have been part of commit "symbols: stop dealing with YYEMPTY
as b4_symbol(-2, ...)" (cd40ec9526).
Give names to all the special symbols: "eof", "error" and "undef".
* data/skeletons/bison.m4 (b4_symbol): Let `b4_symbol(eof, ...)` mean
`b4_symbol(0, ...)`, `b4_symbol(error, ...)` mean `b4_symbol(1, ...)`,
and , `b4_symbol(undef, ...)` mean `b4_symbol(2, ...)`..
* data/skeletons/c.m4, data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/glr2.cc, data/skeletons/lalr1.cc,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/yacc.c:
Prefer symbols to numbers.
All methods are now declared as const. Method getExpectedTokens() has now the
functionality of reporting only the number of expected tokens.
* data/skeletons/lalr1.d: Fix Context class.
In D's case, yyerrok() is a private method of the Parser class.
It can be called directly as `yyerrok()` from the grammar rules section.
* data/skeletons/lalr1.d: Add yyerrok().
* examples/d/calc/calc.y, examples/d/simple/calc.y: Demonstrate yyerrok().
* tests/calc.at: Update D tests to use yyerrok().
examples/java/calc/Calc.java:1531: warning: [deprecation] Integer(String) in Integer has been deprecated
yylval = new Integer(st.sval);
^
* examples/java/calc/Calc.y, examples/java/simple/Calc.y,
* tests/calc.at, tests/scanner.at: Use Integer.parseInt.
Vector is synchronized, which is completely useless in our case (and
not even relevant when concurrency matters). No seasoned Java
programmer would use it.
Reported by Uxio Prego.
* data/skeletons/lalr1.java: Replace Vector with ArrayList.
Unfortunately its API is not as rich, and lacks lastElement and
setSize.
* tests/calc.at: Add tests for LAC in pull and push parsers.
Skip LAC: line from the logs.
* tests/local.at (reportSyntaxError): Output the error message in a
single call, to avoid having the error message on stderr be
interrupted by the debug traces of LAC in getExpectedTokens.
Shamelessly stolen from Adrian Vogelsgesang's implementation in
lalr1.cc.
* data/skeletons/lalr1.java (yycdebugNnl, yyLacCheck, yyLacEstablish)
(yyLacDiscard, yylacStack, yylacEstablished): New.
(Context): Use it.
* examples/java/calc/Calc.y: Enable LAC.
This will provide the user an interface for creating custom error messages.
* data/skeletons/lalr1.d: Add the Context class.
* doc/bison.texi: Document it.
* upstream/maint:
doc: fix typo
maint: post-release administrivia
version 3.7.3
build: don't link bison against libreadline
gnulib: update
glr.cc: fix: use symbol_name
build: fix a concurrent build issue in examples
When printing items, it is clearer to put the dot after %emtpy rather
than before:
0 $accept: . unit "end of file"
1 unit: . assignments exp
- 2 assignments: . %empty
+ 2 assignments: %empty .
3 | . assignments assignment
Also, use the Unicode characters if they are supported.
* src/gram.c (item_print): Put the dot after %emtpy.
* tests/conflicts.at, tests/reduce.at, tests/report.at: Adjust.
Currently `b4_percent_define_ifdef([foo])` assigns a default value to
`foo` when invoked. As a consequence, skeletons such as lalr1.d
cannot specify their specific default values: `foo` was defined in
bison.m4.
Instead, provide `foo` with a default value when `b4_foo_if` is
invoked.
I could not measure a runtime difference between both cases.
* data/skeletons/bison.m4 (_b4_percent_define_define): New.
Helps getting rid of spurious indentation that resulted in spurious
white space in the output.
(b4_percent_define_if_define): Move the definition to...
(_b4_percent_define_if_define): when the defined macros is called.
* tests/local.at (AT_TOKEN_TRANSLATE_IF): New, moved from...
* tests/calc.at: here.
Instead of sorting per feature (main, yylex, calc.y) and then by
language, do the converse, so that C bits are together, etc.
This is something that has always bothered me: with pure parsers (and
they all should be) the user does not have an (easy) access to yynerrs
at the end of the parse. In the case of error recovery, that's the
only direct means to know if there were errors. The usual approach
being having the user maintain a counter incremented each time yyerror
is called.
So here, also capture yynerrs in the return value of the start-symbol
parsing functions.
* data/skeletons/yacc.c (yy_parse_impl_t): New.
(yy_parse_impl): Use it.
(b4_accept): Fill it.
* examples/c/lexcalc/parse.y, examples/c/lexcalc/scan.l: No longer
pass nerrs as lex- and parse-param, just use the resulting yynerrs.
bistromathic and reccalc both demonstrate %param.
After all, why not?
* src/reader.c (switching_token): Use symbol_id_get.
(check_start_symbols): Require that the start symbol is a token only
if it's the only one.
* examples/c/lexcalc/parse.y: Let NUM be a start symbol.
For each start symbol, generate a parsing function with a richer
return value than the usual of yyparse. Reserve a place for the
returned semantic value, in order to avoid having to pass a pointer as
argument to "return" that value. This also makes the call to the
parsing function independent of whether a given start-symbol is typed.
For instance, if the grammar file contains:
%type <int> expression
%start input expression
(so "input" is valueless) we get
typedef struct
{
int yystatus;
} yyparse_input_t;
yyparse_input_t yyparse_input (void);
typedef struct
{
int yyvalue;
int yystatus;
} yyparse_expression_t;
yyparse_expression_t yyparse_expression (void);
This commit also changes the implementation of the parser termination:
when there are multiple start symbols, it is the initial rules that
explicitly YYACCEPT. They do that after having exported the
start-symbol's value (if it is typed):
switch (yyn)
{
case 1: /* $accept: YY_EXPRESSION expression $end */
{ ((*yyvalue).TOK_expression) = (yyvsp[-1].TOK_expression); YYACCEPT; }
break;
case 2: /* $accept: YY_INPUT input $end */
{ YYACCEPT; }
break;
I have tried several ways to deal with termination, and this is the
one that appears the best one to me. It is also the most natural.
* src/scan-code.h, src/scan-code.l (obstack_for_actions): New.
* src/reader.c (grammar_rule_check_and_complete): Generate the actions
of the rules for each start symbol.
* data/skeletons/bison.m4 (b4_symbol_slot): New, with safer semantics
than type and type_tag.
* data/skeletons/yacc.c (b4_accept): New.
Generates the body of the action of the start rules.
(_b4_declare_sub_yyparse): For each start symbol define a dedicated
return type for its parsing function.
Adjust the declaration of its parsing function.
(_b4_define_sub_yyparse): Adjust the definition of the function.
* examples/c/lexcalc/parse.y: Check the case of valueless symbols.
* examples/c/lexcalc/lexcalc.test: Check start symbols.
So far we were not checking the generated rule 0 at all. Now there
can be several of them. Instead of not checking at all, let's be more
selective on the check to run on them.
* src/reader.c (grammar_rule_check_and_complete): Don't check for
value usage for generated rules, it is ok to have a valued start
symbol, in which case it is ok for the generated rule ("accept: start
$end {}") to not use $1.
(packgram): Call grammar_rule_check_and_complete for all the rules.
Currently the core of the initial state is limited to the single rule
on $accept.
* src/lr0.c (generate_states): There may now be several rules on
$accept.
* src/graphviz.c (conclude_red): Recognize "final" transitions by the
fact that we reduce to "$accept".
* src/print.c (print_reduction): Likewise.
* src/print-xml.c (print_reduction): Likewise.
Now that the parser can read several start symbols, let's process
them, and create the corresponding rules.
* src/parse-gram.y (grammar_declaration): Accept a list of start symbols.
* src/reader.h, src/reader.c (grammar_start_symbol_set): Rename as...
(grammar_start_symbols_set): this.
* src/reader.h, src/reader.c (start_flag): Replace with...
(start_symbols): this.
* src/reader.c (grammar_start_symbols_set): Build a list of start
symbols.
(switching_token, create_start_rules): New.
(check_and_convert_grammar): Use them to turn the list of start
symbols into a set of rules.
* src/reduce.c (nonterminals_reduce): Don't complain about $accept,
it's an internal detail.
(reduce_grammar): Complain about all the start symbols that don't
derive sentences.
* src/symtab.c (startsymbol, startsymbol_loc): Remove, replaced by
start_symbols.
symbols_pack): Move the check about the start symbols
to...
* src/symlist.c (check_start_symbols): here.
Adjust to multiple start symbols.
* tests/reduce.at (Empty Language): Generalize into...
(Bad start symbols): this.
Currently this example crashes on input such as "T (x) + y;".
The same example with glr.c works properly.
* examples/c++/glr/Makefile, examples/c++/glr/README.md,
* examples/c++/glr/c++-types.test, examples/c++/glr/c++-types.yy,
* examples/c++/glr/local.mk, examples/c++/local.mk: New.
Based on examples/c/glr/c++-types.y.
* data/skeletons/lalr1.d: Change the return value.
* examples/d/calc/calc.y, examples/d/simple/calc.y: Adjust.
* tests/scanner.at: Adjust.
* tests/calc.at (_AT_DATA_CALC_Y(d)): New, extracted from...
(_AT_DATA_CALC_Y(c)): here.
The two grammars have been sufficiently different to be separated.
Still trying to be them together results in a maintenance burden. For
the same reason, instead of specifying the results for D and for the
rest, compute the expected results with D from the regular case.
The D skeleton was not properly supporting @1 etc.
Reported by Adela Vais.
https://lists.gnu.org/r/bison-patches/2020-09/msg00049.html
* data/skeletons/d.m4 (b4_rhs_location): Fix it.
* tests/calc.at: Check the support of @n for all the skeletons.
Based on the test case 668 (cxx-type.at:437) "GLR: Merge conflicting
parses, pure, locations".
* examples/c/glr/Makefile, examples/c/glr/README.md,
* examples/c/glr/c++-types.test, examples/c/glr/c++-types.y,
* examples/c/glr/local.mk: New.
This is consistent with --defines being deprecated in favor of
--header. The directive %defines is also too similar to %define.
And %header matches nicely with api.header.name.
* src/scan-gram.l (%defines): Deprecate to %header.
(%header): Scan it.
* src/parse-gram.y (PERCENT_DEFINES): Replace with...
(PERCENT_HEADER): this.
* data/skeletons/lalr1.java
* doc/bison.texi
* tests/actions.at, tests/c++.at, tests/calc.at, tests/conflicts.at,
* tests/input.at, tests/java.at, tests/local.at, tests/output.at,
* tests/synclines.at, tests/types.at:
Convert most tests to check %header instead of %defines.
The name "defines" is incorrect, the generated file contains far more
than just #defines.
* src/getargs.h, src/getargs.c (-H, --header): New option.
With optional argument, just like --defines, --xml, etc.
(defines_flag): Rename as...
(header_flag): this.
Adjust dependencies.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/glr2.cc, data/skeletons/lalr1.cc,
* data/skeletons/yacc.c:
Adjust.
* examples, doc/bison.texi: Adjust.
* tests/headers.at, tests/local.at, tests/output.at: Convert most
tests from using --defines to using --header.
231. conflicts.at:1096: testing Syntax error in consistent error state: glr2.cc ...
tests/conflicts.at:1096: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o input input.cc $LIBS
input.cc: In member function 'YYRESULTTAG glr_stack::yyresolveValue(glr_state*)':
input.cc:2674:36: error: 'yysval' may be used uninitialized in this function [-Werror=uninitialized]
Do not initialize the variable: this way ASAN can really make sure we
do set it to a proper value.
If we initialize it, ASAN would report nothing.
* data/skeletons/c.m4 (YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN): Disable
GCC 4.6's -Wuninitialized.
* data/skeletons/glr2.cc: Disable the warning locally.
231. conflicts.at:1096: testing Syntax error in consistent error state: glr2.cc ...
tests/conflicts.at:1096: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o input input.cc $LIBS
input.cc: In function 'int yyparse(yy::parser&)':
input.cc:3147:41: error: 'yyarg' may be used uninitialized in this function [-Werror=maybe-uninitialized]
return yytnamerr_ (yytname_[yysymbol]);
^
input.cc:2058:34: note: 'yyarg' was declared here
yy::parser::symbol_kind_type yyarg[YYERROR_VERBOSE_ARGS_MAXIMUM];
^
* data/skeletons/glr2.cc (yyreportSyntaxError): Initialize yyarg.
Fix a warning triggered in GCC (at least from 4.6 to 4.9):
input.cc: In constructor 'glr_stack_item::glr_stack_item(bool)':
input.cc:1371:5: error: declaration of 'is_state' shadows a member of 'this' [-Werror=shadow]
: is_state_(is_state)
^
* data/skeletons/glr2.cc (glr_stack_item): Alpha-convert.
On the CI, tests fail with GCC 4.6 to GCC 6 as follows:
tests/synclines.at:440: COLUMNS=1000; export COLUMNS; NO_TERM_HYPERLINKS=1; export NO_TERM_HYPERLINKS; bison --color=no -fno-caret -o \"\\\"\".cc \"\\\"\".y
tests/synclines.at:440: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o \"\\\"\" \"\\\"\".cc $LIBS
stderr:
"\"".cc: In member function 'glr_state& glr_stack_item::getState()':
"\"".cc:1404:47: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
return *reinterpret_cast<glr_state*>(&raw_);
^
"\"".cc: In member function 'const glr_state& glr_stack_item::getState() const':
"\"".cc:1408:53: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
return *reinterpret_cast<const glr_state*>(&raw_);
^
"\"".cc: In member function 'semantic_option& glr_stack_item::getOption()':
"\"".cc:1413:53: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
return *reinterpret_cast<semantic_option*>(&raw_);
^
"\"".cc: In member function 'const semantic_option& glr_stack_item::getOption() const':
"\"".cc:1417:59: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
return *reinterpret_cast<const semantic_option*>(&raw_);
^
See also be6fa942ac.
* data/skeletons/glr2.cc (glr_stack_item): Use a temporary void*
variable to avoid type-punning issues with reinterpret_cast.
Currently the compiler attributes are defined in
b4_shared_declarations (that can in the header if it exists, otherwise
in the implementation file). This is not needed, only the
implementation file needs them.
Besides, glr2.cc was also defining these macros in the implementation
file, so we had two definitions.
* data/skeletons/glr.cc, data/skeletons/glr2.cc: Define the compiler
attribute macros only in the implementation files.
* tests/regression.at (Lex and parse params): Generate a header, to
make it easy to check that the header is self-sufficient.
input.cc: In constructor 'glr_stack_item::glr_stack_item(bool)':
input.cc:1423:5: error: declaration of 'isState' shadows a member of 'this' [-Werror=shadow]
: isState_(isState) {
^
test.cc:1165:45: error: declaration of 'begin' shadows a member of 'this' [-Werror=shadow]
test.cc:1167:45: error: declaration of 'end' shadows a member of 'this' [-Werror=shadow]
* data/skeletons/glr2.cc (isState): Rename as...
(is_state): this.
Formatting changes.
(reduceToOneStack): Rename variables to avoid name clashes.
Commit af000bab11 ("doc: work around
Texinfo 6.7 bug"), published in 3.4.91, added a dependency on the
"all" target.
This is a super bad idea, since "make all" will run this
target *before* "all", which builds bison. It turns out that this new
dependency actually needed bison to be built. So all the regular
process (i) build $(BUILT_SOURCES) and then (ii) build bison, was
wrecked since some of the $(BUILT_SOURCES) depended on bison...
It was "easy" to see in the logs of "make V=1" because we were
building bison files (such as src/files.o) *before* displaying the
banner for "all-recursive". With this fix, we finally get again the
proper sequence:
rm -f examples/c/reccalc/scan.stamp examples/c/reccalc/scan.stamp.tmp
/opt/local/libexec/gnubin/mkdir -p examples/c/reccalc
touch examples/c/reccalc/scan.stamp.tmp
flex -oexamples/c/reccalc/scan.c --header=examples/c/reccalc/scan.h ./examples/c/reccalc/scan.l
mv examples/c/reccalc/scan.stamp.tmp examples/c/reccalc/scan.stamp
rm -f lib/fcntl.h-t lib/fcntl.h && \
{ echo '/* DO NOT EDIT! GENERATED AUTOMATICALLY! */'; \
...
} > lib/fcntl.h-t && \
mv lib/fcntl.h-t lib/fcntl.h
...
mv -f lib/alloca.h-t lib/alloca.h
make all-recursive
Reported by Mingli Yu <[email protected]>.
https://github.com/akimd/bison/issues/31https://lists.gnu.org/r/bison-patches/2020-05/msg00055.html
Reported by Claudio Calvelli <[email protected]>.
https://lists.gnu.org/r/bug-bison/2020-09/msg00001.htmlhttps://bugs.gentoo.org/716516
* doc/local.mk (all): Rename as...
(all-local): this.
So that we don't compete with BUILT_SOURCES.
The yyerror stand-alone function was used to bounce from glr.c's call
to yyerror to glr.cc's parser.error. Now that glr.c is out of the
way, just directly use parser.error.
* data/skeletons/glr2.cc (yyerror): Remove.
Adjust callers.
(b4_yyerror_args, b4_lyyerror_args, b4_pure_formals): Remove.
Now unused.
On a case such as
%%
exp
: empty "a"
| "a" empty
empty
: %empty
we used to display
warning: shift/reduce conflict on token "a" [-Wcounterexamples]
Example: • "a"
Shift derivation
exp
↳ 2: • "a" empty
↳ 2: ε
Example: • "a"
Reduce derivation
exp
↳ 1: empty "a"
↳ 3: •
where the shift derivation shows an item "2: empty → ε", with an
explicit "ε", but the reduce derivation shows "3: empty → •", without
"ε".
For consistency, let's always show ε/%empty in rules with an empty
rhs:
Reduce derivation
exp
↳ 1: empty "a"
↳ 3: ε •
* src/derivation.c (derivation_width, derivation_print_tree_impl):
Always show ε/%empty in counterexamples.
* tests/diagnostics.at: Check that case.
* tests/conflicts.at, tests/counterexample.at: Adjust.
* data/skeletons/glr2.cc: Fix some documentation.
Be consistent between class/struct.
(yydoAction, yyresolveAction): Avoid passing yyparser where useless.
This is a fork of glr.cc to be c++-first instead of a wrapper around
glr.c.
* data/skeletons/glr2.cc: New.
* data/skeletons/bison.m4, data/skeletons/c++.m4: Adjust.
* data/skeletons/c.m4 (b4_user_args_no_comma): New.
* src/reader.c (grammar_rule_check_and_complete): glr2.cc is C++.
* tests/actions.at, tests/c++.at, tests/calc.at, tests/conflicts.at,
* tests/input.at, tests/local.at, tests/regression.at, tests/scanner.at,
* tests/synclines.at, tests/types.at: Also check glr2.cc.
Reported by Dennis Clarke.
https://lists.gnu.org/r/bug-bison/2020-08/msg00013.html
* examples/d/local.mk, examples/java/calc/local.mk,
* examples/java/simple/local.mk: Pass bison's options before its
argument, in case we're in POSIX mode.
An assertion failed when the last character is a '\' and we're in a
character or a string.
Reported by Agency for Defense Development.
https://lists.gnu.org/r/bug-bison/2020-08/msg00009.html
* src/scan-gram.l: Catch unterminated escapes.
* tests/input.at (Unexpected end of file): New.
Reported by Agency for Defense Development.
https://lists.gnu.org/r/bug-bison/2020-08/msg00008.html
On an empty such as
%token FOO
BAR
FOO 0
%%
input: %empty
we crash because when we find FOO 0, we decrement ntokens (since FOO
was discovered to be EOF, which is already known to be a token, so we
increment ntokens for it, and need to cancel this). This "works well"
when EOF is properly defined in one go, but here it is first defined
and later only assign token code 0. In the meanwhile BAR was given
the token number that we just decremented.
To fix this, assign symbol numbers after parsing, not during parsing,
so that we also saw all the explicit token codes. To maintain the
current numbers (I'd like to keep no difference in the output, not
just equivalence), we need to make sure the symbols are numbered in
the same order: that of appearance in the source file. So we need the
locations to be correct, which was almost the case, except for nterms
that appeared several times as LHS (i.e., several times as "foo:
..."). Fixing the use of location_of_lhs sufficed (it appears it was
intended for this use, but its implementation was unfinished: it was
always set to "false" only).
* src/symtab.c (symbol_location_as_lhs_set): Update location_of_lhs.
(symbol_code_set): Remove broken hack that decremented ntokens.
(symbol_class_set, dummy_symbol_get): Don't set number, ntokens and
nnterms.
(symbol_check_defined): Do it.
(symbols): Don't count nsyms here.
Actually, don't count nsyms at all: let it be done in...
* src/reader.c (check_and_convert_grammar): here. Define nsyms from
ntokens and nnterms after parsing.
* tests/input.at (EOF redeclared): New.
* examples/c/bistromathic/bistromathic.test: Adjust the traces: in
"%nterm <double> exp %% input: ...", exp used to be numbered before
input.
* cfg.mk (_space_before_paren_exempt): Be less laxist.
* src/output.c, src/reader.c: Fix space before paren issues.
Pacify the warnings where applicable.
Older versions of GCC (4.1.2 here) don't like repeated typedefs.
CC src/bison-parse-simulation.o
src/parse-simulation.c:61: error: redefinition of typedef 'parse_state'
src/parse-simulation.h:74: error: previous declaration of 'parse_state' was here
make: *** [Makefile:7876: src/bison-parse-simulation.o] Error 1
Reported by Nelson H. F. Beebe.
* src/parse-simulation.c (parse_state): Don't typedef,
parse-simulation.h did it already.
This reverts commit d0bec3175f (which
should have read "We have a clash...", not "With have a clash...").
Now that `max()` was renamed `max_int()`, we can use `max` again, as
elsewhere in the code.
* src/counterexample.c (visited_hasher): Alpha reconversion.
Reported by Maarten De Braekeleer.
https://lists.gnu.org/r/bison-patches/2020-07/msg00080.html
We don't want to use gnulib's min and max macros, since we use
function calls in min/max arguments.
* src/location.c (max_int, min_int): Move to...
* src/system.h: here.
* src/counterexample.c, src/derivation.c: Use max_int instead of max.
From
foo.y:1.7-11: error: %type redeclaration for bar
1 | %type <foo> bar bar
| ^~~~~
foo.y:1.7-11: note: previous declaration
1 | %type <foo> bar bar
| ^~~~~
to
foo.y:1.17-19: error: %type redeclaration for bar
1 | %type <foo> bar bar
| ^~~
foo.y:1.13-15: note: previous declaration
1 | %type <foo> bar bar
| ^~~
* src/symlist.h, src/symlist.c (symbol_list_type_set): There's no need
for the tag's location, use that of the symbol.
* src/parse-gram.y: Adjust.
* tests/input.at: Adjust.
We crash if the input contains a string containing a NUL byte.
Reported by Suhwan Song.
https://lists.gnu.org/r/bug-bison/2020-07/msg00051.html
* src/flex-scanner.h (STRING_FREE): Avoid accidental use of
last_string.
* src/scan-gram.l: Don't call STRING_FREE without calling
STRING_FINISH first.
* tests/input.at (Invalid inputs): Check that case.
The original Texinfo macros introducing colors were made for
diagnostics, which are printed in bold. So by copy-paste accident the
styles we introduced for counterexamples were also in bold. They
should not.
* doc/bison.texi: Separate the styling of diagnostics from the styling
for counterexamples.
Don't use bold in the latter case.
With have a clash with the "max" function.
src/counterexample.c: In function 'visited_hasher':
src/counterexample.c:720:48: error: declaration of 'max' shadows a global declaration [-Werror=shadow]
src/counterexample.c:116:12: error: shadowed declaration is here [-Werror=shadow]
* src/counterexample.c (visited_hasher): Alpha conversion.
Fix 6b78e50cef, "cex: make "rerun with
'-Wcex'" a note instead of a warning"
* tests/conflicts.at (-W versus %expect and %expect-rr): Fix
expectations.
Unfortunately I found no way to use the ↳ glyph in Texinfo, so I used
@arrow{} instead, which has a different width, so we have to have all
the examples doubled, once for TeX, another for the rest of the world.
* doc/bison.texi: Use the current display in the examples.
* doc/calc.y, doc/ids.y, doc/if-then-else.y, doc/sequence.y: New.
Currently the suggestion to rerun is a -Wother warning:
warning: 2 shift/reduce conflicts [-Wconflicts-sr]
warning: rerun with option '-Wcounterexamples' to generate conflict counterexamples [-Wother]
Instead, let's attach it as a subnote of the diagnostic (in the
current case, -Wconflicts-sr):
warning: 2 shift/reduce conflicts [-Wconflicts-sr]
note: rerun with option '-Wcounterexamples' to generate conflict counterexamples
* src/conflicts.c (conflicts_print): Do that.
Adjust the test suite.
From
input.y: warning: reduce/reduce conflict on token $end [-Wcounterexamples]
Example: A b .
First derivation
a
`-> A b .
Second derivation
a
`-> A b
`-> b .
to
input.y: warning: reduce/reduce conflict on token $end [-Wcounterexamples]
Example: A b .
First reduce derivation
a
`-> A b .
Second reduce derivation
a
`-> A b
`-> b .
* src/counterexample.c (print_counterexample): here.
Compute the width of the labels to properly align the values.
* tests/conflicts.at, tests/counterexample.at, tests/diagnostics.at,
* tests/report.at: Adjust.
Now that the derivation is no longer printed on one line, aligning the
example and the derivation is no longer useful. It can actually be
harmful, as it makes the overall structure less clear.
* src/derivation.h, src/derivation.c (derivation_print_leaves): Remove
the `prefix` argument.
* src/counterexample.c (print_counterexample): Put the example next to
its label.
* tests/conflicts.at, tests/counterexample.at, tests/diagnostics.at,
* tests/report.at: Adjust.
Now that we use complain, the "sections" are clearer.
* src/counterexample.c (print_counterexample): Use the empty line only
in reports.
* tests/counterexample.at, tests/diagnostics.at, tests/report.at: Adjust.
This is more consistent, and brings benefits: users know that these
diagnostics are attached to -Wcounterexamples, and they can also click
on the hyperlink if permitted by their terminal.
We go from
warning: 1 reduce/reduce conflict [-Wconflicts-rr]
Reduce/reduce conflict on token $end:
Example A b .
First derivation a -> [ A b . ]
Second derivation a -> [ A b -> [ b . ] ]
to
warning: 1 reduce/reduce conflict [-Wconflicts-rr]
input.y: warning: reduce/reduce conflict on token $end [-Wcounterexamples]
Example A b .
First derivation a -> [ A b . ]
Second derivation a -> [ A b -> [ b . ] ]
with an hyperlink on -Wcounterexamples.
* src/counterexample.c (counterexample_report_reduce_reduce):
Use complain.
* tests/counterexample.at, tests/diagnostics.at, tests/report.at:
Adjust.
* src/complain.c (begin_hyperlink, end_hyperlink): New.
(warnings_print_categories): Use them.
* tests/local.at (AT_SET_ENV): Disable hyperlinks in the tests, they
contain random id's, and brackets (which is not so nice for M4).
Unfortunately Texinfo somewhat mangles anchors such as `-Werror` into
`g_t_002dWerror`, so let's not include the dash.
* doc/bison.texi (Diagnostics): here.
The code was written on top of buffers of `char[26]`, and then was
changed to use `char *`, yet was still using `sizeof buf`, which
became `sizeof (char *)` instead of `sizeof (char[26])`.
Reported by Dagobert Michelsen.
https://lists.gnu.org/r/bug-bison/2020-07/msg00023.html
* src/glyphs.h, src/glyphs.c: Get rid of uses of `char *`, use only
glyph_buffer_t.
Sometimes, understanding the derivations is difficult, because they
are serialized to fit in one line. For instance, the example taken
from the NEWS file:
%token ID
%%
s: a ID
a: expr
expr: expr ID ',' | "expr"
gave
First example expr • ID ',' ID $end
Shift derivation $accept → [ s → [ a → [ expr → [ expr • ID ',' ] ] ID ] $end ]
Second example expr • ID $end
Reduce derivation $accept → [ s → [ a → [ expr • ] ID ] $end ]
Printing as trees, it gives:
First example expr • ID ',' ID $end
Shift derivation
$accept
↳ s $end
↳ a ID
↳ expr
↳ expr • ID ','
Second example expr • ID $end
Reduce derivation
$accept
↳ s $end
↳ a ID
↳ expr •
* src/glyphs.h, src/glyphs.c (down_arrow, empty, derivation_separator):
New.
* src/derivation.c (derivation_print, derivation_print_impl): Rename
as...
(derivation_print_flat, derivation_print_flat_impl): These.
(fputs_if, derivation_depth, derivation_width, derivation_print_tree)
(derivation_print_tree_impl, derivation_print): New.
* src/counterexample.c (print_counterexample): Adjust.
* tests/conflicts.at, tests/counterexample.at, tests/diagnostics.at,
* tests/report.at: Adjust.
When reporting counterexamples for s/r conflicts, put the shift first.
This is more natural, and displays the default resolution first, which
is also what happens for r/r conflicts where the smallest rule number
is displayed first, and "wins".
* src/counterexample.c (counterexample): Add a shift_reduce member.
(new_counterexample): Adjust.
Swap the derivations when this is a s/r conflict.
(print_counterexample): For s/r conflicts, prefer "Shift derivation"
and "Reduce derivation" rather than "First/Second derivation".
* tests/conflicts.at, tests/counterexample.at, tests/report.at: Adjust.
* NEWS, doc/bison.texi: Ditto.
Currently we use both names. Let's stick to the short one.
* src/AnnotationList.c, src/conflicts.c, src/counterexample.c,
* src/getargs.c, src/getargs.h, src/graphviz.c, src/ielr.c,
* src/lalr.c, src/print-graph.c, src/print-xml.c, src/print.c,
* src/state-item.c, src/state.c, src/state.h, src/tables.c:
s/lookahead_token/lookahead/gi.
* src/counterexample.c, src/parse-simulation.c: It is more usual in
Bison to use sizeof on expressions than on types, especially for
allocation.
Let the compiler do it's job instead of calling memcpy ourselves.
There are too many gl_list_t in there, it's hard to understand what is
going on. Introduce and use more precise types. I sure can be wrong
in some places, it's hard to tell without proper tool support.
* src/counterexample.c, src/lssi.c, src/lssi.h, src/parse-simulation.c,
* src/parse-simulation.h, src/state-item.c, src/state-item.h
(si_bfs_node_list, search_state_list, ssb_list, lssi_list)
(state_item_list): New.
It does not make a lot of sense to use ::= in our counterexamples,
that's not something that belongs to the Bison "vocabulary". Using
the colon makes sense, but it's too discreet. Let's use the arrow,
which we already use in some reports (HTML and Dot).
* src/gram.h (print_dot_fallback): Generalize into...
(print_fallback): this.
(print_arrow): New.
* src/derivation.c: Use it.
* NEWS, tests/conflicts.at, tests/counterexample.at,
* tests/diagnostics.at, tests/report.at: Adjust.
* doc/bison.texi: Ditto.
Unfortunately the literal `→` is output as `↦`. So we need to use
@arrow.
Prefer `&foos[i]` to `foos + i` when `foos` is an array. IMHO, it
makes the semantics clearer.
* src/counterexample.c, src/lssi.c, src/parse-simulation.c,
* src/state-item.c: With arrays, prefer the array notation rather than
the pointer one.
* examples/c/bistromathic/parse.y (user_context): We need the current
line.
(yyreport_syntax_error): Quote the guilty line, with squiggles.
* examples/c/bistromathic/bistromathic.test: Adjust.
Currently autocompletion on a line with errors leaks the error
messages. It can be useful to let the user know, but GNU Readline
does not provide us with an nice way to display the error. So we
actually break into the current line of the user.
So instead, do not show these errors.
* examples/c/bistromathic/parse.y (user_context): New.
Use %param to pass it to the parser and scanner.
Keep quiet when in computing autocompletion.
That quite defeats the whole point of locations... But anyway, we
should not see these messages at all.
* examples/c/bistromathic/parse.y (expected_tokens): Fix (useless)
location tracking.
Currently when we output useless rules, they appear before the
grammar, but using the same invocation. As a result, the anchor is
defined twice, and the wrong one, being first, is honored.
* data/xslt/xml2xhtml.xsl (rule): Take a new 'anchor' parameter to
decide whether being an anchor, or a target.
Let it be true when output the grammar.
* tests/report.at: Adjust.
The text and Dot reports are expected to be identical when generated
directly (--report, --graph) or indirectly (via XML). The xml
testsuite had not be run for ages, let it catch up a bit.
* src/print-xml.c: Pass the type of the symbols.
* data/xslt/xml2text.xsl
Catch up with the new layout.
Display the symbol types.
Use '•', not '.'
* tests/local.at: Smash '•' to '.' when matching against the direct
text report.
* tests/report.at: Adjust XML expectations.
* data/skeletons/c++.m4, data/skeletons/glr.c, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java, data/skeletons/yacc.c:
Be more accurate about yychar and yytoken.
Don't name local variables as if they were members.
* examples/c/bistromathic/parse.y, examples/c/lexcalc/parse.y,
* examples/c/reccalc/parse.y: here.
Add some comments.
* src/parse-gram.y (api_version): Pull out of handle_require.
Bump to 3.7.
* data/bison-default.css: Cobalt does not seem to be supported.
* doc/bison.texi (Counterexamples): A new section.
(Understanding): Show the counterexamples as it shows in the report:
with its items.
(Bison Options): Document -Wcex and -rcex.
In commit c80cdf2db2 ("doc: simplify
uses of @ref", Jan 27 2020, released in Bison 3.6), I broke the
references to the Bibliography. For instance:
For a more detailed exposition of the mysterious behavior in LALR parsers
-and the benefits of IELR, @pxref{Bibliography,,Denny 2008 March}, and
-@ref{Bibliography,,Denny 2010 November}.
+and the benefits of IELR, @pxref{Bibliography}, and
+@ref{Bibliography}.
which results in "see Bibliography" twice, instead of the more precise
reference.
* doc/bison.texi (@pcite, @tcite): New.
Use them instead of @ref to Bibliography.
Cite only the first author (that's what we did for the other entries).
Currently when a push parser finishes its parsing (i.e., it did not
return YYPUSH_MORE), it also clears its state. It is therefore
impossible to see if it had parse errors.
In the context of autocompletion, because error recovery might have
fired, the parser is actually already in a different state. For
instance on `(1 + + <TAB>` in the bistromathic, because there's a
`exp: "(" error ")"` recovery rule, `1 + +` tokens have already been
popped, replaced by `error`, and autocompletions think we are ready
for the closing ")". So here, we would like to see if there was a
syntax error, yet `yynerrs` was cleared.
In the case of a successful parse, we still have a problem: if error
recovery succeeded, we won't know it, since, again, `yynerrs` is
clearer.
It seems much more natural to leave the parser state available for
analysis when there is a failure.
To reuse the parser, we should either:
1. provide an explicit means to reinitialize a parser state for future
parses.
2. automatically reset the parser state when it is used in a new
parse.
Option 2 requires to check whether we need to reinitialize the parser
each time we call `yypush_parse`, i.e., each time we give a new token.
This seems expensive compared to Option 1, but benchmarks revealed no
difference. Option 1 is incompatible with the documentation
("After `yypush_parse` returns a status other than `YYPUSH_MORE`, the
parser instance `yyps` may be reused for a new parse.").
So Option 2 wins, reusing the private `yynew` member to record that a
parse was finished, and therefore that the state must reset in the
next call to `yypull_parse`.
While at it, this implementation now reuses the previously enlarged
stacks from one parse to another.
* data/skeletons/yacc.c (yypstate_new): Set up the stacks in their
initial configurations (setting their bottom to the stack array), and
use yypstate_clear to reset them (moving their top to their bottom).
(yypstate_delete): Adjust.
(yypush_parse): At the beginning, clear yypstate if needed, and at the
end, record when yypstate needs to be clearer.
* examples/c/bistromathic/parse.y (expected_tokens): Do not propose
autocompletion when there are parse errors.
* examples/c/bistromathic/bistromathic.test: Check that case.
Currently, completion when there is a syntax error shows broken
locations.
* examples/c/bistromathic/parse.y (expected_tokens): Initialize the
location.
* examples/c/bistromathic/bistromathic.test: Check that.
The previous commit ("yacc.c: declare and initialize and the same
time") made b4_initialize_parser_state_variables useless.
* data/skeletons/yacc.c (b4_initialize_parser_state_variables): Inline
into...
(yypstate_clear): here.
In order to factor the code of push and pull parsers, the declaration
of the parser's state variable was common (being local variable in
pull parsers, and struct members in push parsers). This result in
rather poor style in pull parser, with first variable declarations,
and then their initializations.
The initialization is about to differ between push and pull parsers,
so it is no longer worth keeping both cases together.
* data/skeletons/yacc.c (b4_declare_parser_state_variables): Accept an
argument, and when it is set, initialize the variables.
Adjust dependencies.
Currently yypull_parse takes a yypstate* as argument, and accepts it
to be NULL. This does not seem to make a lot of sense: rather it is
its callers that should do that.
I believe this is historical: yypull_parse was introduced
first (c3d503425f), with yyparse being a
macro. So yyparse could hardly deal with memory allocation properly.
In 7172e23e8f that yyparse was turned
into a genuine function. At that point, it should have allocated its
own yypstate*, which would have left yypull_parse deal with only one
single non-null ypstate* argument.
Fortunately, it is nowhere documented that it is valid to pass NULL to
yypull_parse. It is now forbidden.
* data/skeletons/yacc.c (yypull_parse): Don't allocate a yypstate.
Needs a location to issue the error message.
(yyparse): Allocate the yypstate.
* README-hacking.md (Working from the Repository): Make it first to
make it easier to find the instructions to build from the repo.
(Implementation Notes): New.
* README: Provide more links.
Reported by Martin Blais and Yuriy Solodkyy.
https://lists.gnu.org/r/help-bison/2020-05/msg00011.htmlhttps://lists.gnu.org/r/bug-bison/2020-06/msg00038.html
While at it, modernize filename_type as api.filename.type and document
it properly.
* data/skeletons/c++.m4 (filename_type): Rename as...
(api.filename.type): this.
Default to const std::string.
* data/skeletons/location.cc (position, location): Expose the
filename_type type.
Use api.filename.type.
* doc/bison.texi (%define Summary): Document api.filename.type.
(C++ Location Values): Document position::filename_type.
* src/muscle-tab.c (muscle_percent_variable_update): Ensure backward
compatibility.
* tests/c++.at: Check that using const file names is ok.
tests/input.at: Check backward compat.
Reported by Dwight Guth.
https://lists.gnu.org/r/bug-bison/2020-06/msg00037.html
* src/AnnotationList.c (AnnotationList__computePredecessorAnnotations):
Beware that SBITSET__FOR_EACH nests _two_ for-loops, so "break" does
not actually break out of it.
That was the only occurrence in the code.
* src/Sbitset.h (SBITSET__FOR_EACH): Warn passersby.
* src/AnnotationList.c, src/ielr.c: Fix include order.
Prefer `res` to `result`.
Reduce scopes.
Be free of the oldish 76 cols limitation when it clutters too much the
code.
Denest when possible (we're starving for horizontal width).
AFAICT, "dotted rule" is a more frequent synonym of "item" than
"pointed rule". So let's migrate to using "dot" only.
* doc/bison.texi: Use dot/'•' rather than point/'.'.
* src/print-xml.c (print_core): Use dot rather than point. This is
not backward compatible, but AFAICT, we don't have actual user of the
XML output (but ourselves). So...
* data/xslt/xml2dot.xsl, data/xslt/xml2text.xsl,
* data/xslt/xml2xhtml.xsl, tests/report.at: ... adjust.
It makes no sense, and is actually confusing, to display twice the
same example with no visible difference.
* src/complain.h, src/complain.c (is_styled): New.
* src/counterexample.c (print_counterexample): Display the unified
example a second time only if it makes a difference.
* tests/conflicts.at, tests/counterexample.at, tests/report.at: Adjust.
* tests/diagnostics.at: Make sure we do display the unifying examples
twice when colors are enabled. And check those colors.
I implemented this to print A ::= [ ], but A ::= [ %empty ] might be
clearer.
* src/parse-simulation.c (nullable_closure): Don't generate null
nonterminal derivations as leaves.
* src/derivation.c (derivation_print_impl): Don't print seperator
spaces for null nonterminal.
* tests/counterexample.at: Update test results.
This was a hack to make it easier for people to migrate from yacc.c to
lalr1.cc and from glr.c to glr.cc: when set, YYSTYPE and YYLTYPE were
`#defined`. It was never documented (just mentioned in NEWS for Bison
2.2, 2006-05-19), but was used to simplify the test suite. Stop that:
adjust the test suite to the skeletons, not the converse.
In C++ use yy::parser::semantic_type, yy::parser::location_type, and
yy::parser::token::MY_TOKEN, instead of YYSTYPE, YYLTYPE and MY_TOKEN.
* data/skeletons/glr.cc, data/skeletons/lalr1.cc: Remove its support.
* tests/actions.at, tests/c++.at, tests/calc.at: Adjust.
Use of print_unicode_char suggested by Bruno Haible.
https://lists.gnu.org/r/bug-gettext/2020-06/msg00012.html
* src/gram.h (print_dot_fallback, print_dot): New.
* src/gram.c, src/derivation.c: Use it.
* tests/counterexample.at, tests/report.at: Adjust the test suite.
* .travis.yml, README-hacking.md: Adjust.
And let --report=all include the counterexamples.
* src/getargs.h, src/getargs.c (report_cex): New.
* src/main.c: Compute counterexamples when -rcex is specified.
* src/print.c: Include the counterexamples when -rcex is specified.
* tests/conflicts.at, tests/existing.at, tests/local.at: Adjust.
Instead of
Shift/reduce conflict on token D:
Example A a • D
First derivation s ::=[ A a a ::=[ b ::=[ c ::=[ • ] ] ] d ::=[ D ] ]
Example A a • D
Second derivation s ::=[ A a d ::=[ • D ] ]
display
Shift/reduce conflict on token D:
Example A a • D
First derivation s ::=[ A a a ::=[ b ::=[ c ::=[ • ] ] ] d ::=[ D ] ]
Example A a • D
Second derivation s ::=[ A a d ::=[ • D ] ]
* src/counterexample.c (print_counterexample): Indent.
* tests/counterexample.at: Adjust.
Showing the items (with the state numbers) is really something we
should restrict to the report.
* src/counterexample.c (counterexample_report_shift_reduce)
(counterexample_report_reduce_reduce): Don't show the pointed rules,
we will do that in the report.
* tests/counterexample.at: Adjust.
* src/conflicts.c (find_state_item_number, report_state_counterexamples):
Move to...
* src/counterexample.h, src/counterexample.c (find_state_item_number)
(counterexample_report_state): this.
Add support for `out` as an argument.
(counterexample_report_reduce_reduce, counterexample_report_shift_reduce):
Accept an `out` argument, and be static.
* upstream/maint:
maint: post-release administrivia
version 3.6.4
glr.cc: don't leak glr.c/glr.cc scaffolding to the user
Some fixes were needed to adjust to recent changes in glr.cc and
glr.c.
* data/skeletons/glr.cc: Stop messing with the user's epilogue to
insert glr.cc code. We need that code to be inserted _before_ the
user's epilogue, not after. So define b4_glr_cc_pre_epilogue.
* data/skeletons/glr.c: Use it.
Until we have a decent reimplementation of glr.cc, we have to use
tricks to shoehorn C++ symbols to the C engine of glr.c. Some of them
are done via #define. Unfortunately in Bison 3.6 some of these we
done in the header file, which broke valid user code.
Reported by Egor Pugin.
https://lists.gnu.org/r/bug-bison/2020-06/msg00003.html
* data/skeletons/glr.cc: Stop playing tricks with b4_pre_epilogue.
(b4_glr_cc_setup, b4_glr_cc_cleanup): New.
Much cleaner way to instal glr.cc's scaffolding around glr.c.
* data/skeletons/glr.c: Adjust to use them.
From
"number" shift, and go to state 1
"Ñùṃéℝô" shift, and go to state 2
to
"number" shift, and go to state 1
"Ñùṃéℝô" shift, and go to state 2
* src/print.c: Use mbswidth, not strlen, to compute visual columns.
* tests/report.at: Adjust.
Currently we use "quotearg" to escape the strings output in Dot. As a
result, if the user's locale is C for instance, all the non-ASCII are
escaped. Unfortunately graphviz does not interpret this style of
escaping.
For instance:
5 -> 2 [style=solid label="\"\303\221\303\271\341\271\203\303\251\342\204\235\303\264\""]
was displayed as a sequence of numbers. We now output:
5 -> 2 [style=solid label="\"Ñùṃéℝô\""]
independently of the user's locale.
* src/system.h (obstack_backslash): New.
* src/graphviz.h, src/graphviz.c (escape): Remove, use
obstack_backslash instead.
* src/print-graph.c: Likewise.
* tests/report.at: Adjust.
Currently our scanner decodes all the escapes in the strings, and we
later reescape the strings when we emit them.
This is troublesome, as we do not respect the user input. For
instance, when the user writes in UTF-8, we destroy her string when we
write it back. And this shows everywhere: in the reports we show the
escaped string instead of the actual alias:
0 $accept: . exp $end
1 exp: . exp "\342\212\225" exp
2 | . exp "+" exp
3 | . exp "+" exp
4 | . "number"
5 | . "\303\221\303\271\341\271\203\303\251\342\204\235\303\264"
"number" shift, and go to state 1
"\303\221\303\271\341\271\203\303\251\342\204\235\303\264" shift, and go to state 2
This commit preserves the user's exact spelling of the string aliases,
instead of interpreting the escapes and then reescaping. The report
now shows:
0 $accept: . exp $end
1 exp: . exp "⊕" exp
2 | . exp "+" exp
3 | . exp "+" exp
4 | . "number"
5 | . "Ñùṃéℝô"
"number" shift, and go to state 1
"Ñùṃéℝô" shift, and go to state 2
Likewise, the XML (and therefore HTML) outputs are fixed.
* src/scan-gram.l (STRING, TSTRING): Do not interpret the escapes in
the resulting string.
* src/parse-gram.y (unquote, parser_init, parser_free, unquote_free)
(handle_defines, handle_language, obstack_for_unquote): New.
Use them to unquote where needed.
* tests/regression.at, tests/report.at: Update.
This is to record the current state of the report, which escapes the
UTF-8 characters (as parse.error="verbose" does), but shouldn't (as
parse.error="detailed" does).
* tests/report.at: here.
Suggesting -Wcounterexamples when there are conflicts is probably not
what the user wants. If she knows her conflicts and has set
%expect/%expect-rr appropriately, we shouldn't warn.
The commit also swaps the counterexamples and the report of conflicts,
into, IMHO, a more natural order: from
Shift/reduce conflict on token B:
1: 3 a: A .
1: 8 y: A . B
Example A • B C
First derivation s ::=[ a ::=[ A • ] x ::=[ B C ] ]
Example A • B C
Second derivation s ::=[ y ::=[ A • B ] c ::=[ C ] ]
input.y: warning: 1 shift/reduce conflict [-Wconflicts-sr]
input.y:4.4: warning: rule useless in parser due to conflicts [-Wother]
to
input.y: warning: 1 shift/reduce conflict [-Wconflicts-sr]
Shift/reduce conflict on token B:
1: 3 a: A .
1: 8 y: A . B
Example A • B C
First derivation s ::=[ a ::=[ A • ] x ::=[ B C ] ]
Example A • B C
Second derivation s ::=[ y ::=[ A • B ] c ::=[ C ] ]
input.y:4.4: warning: rule useless in parser due to conflicts [-Wother]
* src/conflicts.c (rule_conflicts_print): Rename as...
(report_rule_expectation_mismatches): this.
Move the handling of report_counterexamples to...
(conflicts_print): Here.
Display this warning when applicable.
Plural vs. singular is always a problem...
But we already have conflicts-sr and conflicts-rr, so counterexamples
makes more sense than counterexample. Besides, -Wcounterexample will
still be accepted as an unambiguous prefix of -Wcounterexamples.
Add -Wcex as a convenient alias.
While at it, use only "counterexample", never "counter example".
* src/complain.h, src/complain.c
(Wcounterexample, warning_counterexample): Rename as...
(Wcounterexamples, warning_counterexamples): these.
(argmatch_warning_docs): Rename -Wcounterexample as -Wcounterexamples.
(argmatch_warning_args): Likewise.
Add support for -Wcex.
Adjust dependencies.
While defining api.header.include worked as expected, its default
value was incorrectly defined. As a result, by default, the generated
parsers still duplicated the content of the generated header instead
of including it.
* data/skeletons/yacc.c (api.header.include): Fix its default value.
* tests/output.at: Check it.
* doc/bison.texi (%define Summary): Document api.header.include.
While at it, move the definition of api.namespace at the proper
place.
Use colors to show the counterexamples and the derivations in color,
to highlight their structure. Align the outputs, and add i18n
support. Reduce width by using a one-space separator instead of
two-space.
From
Example A • B C
First derivation s ::=[ a ::=[ A • ] x ::=[ B C ] ]
Second derivation s ::=[ y ::=[ A • B ] c ::=[ C ] ]
to
Example A • B C
First derivation s ::=[ a ::=[ A • ] x ::=[ B C ] ]
Example A • B C
Second derivation s ::=[ y ::=[ A • B ] c ::=[ C ] ]
with colors.
* data/bison-default.css (cex-dot, cex-0, cex-1, cex-2, cex-3, cex-4)
(cex-5, cex-6, cex-7, cex-step, cex-leaf): New.
* src/derivation.c (derivation_print_styled_impl): New.
(derivation_print, derivation_print_leaves): Use it.
* src/counterexample.c: Reformat the output.
* tests/counterexample.at: Adjust.
It's unfortunate that the traditions between formal language theory
and Yacc differs, but here, tokens should be upper case, and
nonterminals should be lower case.
* tests/counterexample.at: Comply with this.
In Bison we refer to "shift/reduce" conflicts, not "shift-reduce" (in
Bison 3.6.3 186 occurrences vs 15). Enforce consistency on this.
Instead of "spending" a second line for each conflict to report the
lookaheads, put that on the same line as the type of conflict. Also,
prefer "token" to "symbol". Maybe we should even prefer "lookahead".
While at it, enable internationalization, with plurals where
appropriate.
As a consequence, instead of
Shift-Reduce Conflict:
6: 3 b: . %empty
6: 6 d: c . A
On Symbol: A
display
Shift/reduce conflict on token A:
6: 3 b: . %empty
6: 6 d: c . A
* NEWS, doc/bison.texi, src/conflicts.c: Spell it "shift/reduce", not
"shift-reduce".
* src/counterexample.c (counterexample_report_shift_reduce)
(counterexample_report_reduce_reduce): Reformat and internationalize
output.
* tests/counterexample.at: Adjust expectations.
For instance, in the case of Bison's own parser:
- case 40:
+ case 40: /* grammar_declaration: "%code" "identifier" "{...}" */
{
muscle_percent_code_grow ((yyvsp[-1].ID), (yylsp[-1]),
translate_code_braceless ((yyvsp[0].BRACED_CODE), (yylsp[0])),
(yylsp[0]));
code_scanner_last_string_free ();
}
break;
* data/skeletons/c.m4: Modified.
* data/skeletons/d.m4: Modified.
* data/skeletons/java.m4: Modified.
* src/output.c (output_escaped): New.
(quoted_output): Use it, and rename as...
(output_quoted): this.
Adjust dependencies.
(rule_output): New.
(user_actions_output): Use it.
* data/skeletons/c.m4, data/skeletons/d.m4, data/skeletons/java.m4
(b4_case): Add support for $3, an optional comment.
We were still using GCC9, because GCC10 was failing.
* .travis.yml (PPC64le): Use GCC10.
While at it, use -O2 instead of -O3: it's certainly nicer for the
CPUs, and allows to test different sets of compiler flags (we use -O3
in several other configurations).
* upstream/maint:
maint: post-release administrivia
version 3.6.3
build: check -Wmissing-prototypes
tests: show logs
c++: fix printing of state number on streams
First, we should avoid code such as
gl_list_iterator_t it = gl_list_iterator (deriv->children);
derivation *child = NULL;
while (gl_list_iterator_next (&it, (const void **) &child, NULL))
{
derivation_print (child, f);
because of -Wstrict-aliasing (whose job is to catch type-punning
issues). See https://lists.gnu.org/r/bug-bison/2020-05/msg00039.html.
Rather we need
gl_list_iterator_t it = gl_list_iterator (deriv->children);
const void **p = NULL;
while (gl_list_iterator_next (&it, &p, NULL))
{
derivation *child = (derivation *) p;
derivation_print (child, f);
Second, list iterators actually have destructors. Even though they
are noop in the case of linked-lists, we should use them.
Let's address both issues with typed wrappers (such as
derivation_list_next) that take care of both issues, and besides allow
to scope the iterators within the loop:
derivation *child;
for (gl_list_iterator_t it = gl_list_iterator (deriv->children);
derivation_list_next (&it, &child);
)
{
derivation_print (child, f);
* src/derivation.h, src/derivation.c (derivation_list_next): New.
Use it where appropriate.
* src/counterexample.c (search_state_list_next): New.
Use it where appropriate.
* src/parse-simulation.h, src/parse-simulation.c
* src/state-item.h (state_item_list_next): New.
Use it where appropriate.
pstate_clear is lacking a prototype.
Reported by Ryan
https://lists.gnu.org/r/bug-bison/2020-05/msg00101.html
Besides, none of the C examples were compiled with the warning flags.
* configure.ac (warn_c): Add -Wmissing-prototypes.
* data/skeletons/yacc.c (pstate_clear): Make it static.
* examples/local.mk (TEST_CFLAGS): New.
* examples/c/bistromathic/local.mk, examples/c/calc/local.mk,
* examples/c/lexcalc/local.mk, examples/c/mfcalc/local.mk,
* examples/c/pushcalc/local.mk, examples/c/reccalc/local.mk,
* examples/c/rpcalc/local.mk:
Use it.
GCC's warn_unused_result is not silenced by a cast to void, so we have
to "use" scanf's result.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25509https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66425
Flex generated code produces too many warnings, including things such
as, with ICC:
examples/c/lexcalc/scan.c(1088): error #1682: implicit conversion
of a 64-bit integral type to a smaller integral type (potential portability problem)
2259 YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]),
2260 ^
2261
2262
I am tired of trying to fix Flex's output. The project does not seem
maintained. We ought to avoid it. So, for the time being, don't try
to enable warnings with Flex.
* examples/c/bistromathic/parse.y, examples/c/reccalc/scan.l: Fix
warnings.
* doc/bison.texi: Discard scanf's return value to defeat
-Werror=unused-result.
item_number is used for elements of ritem as well as indices into
ritem which is fairly confusing. Introduce item_index to represent
indices into ritem.
* src/gram.h (item_index): Introduce it for ritem indices.
* src/closure.h, src/closure.c, src/ielr.c, src/lr0.c,
* src/print-graph.c, src/state.h, src/state.h:
Replace uses of item_number with item_index where appropriate.
Teaches bison about a new command line option, --file-prefix-map OLD=NEW
(based on the -ffile-prefix-map option from GCC) which causes it to
replace and file path of OLD in the text of the output file with NEW,
mainly for header guards and comments. The primary use of this is to
make builds reproducible with different input paths, and in particular
the debugging information produced when the source code is compiled. For
example, a distro may know that the bison source code will be located at
"/usr/src/bison" and thus can generate bison files that are reproducible
with the following command:
bison --output=/build/bison/parse.c -d --file-prefix-map=/build/bison/=/usr/src/bison/ parse.y
Importantly, this will change the header guards and #line directives
from:
#ifndef YY_BUILD_BISON_PARSE_H
#line 100 "/build/bison/parse.h"
to
#ifndef YY_USR_SRC_BISON_PARSE_H
#line 100 "/usr/src/bison/parse.h"
which is reproducible.
See https://lists.gnu.org/r/bison-patches/2020-05/msg00016.html
Signed-off-by: Joshua Watt <[email protected]>
* src/files.h, src/files.c (spec_mapped_header_file)
(mapped_dir_prefix, map_file_name, add_prefix_map): New.
* src/getargs.c (-M, --file-prefix-map): New option.
* src/output.c (prepare): Define b4_mapped_dir_prefix and
b4_spec_header_file.
* src/scan-skel.l (@ofile@): Output the mapped file name.
* data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, data/skeletons/location.cc,
* data/skeletons/yacc.c:
Adjust.
* doc/bison.texi: Document.
* tests/input.at, tests/output.at: Check.
Instead of `On Symbols: {b,c,}`, display `On Symbols: b, c`.
* src/counterexample.c (counterexample_report_reduce_reduce): We don't
need braces.
Use commas as a separator, not a terminator.
* tests/counterexample.at: Adjust.
Instead of generating switch statements with numbers, let's use the
symbol kinds. Not only is this more readable, it also makes reading
diff easier, as a change in symbol numbers won't have such a large
effect on the implementation of symbol actions.
* data/skeletons/bison.m4 (_b4_symbol_case): Use the symbol kind
rather than its number.
This should have been done in 3.6, but I wanted to avoid introducing
conflicts into Vincent's work on counterexamples. It turns out it's
completely orthogonal.
* data/README.md, data/skeletons/bison.m4, data/skeletons/c++.m4,
* data/skeletons/c.m4, data/skeletons/glr.c, data/skeletons/java.m4,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/variant.hh, data/skeletons/yacc.c, src/conflicts.c,
* src/derives.c, src/gram.c, src/gram.h, src/output.c,
* src/parse-gram.c, src/parse-gram.y, src/print-xml.c, src/print.c,
* src/reader.c, src/symtab.c, src/symtab.h, tests/input.at,
* tests/types.at:
s/user_token_number/code/g.
Plus minor changes.
* src/parse_simulation.c: Fix bug in parse_state_free.
Free new_root when simulate_reduction generates zero states.
* src/parse-simulation.c, src/parse-simulation.h
(parse_state_list, parse_state_list_append): New.
* src/parse-simulation.c, src/parse-simulation.h,
* src/counterexample.c: Replace all uses of lists of parse states and
appends to parse_state_lists with the new API.
* src/derivation.h, src/derivation.c: Make derivation struct opaque.
Add derivation_list type for clarity.
(derivation_list_new): New.
(derivation_list_append): New.
(derivation_list_prepend): New.
(derivation_new_leaf): New constructor for derivations with no
children.
* src/counterexample.c, src/parse-simulation.c,
* src/parse-simulation.h: Replace uses of gl_list_t containing
derivations with derivation_list and its API.
Replace calls of dervation_new using null children with
derivation_new_leaf.
* src/parse-simulation.c: replace ps_chunk and its API with typed
versions si_chunk and deriv_chunk.
* src/parse-simlation.h, src/parse-simulation.c: Remove
parse_state_retain_deriv in favor of derivation reference counting.
* src/counterexample.c: Remove search_state_retain_deriv.
There's no need to go for the heap when using gnulib's hash module.
* src/state-item.c (hash_pair_lookup, hash_pair_remove,
state_sym_lookup): Use the heap Luke.
That removes a leak from hash_pair_lookup.
(init_prods): Use hash_pair_insert instead of duplicating it.
* src/state-item.h, src/state-item.c (state_item): Make the state
const.
(state_item_set): Make it clearer that it works in the state_items
global array.
* src/counterexample.c: Use 'res' as a variable name for returned
value, as elsewhere.
Avoid uninitialized variables, especially pointers.
Avoid assignment where possible.
The CI has "failures" such as (253, "Null nonterminals"):
@@ -21,7 +21,7 @@
3: 3 b: . %empty
3: 4 c: . %empty
On Symbols: {A,}
-time limit exceeded: 6.000000
+time limit exceeded: 11.000000
First Example c • c A A $end
First derivation $accept ::=[ a ::=[ c d ::=[ a ::=[ b ::=[ • ] d ::=[ c A A ] ] ] ] $end ]
Second Example c • A $end
* tests/counterexample.at (AT_BISON_CHECK_CEX): New.
Use it to neutralize differences in timeout values.
* src/parse-simulation.c: Replace reference counting with
parse_state_retain everywhere.
(free_parse_state): Make this function iterative instead of
recursive. Long parse_state chains were causing stack exhaustion.
* tests/counterexample.at: Fix expectations.
Fixes the SEGV in test 247 (counterexample.at:195): "S/R after first
token".
* src/counterexample.c: here.
* tests/counterexample.at: Fix expectations.
* src/counterexample.c, src/derivation.c:
Do not output diagnostics on stdout, that's the job of stderr, and the
testsuite heavily depend on this.
Do not leave trailing spaces in the output.
* tests/counterexample.at: Use AT_KEYWORDS.
Specify the expected outputs.
* tests/local.mk: Add counterexample.at.
* src/complain.h, src/complain.c: Add support for -Wcounterexample.
* src/conflicts.c (report_counterexamples): New.
(rule_conflicts_print): Use it when -Wcounterexample is given.
* src/getargs.h, src/getargs.c: Add support for --trace=cex.
* src/main.c (main): Init and deinit counterexample generation.
In Bison 3.6.2, the comments with brackets lose their brackets, for
improper m4 quotation.
* data/skeletons/bison.m4 (b4_gsub): New.
* data/skeletons/c-like.m4 (_b4_comment): Use it.
* tests/m4.at: Check b4_gsub.
Let --trace=m4-early dump all the logs from the start (as --trace=m4
used to do), and have --trace=m4 now start traces only when actually
working of the user's grammar.
Can make a big difference in the case of small inputs. E.g.
$ bison -S tests/testsuite.dir/001/input.m4 tests/testsuite.dir/001/input.y --trace=m4 |& wc
3952 19446 251068
$ bison -S tests/testsuite.dir/001/input.m4 tests/testsuite.dir/001/input.y --trace=m4-early |& wc
19491 131904 1830495
* data/skeletons/traceon.m4: New.
* src/getargs.h, src/getargs.c: Introduce --trace=m4-early.
* src/output.c (output_skeleton): Adjust for --trace=m4 and --trace=m4-early.
Commit a4ede8f85b ("package: make bison
a relocatable package") made Bison relocatable, but in fact it still
contains one absolute reference: the M4 variable, which points to the
M4 program. Let's fix that by using relocate(), see if an M4 binary is
available at the relocated location, and otherwise fallback to the
original M4 location.
See https://lists.gnu.org/r/bison-patches/2020-05/msg00078.html,
and https://lists.gnu.org/r/bison-patches/2020-05/msg00087.html.
* src/files.h, src/files.c (m4path): New.
* src/output.c: Use it.
* build-aux/update-test: When given a directory, use the testsuite.log
which it contains.
Do not accept empty "from"s, as substituting the empty string with
something is rarely a good idea.
With input such as
%token<fl> yVL_CLOCK "/*verilator sc_clock*/"
we generate
yVL_CLOCK = 610, /* "/*verilator sc_clock*/" */
which is invalid since the comment will actually be closed on the
first "*/". Let's turn "*/" into "*\/" to avoid this. But GCC will
also warn about "/*" inside a comment, so let's "escape" it too.
Reported by Huang Rui.
https://github.com/akimd/bison/issues/38
* data/skeletons/c-like.m4 (_b4_comment): Escape comment delimiters in
comments.
* tests/input.at (Torturing the Scanner): Check thes cases.
* tests/m4.at: New.
c.m4 contains a definition of _Noreturn which is modeled after
gnulib's lib/_Noreturn.h. The latter was recently
changed (b61bf2f0e8) to not using
[[noreturn]] at all, because the uses of _Noreturn in gnulib are
sometimes incompatible with the rules of [[noreturn]].
As a result glr.cc started to use _Noreturn in C++, which clang
refuses (all the glr.cc tests currently fail with Clang++).
* data/skeletons/c.m4 (b4_attribute_define): Restore the definition of
_Noreturn as [[noreturn]] in modern C++.
The generated code uses _Noreturn in places where [[noreturn]] is
valid.
* etc/bench.pl.in, examples/c/bistromathic/parse.y,
* examples/c/calc/calc.y, examples/c/pushcalc/calc.y: Check scanf's
return value.
* doc/bison.texi: Likewise, but only for the second example, to avoid
cluttering the very simple case.
* build-aux/update-test: When given a directory, use the testsuite.log
which it contains.
Do not accept empty "from"s, as substituting the empty string with
something is rarely a good idea.
* maint:
news: update
maint: post-release administrivia
version 3.6.1
c++: style: reorder generated code
c++: provide yy::parser::symbol_type::name
c++: make parser::symbol_name public
examples: beware of ~/.inputrc
build: also provide lzip compressed tarballs
style: minor fixes
yacc.c: restore ansi-c compatibility
Reported by Paul Eggert.
* src/getargs.c: We don't need it anyway, since we use _Noreturn.
* data/skeletons/c.m4: While at it, update the definition of _Noreturn
stolen from gnulib.
The implementation of yy::parser::symbol_name is emitted even before
the implementation of yy::parser::parser. This makes little sense.
* data/skeletons/lalr1.cc (symbol_name): Move its implementation in
the same place as in the class definition: after "error" and before
"context".
Reported by Martin Blais <[email protected]>.
https://lists.gnu.org/r/help-bison/2020-05/msg00005.html
* data/skeletons/lalr1.cc (symbol_name): Make it public.
Add a private hidden hook to enable testing of private parts.
* tests/local.at (AT_DATA_GRAMMAR_PROLOGUE): Help Emacs find the right
language mode.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Check that we
can read symbol_name.
Suggested by Matias Fonzo <[email protected]>.
* cfg.mk: Post announcements to bison-announce.
* configure.ac: Build lzip packages.
* .travis.yml: Build only xz, we don't care about the other formats
here.
* gnulib: Update.
* bootstrap.conf: Use attribute.
* src/system.h: Remove macros for attributes.
Adjust dependencies.
* src/scan-gram.l (DEPRECATED): Rename as...
(DEPRECATED_DIRECTIVE): this, to avoid the clash with the DEPRECATED macro.
Suggested by Matias Fonzo <[email protected]>.
* cfg.mk: Post announcements to bison-announce.
* configure.ac: Build lzip packages.
* .travis.yml: Build only xz, we don't care about the other formats
here.
I don't think it's fair to have yypstate_new, yypstate_delete,
yypush_parse and yypull_parse to have their own section, on par with
yyparse and yylex. Let them be in a single section about push
parsers. And show new/delete first.
* doc/bison.texi (Push Parser Interface): New.
Fuse the aforementioned sections into it.
The user gives yyexpected_tokens a limit: the max number of tokens she
wants to hear about. That's because an error message that reports a
bazillion of possible tokens is useless.
In that case yyexpected_tokens returned 0, so the user would not know
if there are too many expected tokens or none (yes, that's possible).
There are several ways to tell the user in which situation she's in:
- return some E2MANY, a negative value. Then it makes the pattern
int argsize = yypcontext_expected_tokens (ctx, arg, ARGS_MAX);
if (argsize < 0)
return argsize;
no longer valid, as for E2MANY (i) the user must generate the error
message anyway, and (ii) she should not return E2MANY
- return ARGS_MAX + 1. Then it makes it dangerous for the user, as
she has to iterate update `min (ARGS_MAX, argsize)`.
Returning 0 is definitely simpler and safer for the user, as it tells
her "this is not an error, just generate your message without a list
of expecting tokens". So let's still return 0, but set arg[0] to the
empty token when the list is really empty.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/lalr1.java
* data/skeletons/yacc.c (yyexpected_tokens): Put the empty symbol
first if there are no possible tokens at all.
* examples/c/bistromathic/parse.y: Demonstrate how to use that.
The shell grammar does not allow empty statements in then/else part of
an if, but examples/test failed to catch the syntax errors from the
script it ran. So exited with success anyway.
You would expect 'set -e' to suffice, but with bash 3.2 actually it
does not. As a matter of fact, I could find a way to have this behave
properly:
$ cat test.sh
set -e
cleanup ()
{
status=$?
echo "cleanup: $status"
exit $status
}
trap cleanup 0 1 2 13 15
. $1
s=$?
echo "test.sh: $s"
exit $s
$ cat bistro.test
if true; then
fi
$ /bin/sh ./test.sh ./bistro.test
./bistro.test: line 2: syntax error near unexpected token `fi'
cleanup: 0
$ echo $?
0
Remove the set -e (or the trap), and tada, it works... So we have to
deal with the error by hand.
* examples/test ($exit): Replace with...
($status): this.
Preserve the exit status of the test case.
* examples/c/bistromathic/bistromathic.test: Fix syntax error.
For instance test 386, "glr.cc api.value.type={double}":
types.at:366: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o test test.cc $LIBS
stderr:
test.cc: In function 'ptrdiff_t yysplitStack(yyGLRStack*, ptrdiff_t)':
test.cc:490:4: error: 'PTRDIFF_MAX' was not declared in this scope
(PTRDIFF_MAX < SIZE_MAX ? PTRDIFF_MAX : YY_CAST (ptrdiff_t, SIZE_MAX))
^
test.cc:1805:37: note: in expansion of macro 'YYSIZEMAX'
ptrdiff_t half_max_capacity = YYSIZEMAX / 2 / state_size;
^~~~~~~~~
test.cc:490:4: note: suggested alternative: '__PTRDIFF_MAX__'
(PTRDIFF_MAX < SIZE_MAX ? PTRDIFF_MAX : YY_CAST (ptrdiff_t, SIZE_MAX))
^
test.cc:1805:37: note: in expansion of macro 'YYSIZEMAX'
ptrdiff_t half_max_capacity = YYSIZEMAX / 2 / state_size;
^~~~~~~~~
The failing tests are using glr.cc only, which I don't understand, the
problem is rather in glr.c, so I would expect glr.c tests to also fail.
Reported by Bruno Haible.
https://lists.gnu.org/archive/html/bug-bison/2020-05/msg00053.html
* data/skeletons/yacc.c: Move the block that defines
YYPTRDIFF_T/YYPTRDIFF_MAXIMUM, YYSIZE_T/YYSIZE_MAXIMUM, and
YYSIZEOF to...
* data/skeletons/c.m4 (b4_sizes_types_define): Here.
(b4_c99_int_type): Also take care of the #undefinition of short.
* data/skeletons/yacc.c, data/skeletons/glr.c: Use
b4_sizes_types_define.
* data/skeletons/glr.c: Adjust to use YYPTRDIFF_T/YYPTRDIFF_MAXIMUM,
YYSIZE_T/YYSIZE_MAXIMUM.
One function missing on Solaris 10 Sparc:
CCLD examples/c/bistromathic/bistromathic
Undefined first referenced
symbol in file
strnlen examples/c/bistromathic/bistromathic-parse.o
ld: fatal: symbol referencing errors. No output written to examples/c/bistromathic/bistromathic
Reported by Dagobert Michelsen.
https://lists.gnu.org/r/bug-bison/2020-05/msg00048.html
* examples/c/bistromathic/parse.y (xstrndup): Don't use strnlen.
xstrndup is assembled from gnulib's xstrndup, strndup and strnlen...
AIX 7.1 supports diff -u, but its output does not match the expected
one.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00049.html
* tests/atlocal.in (DIFF_U_WORKS): New.
* tests/local.at (AT_DIFF_U_CHECK): New.
* tests/existing.at (_AT_TEST_EXISTING_GRAMMAR): Use AT_DIFF_U_CHECK.
Readline may emit escape sequences before the prompt.
Reported by Bruno Haible.
https://lists.gnu.org/r/platform-testers/2020-05/msg00001.html.
* examples/c/bistromathic/bistromathic.test: Trust readline _only_ if
we get what we expect on some reference computation.
Appearing on IRIX with gcc -mabi=n32.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00039.html
* examples/c++/variant-11.yy, examples/c/bistromathic/parse.y: Don't
give chars to isdigit, cast them to unsigned char before.
* src/complain.c: Use c_isdigit.
* src/fixits.c (fixits_run): Avoid casts.
* src/lalr.c (goto_print): Use %zu for a size_t.
Don't try to build bistromathic if we don't have readline.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00028.html
* configure.ac (ENABLE_BISTROMATHIC): New.
* examples/c/bistromathic/local.mk: Use it.
* examples/c/bistromathic/bistromathic.test: Exit 77 for skip.
"foo || bar" does not invoke bar on AIX 7.2 when foo does not exist.
It just dies.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00029.html
* examples/c/reccalc/reccalc.test: Check for seq in a subshell.
* data/skeletons/lalr1.java (Location): Make it a static class.
(Lexer.yylex, Lexer.getLVal, Lexer.getStartPos, Lexer.getEndPos):
These are not needed in push parsers.
* examples/java/calc/Calc.y: Demonstrate push parsers in the Java.
* doc/bison.texi: Push parsers have been supported for a long time,
remove incorrect statements stating the opposite.
Reported by Horst von Brand.
https://lists.gnu.org/r/bug-bison/2020-04/msg00033.html
* examples/c/bistromathic/Makefile: libintl might not be needed, but
libm probably is.
* examples/c/bistromathic/parse.y: Include locale.h.
On OpenBSD 6.5, the prompt is repeated, but not the actual command
line... Don't try to cope with that.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00015.html
* examples/c/bistromathic/bistromathic.test: Skip when readline behave
this way.
The option --header was introduced in version 2.5.6.
The option --header-file was introduced in version 2.6.4.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00013.html
So use --header, and do bother with versions that don't support it.
* m4/flex.m4: Check whether flex supports --header.
* configure.ac (FLEX_WORKS, FLEX_CXX_WORKS): Set to false if it doesn't.
* * examples/c/reccalc/local.mk, examples/c/reccalc/Makefile:
Use --header rather than --header-file.
To write unit tests for their scanners, some users depended on
symbol_type::token():
Lexer lex("12345");
symbol_type t = lex.nextToken();
assert(t.token() == token::INTLIT);
assert(t.value.as<int>() == 12345);
But symbol_type::token() was removed in Bison 3.5 because it relied on
a conversion table. So users had to find other patterns, such as
assert(t.type_get() == by_type(token::INTLIT).type_get());
which relies on several private implementation details.
As part of transitioning from "token type" to "token kind", and making
this a public and documented interface, "by_type" was renamed
"by_kind" and "type_get()" was renamed as "kind()". The latter had
backward compatibility mechanisms, not the former.
In Bison 3.6 none of this should be used, but rather
assert(t.kind() == symbol_kind::S_INTLIT);
Reported by Pramod Kumbhar.
https://lists.gnu.org/r/bug-bison/2020-05/msg00012.html
* data/skeletons/c++.m4 (by_type): Make it an alias to by_kind.
I don't plan to fix everything in one go. But this was in the way of
the next commit.
* data/skeletons/lalr1.java: Avoid space before parens.
* tests/java.at: Adjust.
Bison emits strings to translate in the generated code, for builtin
tokens. So they appear only in generated parsers, which are not
shipped, so they are not in the src tree, so we cannot use them in our
POTFILE.
Except src/parse-gram.c, which is in the source tree. And even in the
git repo. But to avoid useless diffs in the repo, we do not keep the
src/parse-gram.c _with_ the #lines. This is done in a dist-hook which
regenerates src/parse-gram.c when we run "make dist".
Unfortunately, then, update-po traverses the whole tree and sees that
the location of the strings to translate in src/parse-gram.c have
changed, so the bison.pot is to be updated. And that is not possible
in the "make dist" which is run within "make distcheck"
(not the one preparing the dist for distcheck, the one run by
distcheck to check that a distributed tarball can build a tarball)
because then the src tree is read-only.
So let's not put src/parse-gram.c in the POTFILE, and expose these
strings to gettextize by hand.
* src/i18n-strings.c: New.
* po/POTFILES.in: Add it, and remove src/parse-gram.c.
Some people have been using that symbol. Some even have #defined it
themselves.
https://lists.gnu.org/r/bison-patches/2020-04/msg00138.html
Let's provide backward compatibility, having it point to YYUNDEF, so
that an error message is generated.
* data/skeletons/yacc.c (YYERRCODE): New, at the exact same location
it was defined before.
These are internal details. `type_get ()` is still there to ensure
backward compatibility, `kind ()` being the modern way.
* data/skeletons/c++.m4 (by_type, by_type::type): Rename as...
(by_kind, by_kind::kind_): this.
Adjust dependencies.
From
public interface Lexer {
/* Token kinds. */
/** Token number, to be returned by the scanner. */
static final int YYEOF = 0;
/** Token number, to be returned by the scanner. */
static final int YYERRCODE = 256;
/** Token number, to be returned by the scanner. */
static final int YYUNDEF = 257;
/** Token number, to be returned by the scanner. */
static final int BANG = 258;
...
/** Deprecated, use b4_symbol(0, id) instead. */
public static final int EOF = YYEOF;
to
public interface Lexer {
/* Token kinds. */
/** Token "end of file", to be returned by the scanner. */
static final int YYEOF = 0;
/** Token error, to be returned by the scanner. */
static final int YYerror = 256;
/** Token "invalid token", to be returned by the scanner. */
static final int YYUNDEF = 257;
/** Token "!", to be returned by the scanner. */
static final int BANG = 258;
...
/** Deprecated, use YYEOF instead. */
public static final int EOF = YYEOF;
* data/skeletons/java.m4 (b4_token_enum): Display the symbol's tag in
comment.
* data/skeletons/lalr1.java: Address overquotation issue.
* examples/java/calc/Calc.y, examples/java/simple/Calc.y: Use YYEOF,
not EOF.
On an invalid character literal such as "'\777'" we used to produce
two errors:
input.y:2.9-12: error: invalid number after \-escape: 777
input.y:2.8-13: error: empty character literal
Get rid of the second one.
* src/scan-gram.l (STRING_GROW_ESCAPE): New.
* tests/input.at: Adjust.
I'm quite pleased to see that the tricky case of glr.c was already
prepared by the changes to support syntax_error exceptions. Better
yet, it is actually syntax_error that becomes a special case of the
general pattern: make yytoken be YYERRCODE.
* data/skeletons/glr.c (YYFAULTYTOK): Remove the now useless (Basil)
Faulty token.
Instead, use the error token.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java: When computing
the action, first check the case of the error token.
* tests/calc.at: Check cases for the error token symbols before and
after it.
* data/skeletons/yacc.c (yyparse): When the scanner returns YYERRCODE,
go directly to error recovery (yyerrlab1).
However, don't keep the error token as lookahead, that token is too
special.
* data/skeletons/lalr1.cc: Likewise.
* examples/c/bistromathic/parse.y (yylex): Use that feature to report
nicely invalid characters.
* examples/c/bistromathic/bistromathic.test: Check that.
* examples/test: Neutralize gratuitous differences such as rule
position.
* tests/calc.at: Check that case in C only.
The other case seem to be working, but that's an illusion that the
next commit will address (in fact, they can enter endless loops, and
report the error several times anyway).
* examples/c/bistromathic/parse.y: here.
* examples/c/bistromathic/bistromathic.test: Check it.
Included a stupid case where the error is actually ignored.
When the user ctrl-d the line, we left the cursor not at col 0.
Let's fix that.
This revealed a few short-comings in the testing framework.
* examples/test (run): Also display the diffs.
And support -n.
* examples/c/bistromathic/bistromathic.test
* examples/c/bistromathic/parse.y
We will not keep YYERRCODE anyway, it causes backward compatibility
issues. So as a first step, let all the skeletons use that name,
until we have a better one.
* data/skeletons/bison.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/lalr1.cc,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/yacc.c, doc/bison.texi, tests/headers.at,
* tests/input.at:
here.
* etc/bench.pl.in: Don't force parse.error=detailed
Use a simpler way to display the pseudo %bison directive.
(&bench_with_gbenchmark): Give details about the compiler.
We could try to avoid the weird "#if 1", but then the indentation of
the inner #if would be wrong. Let' keep it this way.
* data/skeletons/yacc.c: here.
Also, avoid sticking the comment to the directive.
Currently it was only using stubs. Let's actually translate the
strings using gettext.
* examples/c/bistromathic/local.mk: Define LOCALEDIR, BISON_LOCALEDIR
and link with libintl.
* examples/c/bistromathic/parse.y: Use them.
Remove useless includes.
Take ENABLE_NLS into account.
(error_format_string): New.
(yyreport_syntax_error): Rewrite to rely on a format string, which is
more appropriate for internationalization.
* examples/c/bistromathic/Makefile: We no longer use Flex.
We need readline and intl.
* doc/bison.texi: Point to bistromathic for a better option for
internationalization.
* po/POTFILES.in: Add bistromathic.
In C/C++, N_ is a no-op. Define it if the user didn't.
Suggested by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2020-04/msg00010.html
* src/output.c (prepare_symbol_names): Rename has_translations as
has_translations_flag.
* data/skeletons/bison.m4 (b4_has_translations_if): New.
* data/skeletons/java.m4 (b4_trans): Use it.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/yacc.c
(N_): Provide a default definition.
* src/output.c (prepare_symbol_names): Don't play tricks with the
symbols, it's quite too late.
(has_translations): Move to...
* src/symtab.c: here.
(symbols_pack): Use it to enable translation for special symbols.
On macOS, wc -l always prepends the result with a tab, even when fed
by stdin. But anyway, we should have used `grep -c -v`, which appears
to be portable according to Autoconf's "Limitations of Usual Tools"
section.
Reported by Denis Excoffier.
https://lists.gnu.org/r/bug-bison/2020-04/msg00009.html
* tests/calc.at (_AT_CHECK_CALC): Use grep's -c instead.
macOS' version of readline does not repeat stdin on stdout in
non-interactive mode, contrary to the current version of GNU readline.
* examples/test: Add support for strip_prompt.
* examples/c/bistromathic/bistromathic.test (strip_prompt): Set it
when needed.
Early exit when needed.
symbol_type::token () was removed: it returned the token kind of a
symbol. To do that, one needs to convert from the symbol kind to the
token kind, which requires a table.
This broke some users' unit tests for scanners, see
https://lists.gnu.org/r/bug-bison/2020-01/msg00001.htmlhttps://lists.gnu.org/r/bug-bison/2020-03/msg00020.htmlhttps://lists.gnu.org/r/help-bison/2020-04/msg00005.html
Instead of making this possible again, let's check the symbol's kind
instead. So give proper access to a symbol's kind.
That feature existed, undocumented, as 'type_get()'. Let's rename
this as 'kind()'.
* data/skeletons/c++.m4, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc (type_get): Rename as...
(kind): This.
(type_get): Install a backward compatibility alias.
* doc/bison.texi (Complete Symbols): Document symbol_type and
symbol_type::kind.
* data/skeletons/c++.m4: Define the old names in terms on the new
ones, instead of the converse.
* doc/bison.texi (C++ Parser Interface): Be more extensive about
token_kind_type.
Not all the symbols have a fixed symbol code. UNDEF's one is fixed:
-2.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/yacc.c: here.
Why didn't I think about this before??? symbolName should be a method
of SymbolKind.
* data/skeletons/lalr1.java (YYParser::yysymbolName): Move as...
* data/skeletons/java.m4 (SymbolKind::getName): this.
Make the table a static final table, not a local variable.
Adjust dependencies.
* doc/bison.texi (Java Parser Interface): Document i18n.
(Java Parser Context Interface): Document SymbolKind.
* examples/java/calc/Calc.y, tests/local.at: Adjust.
* doc/bison.texi (C++ Parser Context): New.
* data/skeletons/lalr1.cc (parser::yysymbol_name): Rename as...
(parser::symbol_name): this.
(A Complete C++ Example): Promote LAC, now that we have it.
Promote parse.error detailed over verbose.
* examples/c++/calc++/calc++.test, tests/local.at: Adjust.
* NEWS (Deep overhaul of the symbol and token kinds): New.
* doc/bison.texi: Promote YYEOF over "0" in scanners.
(Token Decl): No longer show YYEOF here, it now works by default.
(Token I18n): More details about YYEOF here.
(Calc++): Just use YYEOF.
I have been hesitating a lot before doing it ---after all the user
must not use this kind, so what's the point of showing it in
yytoken_kind_t. And eventually I chose to play it safe with the
typing system and make it possible to use yytoken_kind_t for all the
tokens, even the "empty token".
* data/skeletons/c.m4: Give an id and a tag to YYEMPTY.
(b4_token_enums): Define YYEMPTY.
* data/skeletons/c++.m4 (b4_token_enums): Define YYEMPTY.
* data/skeletons/glr.c, data/skeletons/glr.cc, data/skeletons/yacc.c:
(YYEMPTY): Remove.
Use b4_symbol(-2, id) instead.
The user should think of yypcontext fields as accessible only via
yypcontext_* functions. So let's rename yyexpected_tokens to reflect
that.
Let's _not_ rename yyreport_syntax_error, as the user may define this
function, and is not allowed to access directly the fields of
yypcontext_t: she *must* use the "accessors". This is comparable to
the case of C++/Java where the user defines
parser::report_syntax_error, not parser::context::report_syntax_error.
* data/skeletons/glr.c, data/skeletons/yacc.c (yyexpected_tokens):
Rename as...
(yypcontext_expected_tokens): this.
Adjust dependencies.
The name "$end" is nice in the report, in particular it avoids that
pointed-rules (aka items) be too long. It also helps keeping them
"standard".
But it is bad in error messages, we should report "end of file" (or
maybe "end of input", this is debatable). So, unless the user already
defined the alias for the error token herself, make it "end of file".
It should even be translated if the user already translated some
tokens, so that there is now no strong reason to redefine the $end
token.
* src/output.c (prepare_symbol_names): Issue "end of file" instead of
"$end".
* data/skeletons/lalr1.java (yytnamerr_): Remove the renaming hack.
* build-aux/update-test: Accept files with names containing a "+",
such as c++.at.
* tests/actions.at, tests/c++.at, tests/conflicts.at,
* tests/glr-regression.at, tests/regression.at, tests/skeletons.at:
Adjust.
Yet, don't change the structure identifier to avoid introducing
conflicts in Vincent Imbimbo's PR (which, amusingly enough, is about
conflicts).
* src/symtab.c: here.
* tests/diagnostics.at, tests/input.at: Adjust.
yy::parser features a parse() function, not a yyparse() one.
* data/skeletons/lalr1.cc (yyreport_syntax_error)
(context::yyexpected_tokens): Rename as...
(report_syntax_error, context::expected_tokens): these.
Currently EOF is handled in an adhoc way, with a #define YYEOF 0 in
the implementation file. As a result, the user has to define her own
EOF token if she wants to use it, which is a pity.
Give the $end token a visible kind name, YYEOF. Except that in C,
where enums are not scoped, we would have collisions between all the
definitions of YYEOFs in the header files, so in C, make it
<api.PREFIX>EOF.
* data/skeletons/c.m4 (YYEOF): Override its name to avoid collisions.
Unless the user already gave it a different name.
* data/skeletons/glr.c (YYEOF): Remove.
Use ]b4_symbol(0, [id])[ instead.
Add support for "pre_epilogue", for glr.cc.
* data/skeletons/glr.cc: Remove dead code (never emitted #undefs).
* data/skeletons/yacc.c
* src/parse-gram.c
* src/reader.c
* src/symtab.c
* tests/actions.at
* tests/input.at
* data/skeletons/bison.m4 (b4_symbol_token_kind): Give a definition to
$undefined.
(b4_token_visible_if): $undefined has an id.
* src/output.c (prepare_symbol_definitions): Stop lying: $undefined
_is_ a token.
* tests/input.at: Adjust.
There are people out there that do use YYERRCODE (the token kind of
the error token). See for instance
https://github.com/borbolla-automation/SPC_Machines/blob/3812012bb782bfdfe7b325950a35cd337925fcad/unixODBC-2.3.2/Drivers/nn/yylex.c.
Currently, YYERRCODE is defined by yacc.c in an adhoc way as a #define
in the *.c file only. It belongs with the other token kinds.
YYERRCODE is not a nice name, it does not fit in our naming scheme.
YYERROR would be more logical, but it collides with the YYERROR macro.
Shall we keep the same name in all the skeletons? Besides, to avoid
collisions in C, we need to apply the api prefix: YYERRCODE is
actually <PREFIX>ERRCODE. This is not needed in the other languages.
* data/skeletons/bison.m4 (b4_symbol_token_kind): New.
Map the error token to "YYERRCODE".
* data/skeletons/yacc.c (YYERRCODE): Don't define it, it's handled by...
* src/output.c (prepare_symbol_definitions): this.
* tests/input.at (Redefining the error token): Check it.
* data/skeletons/bison.m4 (b4_symbol_kind): Dispatch on the UNDEF
token number rather than its name.
* data/skeletons/c++.m4, data/skeletons/c.m4, data/skeletons/java.m4:
Comment changes.
A forthcoming commit (tokens: properly define the "error" token kind)
revealed a problem in the C++ generated headers: they are not
self-contained. With this file:
%language "c++"
%define api.value.type variant
%code {
static int yylex (yy::parser::semantic_type *lvalp);
}
%token <int> X
%%
exp:
X { printf ("x\n"); }
;
%%
void
yy::parser::error (const std::string& m)
{
std::cerr << m << '\n';
}
static
int yylex (yy::parser::semantic_type *lvalp)
{
static int const input[] = {yy::parser::token::X, 0};
static int toknum = 0;
return input[toknum++];
}
int
main (int argc, char const* argv[])
{
yy::parser p;
return p.parse ();
}
the generated header fails to compile cleanly (foo.cc just #includes
the generated header):
$ clang++-mp-9.0 -c -Wundefined-func-template foo.cc
In file included from foo.cc:1:
bar.tab.hh:550:12: warning: instantiation of function 'yy::parser::basic_symbol<yy::parser::by_type>::basic_symbol' required here, but no definition is available
[-Wundefined-func-template]
struct symbol_type : basic_symbol<by_type>
^
bar.tab.hh:436:7: note: forward declaration of template entity is here
basic_symbol (basic_symbol&& that);
^
bar.tab.hh:550:12: note: add an explicit instantiation declaration to suppress this warning if 'yy::parser::basic_symbol<yy::parser::by_type>::basic_symbol' is explicitly instantiated
in another translation unit
struct symbol_type : basic_symbol<by_type>
^
1 warning generated.
* data/skeletons/c++.m4 (b4_public_types_define): Move the
implementation of the basic_symbol move-ctor to...
(b4_public_types_define): here, its declaration.
* tests/headers.at (Sane headers): Use a declared token so that the
corresponding token constructor is declared. Which triggers the
aforementioned issue.
That's one nice benefit from using enums.
* data/skeletons/lalr1.java (YYSYMBOL_YYEMPTY): No longer define it.
Use 'null' instead.
* examples/java/calc/Calc.y, tests/local.at: Adjust.
* data/skeletons/bison.m4, data/skeletons/c++.m4, data/skeletons/c.m4,
* data/skeletons/glr.cc, data/skeletons/lalr1.cc,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java:
Refer to the "kind" of a symbol, not its "type", where appropriate.
* doc/bison.texi: Replace occurrences of "token type" with "token
kind".
Stop referring to the "macro definitions" of the token kinds, just
name them "definitions".
* maint:
maint: post-release administrivia
version 3.5.4
examples: reccalc: really compile cleanly in C99
news: announce that Bison 3.6 drops YYERROR_VERBOSE
news: update for 3.5.4
style: fix spellos
typo: succesful -> successful
package: improve the readme
java: check and fix support for api.token.raw
java: style: prefer 'int[] foo' to 'int foo[]'
build: fix syntax-check issues
tests: recheck: work properly when the test suite was interrupted
doc: c++: promote api.token.raw
build: fix compatibility with old compilers
examples: reccalc: compile cleanly in C99
The previous fix does not suffice, and actually managed to make things
worse by defining yyscan_t twice in parse.y...
Reported by kencu.
https://trac.macports.org/ticket/59927#comment:29
* examples/c/reccalc/parse.y (yyscan_t): Define it with the same
guards as used by Flex.
The first name is too long. We already have `yypstate`, so
`yypcontext` is ok. We are also migrating to using `*_t` for our
types.
* NEWS, data/skeletons/glr.c, data/skeletons/yacc.c, doc/bison.texi,
* examples/c/bistromathic/parse.y, src/parse-gram.y, tests/local.at:
(yyparse_context_t, yyparse_context_location, yyparse_context_token):
Rename as...
(yypcontext_t, yypcontext_location, yypcontext_token): these.
The Java enums are very different from the C model. As a consequence,
one cannot "build" an enum directly from an integer, we must retrieve
it. That's the purpose of the SymbolType.get class method.
* data/skeletons/java.m4 (b4_symbol_enum, b4_case_code_symbol)
(b4_declare_symbol_enum): New.
* data/skeletons/lalr1.java: Use SymbolType,
SymbolType.YYSYMBOL_YYEMPTY, etc.
* examples/java/calc/Calc.y, tests/local.at: Adjust.
* tests/local.at (AT_LANG_MATCH, AT_YYERROR_DECLARE(java))
(AT_YYERROR_DECLARE_EXTERN(java), AT_PARSER_CLASS): New.
(AT_MAIN_DEFINE(java)): Use AT_PARSER_CLASS.
* tests/scanner.at: Add a test for Java.
* data/skeletons/lalr1.java (yytranslate_): Cast the result.
* data/skeletons/d.m4 (b4_symbol_enum, b4_declare_symbol_enum): New.
* data/skeletons/lalr1.d: Use them.
Use SymbolType, SymbolType.YYSYMBOL_YYEMPTY etc. where appropriate.
(undef_token_, token_number_type, yy_error_token_): Remove.
* tests/local.mk (recheck): Look at the per-test logs, not the overall
log, which, when interrupted, contains only information about... the
tests that passed.
GCC 4.2 dies with
src/InadequacyList.c: In function 'InadequacyList__new_conflict':
src/InadequacyList.c:37: error: #pragma GCC diagnostic not allowed inside functions
src/InadequacyList.c:37: error: #pragma GCC diagnostic not allowed inside functions
src/InadequacyList.c:40: error: #pragma GCC diagnostic not allowed inside functions
Reported by Evan Lavelle.
See https://lists.gnu.org/r/bug-bison/2020-03/msg00021.html
and https://trac.macports.org/ticket/59927.
* src/system.h (GCC_VERSION): New.
Use it to control IGNORE_TYPE_LIMITS_BEGIN and
IGNORE_TYPE_LIMITS_END.
Because of the insane current implementation of glr.cc, things are a
bit nasty. We will rename symbol_number_type as symbol_type_type
later, to keep this commit small.
* data/skeletons/c++.m4 (b4_declare_symbol_enum): New.
Also define YYNTOKENS to avoid type clashes when yyntokens_ was
actually defined in another enum.
Use it.
(symbol_number_type): Be an alias of symbol_type_type.
Use YYSYMBOL_YYEMPTY and the like.
Use symbol_number_type where appropriate.
(empty_symbol): Remove.
(yytranslate_): Use symbol_number_type, not token_number_type.
* data/skeletons/lalr1.cc: Use symbol_number_type where appropriate.
Adjust to the replacement of empty_symbol by YYSYMBOL_YYEMPTY.
(yy_error_token_, yy_undef_token_, yyeof_, yyntokens_): Remove.
Adjust dependencies.
* data/skeletons/glr.cc: Use symbol_number_type where appropriate.
Forward definitions of YYSYMBOL_YYEMPTY, etc. to glr.c.
* tests/headers.at: Accept YYNTOKENS and other YYSYMBOL_*.
* tests/local.at (AT_YYERROR_DEFINE(c++)): Use symbol_number_type.
Now that yacc.c and glr.c both know yysymbol_type_t, convert the
common routines.
* data/skeletons/c.m4 (yydestruct, yy_symbol_value_print)
(yy_symbol_print): Use yysymbol_type_t instead of int.
* data/skeletons/glr.c: Use yySymbol where appropriate.
* data/skeletons/yacc.c (YY_ACCESSING_SYMBOL): New wrapper around
yystos.
Use it.
* tests/local.at (yyreport_syntax_error): Use yysymbol_type_t where
appropriate.
Apply the same changes as in yacc.c. Now yySymbol and yysymbol_type_t
are aliases. We will remove the former later, to avoid cluttering
this commit.
* data/skeletons/glr.c: Use b4_declare_symbol_enum.
Use YYSYMBOL_YYEOF etc. where appropriate.
(YYUNDEFTOK, YYTERROR): Remove.
(YYTRANSLATE, yySymbol, yyexpected_tokens, yysyntax_error_arguments):
Adjust.
(yy_accessing_symbol): New.
Use it where appropriate.
This triggers warnings with several compilers. For instance ICC fills
the logs with pages and pages of
input.c(477): error: a value of type "int" cannot be used to initialize an entity of type "const yysymbol_type_t={yysymbol_type_t}"
0, 2, 2, 2, 2, 2, 2, 2, 2, 2,
^
input.c(477): error: a value of type "int" cannot be used to initialize an entity of type "const yysymbol_type_t={yysymbol_type_t}"
0, 2, 2, 2, 2, 2, 2, 2, 2, 2,
^
And so does G++9 when compiling yacc.c's (C) output
input.c:545:8: error: invalid conversion from 'int' to 'yysymbol_type_t' [-fpermissive]
545 | 0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
| ^
| |
| int
input.c:545:15: error: invalid conversion from 'int' to 'yysymbol_type_t' [-fpermissive]
545 | 0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
| ^
| |
| int
Clang++ is no exception
input.c:545:8: error: cannot initialize an array element of type 'const yysymbol_type_t' with an rvalue of type 'int'
0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
^
input.c:545:15: error: cannot initialize an array element of type 'const yysymbol_type_t' with an rvalue of type 'int'
0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
^
At some point we could use yysymbol_type_t's enumerators to define
yytranslate. Meanwhile...
* data/skeletons/yacc.c (yytranslate): Use the original integral type
to define it.
(YYTRANSLATE): Cast the result into yysymbol_type_t.
Currently we define enumerators only for symbols that have an
identifier. That rules out tokens such as '+', and nonterminals such
as foo-bar and foo.bar. As a consequence we are taking chances: the
compiler might compile yysymbol_type_t as too small an integral type
for some symbol codes.
* data/skeletons/bison.m4 (b4_symbol_sid): Forge a unique symbol
identifier for symbols that don't have an ID.
This is not only cleaner, it also protects us from mixing signed
values (YYEMPTY is #defined as -2) with unsigned types (the
yysymbol_type_t enum is typically compiled as a small unsigned).
For instance GCC 9:
input.c: In function 'yyparse':
input.c:1107:7: error: conversion to 'unsigned int' from 'int'
may change the sign of the result
[-Werror=sign-conversion]
1107 | yyn += yytoken;
| ^~
input.c:1107:10: error: conversion to 'int' from 'unsigned int'
may change the sign of the result
[-Werror=sign-conversion]
1107 | yyn += yytoken;
| ^~~~~~~
input.c:1108:47: error: comparison of integer expressions of
different signedness:
'yytype_int8' {aka 'const signed char'} and
'yysymbol_type_t' {aka 'enum yysymbol_type_t'}
[-Werror=sign-compare]
1108 | if (yyn < 0 || YYLAST < yyn || yycheck[yyn] != yytoken)
| ^~
input.c:702:25: error: operand of ?: changes signedness from 'int'
to 'unsigned int' due to unsignedness of
other operand [-Werror=sign-compare]
702 | #define YYEMPTY (-2)
| ^~~~
input.c:1220:33: note: in expansion of macro 'YYEMPTY'
1220 | yytoken = yychar == YYEMPTY ? YYEMPTY : YYTRANSLATE (yychar);
| ^~~~~~~
input.c:1220:41: error: unsigned conversion from 'int' to
'unsigned int' changes value
from '-2' to '4294967294'
[-Werror=sign-conversion]
1220 | yytoken = yychar == YYEMPTY ? YYEMPTY : YYTRANSLATE (yychar);
| ^
Eventually, it might be interesting to move away from -2 (which is the
only possible negative symbol number) and use the next available
number, to save bits. We could actually even simply use "0" and shift
the rest, which would allow to write "!yytoken" to mean really
"yytoken != YYEMPTY".
* data/skeletons/c.m4 (b4_declare_symbol_enum): Define YYSYMBOL_YYEMPTY.
* data/skeletons/yacc.c: Use it.
* src/parse-gram.y (yyreport_syntax_error): Use YYSYMBOL_YYEMPTY, not
YYEMPTY, when dealing with a symbol.
* tests/regression.at: Adjust.
Now that we have a proper type for internal symbol numbers, let's use
it. More code needs conversion, e.g., printers and destructors, but
they are shared with glr.c, which is not ready yet for this change.
It will also help us deal with warnings such as (GCC9 on GNU/Linux):
input.c: In function 'int yyparse()':
input.c:475:37: error: enumeral and non-enumeral type in conditional expression [-Werror=extra]
475 | (0 <= (YYX) && (YYX) <= YYMAXUTOK ? yytranslate[YYX] : YYSYMBOL_YYUNDEF)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input.c:1024:17: note: in expansion of macro 'YYTRANSLATE'
1024 | yytoken = YYTRANSLATE (yychar);
| ^~~~~~~~~~~
* data/skeletons/yacc.c (yytranslate, yysymbol_name)
(yyparse_context_t, yyexpected_tokens, yypstate_expected_tokens)
(yysyntax_error_arguments):
Use yysymbol_type_t instead of int.
There's a number of advantage in exposing the symbol (internal)
numbers:
- custom error messages can use them to decide how to represent a
given symbol, or a set of symbols.
- we need something similar in uses of yyexpected_tokens. For
instance, currently, bistromathic's completion() reads:
int ntokens = expected_tokens (line, tokens, YYNTOKENS);
[...]
for (int i = 0; i < ntokens; ++i)
if (tokens[i] == YYTRANSLATE (TOK_VAR))
[...]
else if (tokens[i] == YYTRANSLATE (TOK_FUN))
[...]
else
[...]
- now that it's a compile-time expression, we can easily build static
tables, switch, etc.
- some users depended on the ability to get the token number from a
symbol to write test cases for their scanners. But Bison 3.5
removed the table this feature depended upon (a reverse
yytranslate). Now they can check against the actual symbol number,
without having pay (space and time) a conversion.
See https://lists.gnu.org/r/bug-bison/2020-01/msg00001.html, and
https://lists.gnu.org/archive/html/bug-bison/2020-03/msg00015.html.
- it helps us clearly separate the internal symbol numbers from the
external token numbers, whose difference is sometimes blurred in the
code when values coincide (e.g. "yychar = yytoken = YYEOF").
- it allows us to get rid of ugly macros with inconsistent names such
as YYUNDEFTOK and YYTERROR, and to group related definitions
together.
- similarly it provides a clean access to the $accept symbol (which
proves convenient in a current experimentation of mine with several
%start symbols).
Let's declare this type as a private type (in the *.c file, not
the *.h one). So it does not need to be influenced by the api prefix.
* data/skeletons/bison.m4 (b4_symbol_sid): New.
(b4_symbol): Use it.
* data/skeletons/c.m4 (b4_symbol_enum, b4_declare_symbol_enum): New.
* data/skeletons/yacc.c: Use b4_declare_symbol_enum.
(YYUNDEFTOK, YYTERROR): Remove.
Use the corresponding symbol enum instead.
* tests/local.mk (recheck): Look at the per-test logs, not the overall
log, which, when interrupted, contains only information about... the
tests that passed.
* data/skeletons/lalr1.java (yysyntaxErrorArguments): Move it from the
context, to the parser object.
Generate only for detailed and verbose error messages.
* tests/local.at (AT_YYERROR_DEFINE(java)): Use yyexpectedTokens
instead.
We could just "inline yysyntax_error_arguments back" in the routines
it was originally extracted from, but I think the code is nicer to
read this way.
* data/skeletons/glr.c (yysyntax_error_arguments): Generate only for
detailed and verbose error messages.
* data/skeletons/yacc.c: Likewise.
* data/skeletons/lalr1.cc (parser::context::yysyntax_error_arguments):
Move as...
(parser::yysyntax_error_arguments_): this.
And only for detailed and verbose error messages.
Because glr.c shares the same testing routines, we also need to
convert it.
* data/skeletons/glr.c (yyparse_context_token): New.
* tests/local.at (yyreport_syntax_error): here.
Suggested by Adrian Vogelsgesang.
https://lists.gnu.org/archive/html/bison-patches/2020-02/msg00069.html
* data/skeletons/lalr1.java (Context.EMPTY, Context.getToken): New.
(Context.yyntokens): Rename as...
(Context.NTOKENS): this.
Because (i) all the Java coding styles recommend upper case for
constants, and (ii) the Java Skeleton exposes Lexer.EOF, not
Lexer.YYEOF.
* data/skeletons/yacc.c (yyparse_context_token): New.
* examples/c/bistromathic/parse.y (yyreport_syntax_error): Don't use
yysyntax_error_arguments.
* examples/java/calc/Calc.y (yyreportSyntaxError): Likewise.
yyparse returns 0, 1, 2 since ages (accept, reject, memory exhausted).
Some of our auxiliary functions such as yy_lac and
yyreport_syntax_error also need to return error codes and also use 0,
1, 2. Because it uses yy_lac, yyexpected_tokens also needs to return
"problem", "memory exhausted", but in case of success, it needs to
return the number of tokens, so it cannot use 1 and 2 as error code.
Currently it uses -1 and -2, which is later converted into 1 and 2 as
yacc.c expects it.
Let's simplify this and use consistently -1 and -2 for auxiliary
functions that are not exposed (or not yet exposed) to the user. In
particular this will save the user from having to convert
yyexpected_tokens's -2 into yyreport_syntax_error's 2: both return -1
or -2.
* data/skeletons/yacc.c (yy_lac, yyreport_syntax_error)
(yy_lac_stack_realloc): Return -1, -2 for errors instead of 1, 2.
Adjust callers.
* examples/c/bistromathic/parse.y (yyreport_syntax_error): Do take
error codes into account.
Issue a syntax error message even if we ran out of memory.
* src/parse-gram.y, tests/local.at (yyreport_syntax_error): Adjust.
The cost of the file layer is large and makes benchmarks too coarse,
as seen for in following example, first with a file, then with a
literal string:
0. %skeleton "yacc.c" %define parse.lac full
1. %skeleton "yacc-v1.c" %define nofinal %define parse.lac full
2. %skeleton "yacc-v2.c" %define nofinal %define parse.lac full
3. %skeleton "yacc-v3.c" %define nofinal %define parse.lac full
4. %skeleton "yacc.c"
5. %skeleton "yacc-v1.c" %define nofinal
6. %skeleton "yacc-v2.c" %define nofinal
7. %skeleton "yacc-v3.c" %define nofinal
--------------------------------------------------
Benchmark Time CPU Iterations
--------------------------------------------------
BM_y0 32558 ns 32537 ns 21228
BM_y1 32400 ns 32369 ns 21233
BM_y2 33485 ns 33464 ns 20625
BM_y3 32139 ns 32125 ns 21446
BM_y4 31343 ns 31329 ns 21747
BM_y5 31344 ns 31317 ns 22035
BM_y6 31287 ns 31255 ns 22039
BM_y7 31387 ns 31373 ns 22178
--------------------------------------------------
Benchmark Time CPU Iterations
--------------------------------------------------
BM_y0 10642 ns 10634 ns 63601
BM_y1 10657 ns 10654 ns 63625
BM_y2 10441 ns 10432 ns 65957
BM_y3 10558 ns 10554 ns 64546
BM_y4 9521 ns 9516 ns 72011
BM_y5 9179 ns 9157 ns 75028
BM_y6 9360 ns 9356 ns 73770
BM_y7 9365 ns 9359 ns 72609
Of course, at the same time it is less realistic: most users read
files rather that strings, so it might lead to us to pay attention to
costs most people don't see.
* etc/bench.pl.in (&calc_input): Output into a file given as argument.
Output in C syntax.
(&generate_grammar_calc): Use it.
Simplify the grammar: remove operators we don't care about.
Rewrite the scanner to work on a char* instead of a FILE*.
* etc/bench.pl.in (&bench_with_timethese): Also use y$i, as in
&bench_with_gbenchmark.
(&generate_grammar_calc): Don't add a prefix, let the callers do it.
* etc/bench.pl.in (&compiler): New, extracted from...
(&compile): here.
Don't link when using gbm.
(&calc_input): Don't make massive input for micro
benchmarks.
(&generate_grammar_calc): When using gbm, use api.prefix to avoid name
collisions.
Be ready to issue BENCHMARKS instead of a main.
(&bench): Rename as...
(&bench_with_timethese): this.
(&bench_with_gbenchmark): New.
(&bench): New.
Dispatch on these two.
In push parsers, when asking for the list of expected tokens at some
point, it makes no sense to build a yyparse_context_t: the yypstate
alone suffices (the only difference being the lookahead). Instead of
forcing the user to build a useless shell around yypstate, let's offer
yypstate_expected_tokens.
See https://lists.gnu.org/r/bison-patches/2020-03/msg00025.html.
* data/skeletons/yacc.c (yypstate): Declare earlier, so that we can
use it for...
(yypstate_expected_tokens): this new function, when in push parsers.
Adjust dependencies.
* examples/c/bistromathic/parse.y: Simplify: use
yypstate_expected_tokens.
Style fixes.
Reduce scopes (reported by Joel E. Denny).
* upstream/maint:
maint: post-release administrivia
version 3.5.3
news: update for 3.5.3
yacc.c: make sure we properly propagated the user's number for error
diagnostics: don't crash because of repeated definitions of error
style: initialize some struct members
diagnostics: beware of zero-width characters
diagnostics: be sure to close the styling when lines are too short
muscles: fix incorrect decoding of $
code: be robust to reference with invalid tags
build: fix typo
doc: update recommandation for libtextstyle
style: comment changes
examples: use consistently the GFDL header for readmes
style: remove useless declarations
typo: succesful -> successful
README: point to tests/bison, and document --trace
gnulib: update
maint: post-release administrivia
According to https://www.unix.com/man-page/POSIX/1posix/yacc/, the
user is allowed to specify her user number for the error token:
The token error shall be reserved for error handling. The name
error can be used in grammar rules. It indicates places where the
parser can recover from a syntax error. The default value of error
shall be 256. Its value can be changed using a %token
declaration. The lexical analyzer should not return the value of
error.
I think this feature is useless, the user should not have to deal with
that. The intend is probably to give the user a means to use 256 if
she wants to, but provided "error" cleared the path first by being
assigned another number. In the case of Bison, 256 is assigned to
"error" at the end if the user did not use it for a token of hers. So
this feature is useless.
Yet it is valid, and if the user assigns twice a token number to
"error", then the second time we want to complain about it and want to
show the original definition. At this point, we try to display the
built-in definition of "error", whose location is NULL, and we crash.
Rather, the location of the first user definition of "error" should
become its defining location.
Reported byg Ahcheong Lee.
https://lists.gnu.org/r/bug-bison/2020-03/msg00007.html
* src/symtab.c (symbol_class_set): If this is a declaration and the
symbol was not declared yet, keep this as defining location.
* tests/input.at (Redefining the error token): New.
Currenly we rely on (visual) width of the characters to decide where
to open and close the styling of the quoted lines. This breaks when
we deal with zero-width characters: we cannot just rely on (visual)
columns, we need to know whether we are before, inside, or after the
highlighted portion.
* src/location.c (location_caret): col_end: no longer add 1, "regular"
characters have a width of 1, only 0-width characters have 0-width.
opened: replace with 'state', a three-valued enum.
Don't reopen the style if we already did.
* tests/diagnostics.at (Zero-width characters): New.
bar.y:4.12-17: <error>error:</error> redefining user token number of foo
- 4 | %token foo <error>123
+ 4 | %token foo <error>123</error>
| <error>^~~~~~</error>
* src/location.c (location_caret): Be sure to close.
* tests/diagnostics.at (Line is too short, and then you die): New.
Because we want to support $<a->b>$, we must accept -> in type tags,
and reject $<->$, as it is unfinished.
Reported by Ahcheong Lee.
* src/scan-code.l (yylex): Make sure "tag" does not end with -, since
-> does not close the tag.
* tests/input.at (Stray $ or @): Check this.
Currently pstate_new does not set up its variables, this task is left
to yypush_parse. This was probably to share more code with usual pull
parsers, where these (local) variables are indeed initialized by
yyparse.
But as a consequence yyexpected_tokens crashes at the very beginning
of the parse, since, for instance, the stacks are not even set up.
See https://lists.gnu.org/r/bison-patches/2020-03/msg00001.html.
The fix could have very simple, but the documentation actually makes
it very clear that we can reuse a pstate for several parses:
After yypush_parse returns a status other than YYPUSH_MORE, the
parser instance yyps may be reused for a new parse.
so we need to restore the parser to its pristine state so that (i) it
is ready to run the next parse, (ii) it properly supports
yyexpected_tokens for the next run.
* data/skeletons/yacc.c (b4_initialize_parser_state_variables): New,
extracted from the top of yyparse/yypush_parse.
(yypstate_clear): New.
(yypstate_new): Use it when push parsers are enabled.
Define after the yyps macros so that we can use the same code as the
regular pull parsers.
(yyparse): Use it when push parsers are _not_ enabled.
* examples/c/bistromathic/bistromathic.test: Check the completion on
the beginning of the line.
Currently completion on "at" proposes only "atan", but does not
actually complete "at" into "atan".
* examples/c/bistromathic/parse.y (completion): Install the lcp in
matches[0].
* examples/c/bistromathic/bistromathic.test: Check that case.
Currently "(1+<TAB>" does not work as expected, because "+" is not a
word breaking character.
* examples/c/bistromathic/parse.y (init_readline): Specify our word
breaking characters.
* examples/c/bistromathic/bistromathic.test: Avoid trailing spaces.
These macros have been extremely useful when we had to support K&R C,
which we dropped long ago. Now, they merely make the code uselessly
hard to read.
* data/skeletons/c.m4, data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/yacc.c:
Stop using b4_function_define.
Let's use GNU readline and its TAB autocompletion to demonstrate the
use of yyexpected_tokens.
This shows a number of weaknesses in our current approach:
- some macros (yyssp, etc.) from push parsers "leak" in user code, we
need to undefine them
- the context needed by yyexpected_tokens does not need the token,
yypstate actually suffices
- yypstate is not properly setup when first allocated, which results
in a crash of yyexpected_tokens if fired before a first token was
read. We should move initialization from yypush_parse into
yypstate_new.
* examples/c/bistromathic/parse.y (yylex): Take input as a string, not
a file.
(EXIT): New token.
(input): Adjust to work only on a line.
(line): Remove.
(symbol_count, process_line, expected_tokens, completion)
(init_readline): New.
* examples/c/bistromathic/bistromathic.test: Adjust expectations.
This example will soon use GNU readline, so its scanner should be easy
to use (concurrently) on strings, not streams. This is not a place
where Flex shines, and anyway, these are examples of Bison, not Flex.
There's already lexcalc and reccalc that demonstrate the use of Flex.
* examples/c/bistromathic/scan.l: Remove.
* examples/c/bistromathic/parse.y (yylex): New.
Adjust dependencies.
The bistromathic example should not use Flex, it makes it too complex.
But it was the only example to show location tracking with Flex.
* examples/c/lexcalc/lexcalc.test, examples/c/lexcalc/parse.y,
* examples/c/lexcalc/scan.l: Demonstrate location tracking as is done
in bistromathic.
The current implementation of parser::context keeps a copy of the
lookahead. This is troublesome since we support move-only types.
Besides, while GCC is happy with the current implementation, Clang
complains that the ctor it needs to build the copy of the lookahead is
not yet available.
461. calc.at:1120: testing Calculator C++ %defines %locations parse.error=verbose %name-prefix "calc" %verbose ...
calc.at:1120: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -Wno-deprecated -o calc.cc calc.y
calc.at:1120: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o calc calc.cc calc-lex.cc calc-main.cc $LIBS
stderr:
In file included from calc-lex.cc:7:
calc.hh:351:12: error: instantiation of function 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' required here, but no definition is available [-Werror,-Wundefined-func-template]
struct symbol_type : basic_symbol<by_type>
^
calc.hh:273:7: note: forward declaration of template entity is here
basic_symbol (const basic_symbol& that);
^
calc.hh:351:12: note: add an explicit instantiation declaration to suppress this warning if 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' is explicitly instantiated in another translation unit
struct symbol_type : basic_symbol<by_type>
^
1 error generated.
In file included from calc-main.cc:7:
calc.hh:351:12: error: instantiation of function 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' required here, but no definition is available [-Werror,-Wundefined-func-template]
struct symbol_type : basic_symbol<by_type>
^
calc.hh:273:7: note: forward declaration of template entity is here
basic_symbol (const basic_symbol& that);
^
calc.hh:351:12: note: add an explicit instantiation declaration to suppress this warning if 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' is explicitly instantiated in another translation unit
struct symbol_type : basic_symbol<by_type>
^
1 error generated.
stdout:
calc.at:1120: exit code was 1, expected 0
461. calc.at:1120: 461. Calculator C++ %defines %locations parse.error=verbose %name-prefix "calc" %verbose (calc.at:1120): FAILED (calc.at:1120)
* data/skeletons/lalr1.cc (context::yyla_): Make it a const-ref.
Move the implementation out of the declaration.
Address compiler warnings such as
warning: declaration of 'yyla' shadows a member of 'yy::parser::context' [-Wshadow]
* data/skeletons/lalr1.cc (context): Don't use the same names for
variables and members.
Use foo_ for private members, as in parser.
Also, use the + trick in array accesses to please ICC and provide it
with an int.
* data/skeletons/lalr1.cc: added support here
* tests/calc.at: added test cases
* tests/local.at: added yyreport_syntax_error implementation
for C++ test cases
Prefer b4_parse_error_case over the adhoc solution
`m4_case + b4_percent_define_get`. Same for b4_parse_error_bmatch.
* data/skeletons/glr.c: here
* data/skeletons/yacc.c: here
* etc/bench.pl.in: Adjust to the current use of %define's values.
Don't use %error-verbose.
Prefer Bison to CPP (e.g., api.value.type).
Avoid returning characters directly, so that %define api.token.raw works.
* doc/bison.texi (Parser Internationalization): Move most of its
content into...
(Enabling I18n): this new node.
(Token I18n): New.
(Token Decl): Refer to token internationalization.
(Error Reporting Function): Promote parse.error detailed.
Since Bison 2.7, output was indented four spaces for explanatory
statements. For example:
input.y:2.7-13: error: %type redeclaration for exp
input.y:1.7-11: previous declaration
Since the introduction of caret-diagnostics, it became less clear.
Remove the indentation and display submessages as in GCC:
input.y:2.7-13: error: %type redeclaration for exp
2 | %type <float> exp
| ^~~~~~~
input.y:1.7-11: note: previous declaration
1 | %type <int> exp
| ^~~~~
* src/complain.h (SUB_INDENT): Remove.
(warnings): Add "note" to the enum.
* src/complain.h, src/complain.c (complain_indent): Replace by...
(subcomplain): this.
Adjust all dependencies.
* tests/actions.at, tests/diagnostics.at, tests/glr-regression.at,
* tests/input.at, tests/named-refs.at, tests/regression.at:
Adjust expectations.
bison 3.5.2
* tag 'v3.5.2':
version 3.5.2
news: 3.5.2
gnulib: update
doc: update Doxygen template
java: avoid trailing white spaces
m4: fix b4_token_format
doc: clearly state that %yacc only makes sense with yacc.c
doc: spell check
examples: be more robust to spaces in paths
larlr1.cc: Reject unsupported values for parse.lac
We used to emit:
/** Token number,to be returned by the scanner. */
static final int NUM = 258;
/** Token number,to be returned by the scanner. */
static final int NEG = 259;
with no space after the comma. Fix that.
* data/skeletons/bison.m4 (b4_token_format): Quote where appropriate.
This revealed a number of things I had not realized:
- the Java location tracking was aliasing the same pair of positions
for all the symbols (see previous commit).
- in impure parsers, it's quite easy to use incorrect locations for
diagnostics, since yyerror uses yylloc, which is the location of the
lookahead, not that of the current lhs. So we need something like
{
YYLTYPE old_yylloc = yylloc;
yylloc = @$;
yyerror (]AT_PARAM_IF([result, count, nerrs, ])[buf);
yylloc = old_yylloc;
}
Maybe we should do that little yylloc dance in the skeleton instead
of leaving it to the user? It might be costly... But that's only
for users of the impure parsers, which are asking for trouble
anyway.
- in glr.cc invoking yyerror is somewhat cumbersome: the C++ interface
is not available as we are in yyparse (which in C), and yyerror is
used by glr.cc itself to bind it to the user's parser::error. If we
call yyerror, we need:
yyerror (]AT_LOCATION_IF([[&@$, ]])[yyparser, ]AT_PARAM_IF([result, count, nerrs, ])[msg);
However calling yy::parser::error is easier, once we know that the
current parser object is available as 'yyparser'. Which also saves
us from having to pass the parse-params ourselves:
yyparser.error (]AT_LOCATION_IF([[@$, ]])[msg);
* tests/calc.at: Invoke yyerror by hand, instead of using fprintf etc.
Adjust expectations.
* data/skeletons/lalr1.java (Context): Make data members private.
(Context.getLocation): New.
* examples/java/calc/Calc.y, tests/java.at, tests/local.at: Adjust.
* doc/bison.texi (Tokens from Literals): Move to code using
%token-table to...
(Decl Summary: %token-table): here.
* data/skeletons/bison.m4: Implement mutual exclusion.
* tests/input.at: Check it.
* doc/local.mk: Be robust to the removal of doc/.
With texinfo.tex 2019-09-24.13, node names with + are not properly
handled.
https://lists.gnu.org/r/bug-texinfo/2020-02/msg00004.html
* doc/bison.texi: Always use the three-argument form for references to
node with a + in the name.
This reverts commit ebab1ffca8.
This commit removed "useless" initializers, going from
/* YYPACT[STATE-NUM] -- Index in YYTABLE of the portion describing
STATE-NUM. */
private static final byte yypact_[] = yypact_init ();
private static final byte[] yypact_init ()
{
return new byte[]
{
25, -7, -8, 37, -8, 40, -8, 20, -8, 61,
-8, -8, 3, 9, 51, -8, -8, -2, -2, -2,
-2, -2, -2, -8, -8, -8, 1, 66, 66, 3,
3, 3
};
}
to
/* YYPACT[STATE-NUM] -- Index in YYTABLE of the portion describing
STATE-NUM. */
private static final byte[] yypact_ =
{
25, -7, -8, 37, -8, 40, -8, 20, -8, 61,
-8, -8, 3, 9, 51, -8, -8, -2, -2, -2,
-2, -2, -2, -8, -8, -8, 1, 66, 66, 3,
3, 3
};
But it turns out that this was on purpose, to work around the 64KB
limitation in JVM methods. It was introduced on the 2008-11-10 by
Di-an Jan in 09ccae9b18: "Work around
Java's ``code too large'' problem for parser tables". See
https://lists.gnu.org/r/help-bison/2008-11/msg00004.html. A real
test, where we would hit the JVM limitation, would be nice.
To avoid further regressions, add comments.
parse.error has more than two possible values.
* data/skeletons/bison.m4 (b4_error_verbose_if, b4_error_verbose_flag):
Remove.
(b4_parse_error_case, b4_parse_error_bmatch): New.
Adjust dependencies.
* data/skeletons/lalr1.java (yyexpectedTokens)
(yysyntaxErrorArguments): Make them methods of Context.
(Context.yysymbolName): New.
* tests/local.at: Adjust.
The error format should be translated, but contrary to the case of
C/C++, we cannot just depend on macros to adapt on the
presence/absence of '_'. Let's consider that the message format is to
be translated iff there are some internationalized tokens.
* src/output.c (prepare_symbol_names): Define b4_has_translations.
* data/skeletons/java.m4 (b4_trans): New.
* data/skeletons/lalr1.java: Use it to emit translatable or not the
format string.
And allow syntax error messages for 'detailed' and 'verbose' to be
translated.
* data/skeletons/lalr1.java (Context, yyexpectedTokens)
(yysyntaxErrorArguments): New.
(yysyntax_error): Use it.
In Java there is no need for N_ and yytranslate_. So instead of
hard-coding the use of N_ in the table of the symbol names, rely on
b4_symbol_translate.
* src/output.c (prepare_symbol_names): Use b4_symbol_translate instead
of N_.
* data/skeletons/c.m4 (b4_symbol_translate): New.
* data/skeletons/lalr1.java (yysymbolName): New.
Use it.
* examples/java/calc/Calc.y: Use parse.error=detailed.
* tests/calc.at: Check parse.error=detailed.
We used to emit:
/** Token number,to be returned by the scanner. */
static final int NUM = 258;
/** Token number,to be returned by the scanner. */
static final int NEG = 259;
with no space after the comma. Fix that.
* data/skeletons/bison.m4 (b4_token_format): Quote where appropriate.
Unfortunately in the Java skeleton the user cannot override the way
locations are displayed, and locations don't know the structure of the
positions. So they cannot implement the tricks used in the C/C++
skeletons to display "1.1" instead of "1.1-1.2".
* tests/local.at (Java): Add support for column tracking in the
locations, as we did in examples/java/calc.
* tests/calc.at: Use AT_CALC_YYLEX.
Soon calculator tests for Java will move from java.at to calc.at.
Which implies improving the Java testing infrastructure in
local.at (for instance really tracking columns in positions, not just
token number). Detach java.at from local.at.
* tests/java.at (AT_JAVA_POSITION_DEFINE_OLD): New.
Use it.
* examples/java/calc/Calc.y: The StreamTokenizer cannot "peek" for the
next character, it reads it, and keeps it for the next call. So the
current location is one passed the end of the current token. To avoid
this, keep the previous position, and use it to end the current token.
* examples/java/calc/Calc.test: Adjust.
The C, C++ and D skeletons used to show the stack right after popping
the stack during the reduction. Now that the stack is printed after
reaching a new state, that has become useless:
Entering state 1
Stack now 0 1
Reducing stack by rule 5 (line 83):
$1 = token "number" (1)
-> $$ = nterm exp (1)
Stack now 0
Entering state 8
Stack now 0 8
Remove the "Stack now 0" line.
* data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java, data/skeletons/yacc.c:
Here.
Currently, if we have long rules and series of shift, we stack states
without showing stack. Let's be more incremental, and do how the Java
skeleton does.
* data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/yacc.c:
Here.
Adjust test cases.
* tests/torture.at (AT_DATA_STACK_TORTURE): Disable stack traces: this
test produces a very large stack, and showing the stack each time we
shift a token goes quadatric.
The Java skeleton displays
Reading a token:
Next token is token "number" (1)
while the other display
Reading a token: Next token is token "number" (1)
When generating logs in the scanner, the first part is separated from
the second, and the end of the scanner logs have the second part
pasted in. So let's propagate the Java way, but with the colon.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java, data/skeletons/yacc.c: Do it.
Adjust test cases and doc.
* tests/local.at (AT_LANG_MATCH): New.
(AT_YYERROR_DECLARE(java), AT_YYERROR_DECLARE_EXTERN(java)): New.
* tests/calc.at: The grammar file for Java is quite different for the
others, and continuing to assemble it from pieces makes the grammar
file hard to understand. Let's also dispatch on the language to
assemble it, and isolate Java from the others.
Most of this comes from java.at.
This example, so far, was tracking the current token number, not the
current column number. This is not nice for an example...
* examples/java/Calc.y (PositionReader): New.
Use it.
* examples/java/Calc.test: Check the output.
AFAICT, autoboxing/unboxing was added in Java 5 (September 30, 2004).
I think we can afford to use it. It should help us merge some Java
tests with the main ones.
However, beware that != does not unbox: it compares the object
addresses.
* examples/java/Calc.y, tests/java.at: Simplify.
* examples/java/Calc.test, tests/java.at: Improve tests.
When building the test cases, emitting code in the epilogue is very
constraining. Let's make it simpler thanks to %code epilogue.
However, I don't want to document this: it is bad style to use it (we
should avoid having too many ways to write the same thing,
TI!MTOWTDI), just put your code in the true epilogue section.
* data/skeletons/glr.c, data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/yacc.c: Implement support for %code epilogue.
Remove useless comments.
* tests/calc.at, tests/java.at: Simplify.
Modeled after what was done in yacc.c, yet somewhat different since
yyGLRStack play the role of the full yyparse_context_t. It will
probably be defined as a alias, to make interfaces compatible.
LAC is not supported here. And is somewhat tricky.
* data/skeletons/glr.c (yyexpected_tokens, yysyntax_error_arguments): New.
* data/skeletons/glr.c: Move the handling of error messages after the
definition of types. Especially after yyGLRStack, which we will need
in forthcoming patches.
Simplify: yyGLRStack is defined at this point.
The PDF output is more consistent: some nodes were not pointed to with
their title. The HTML output becomes "see section Foo" instead of
"see Foo", but this should be addressed in the next Texinfo release.
Info output is simplified, as it uses only the node name and not its
title. But it's considered easier to read this way.
See https://lists.gnu.org/r/help-texinfo/2020-01/msg00031.html.
* doc/bison.texi: Set @xrefautomaticsectiontitle on.
Simplify all uses of ref.
Add an example to demonstrate the use of push parser. I'm pleasantly
surprised: parse.error=detailed works like a charm with push parsers.
* examples/c/local.mk, examples/c/pushcalc/Makefile
* examples/c/pushcalc/README.md, examples/c/pushcalc/calc.test,
* examples/c/pushcalc/calc.y, examples/c/pushcalc/local.mk:
New.
* examples/c/calc/calc.y: Restore to its original state, with
parse.error=detailed instead of parse.error=custom (this example
should be simple).
* examples/c/calc/calc.test: Check syntax errors.
* examples/c/lexcalc/parse.y: Add comments.
We used to display the unexpected token first:
$ bison foo.y
foo.y:1.8-13: error: syntax error, unexpected %token, expecting character literal or identifier or <tag>
1 | %token %token
| ^~~~~~
GCC uses a different format:
$ gcc-mp-9 foo.c
foo.c:1:5: error: expected identifier or '(' before ')' token
1 | int()()()
| ^
and so does Clang:
$ clang-mp-9.0 foo.c
foo.c:1:5: error: expected identifier or '('
int()()()
^
1 error generated.
They display the unexpected token last (or not at all). Also, they
don't waste width with "syntax error". Let's try that. It gives, for
the same example as above:
$ bison foo.y
foo.y:1.8-13: error: expected character literal or identifier or <tag> before %token
1 | %token %token
| ^~~~~~
* src/complain.h, src/complain.c (syntax_error): New.
* src/parse-gram.y (yyreport_syntax_error): Use it.
As a test case, support translations in Bison itself.
* src/parse-gram.y: Mark the translatable tokens.
While at it, use clearer names.
* tests/input.at: Adjust expectations.
Don't force callers of location_caret to have to deal with flags that
disable it.
* src/location.h, src/location.c (location_caret)
(location_caret_suggestion): Early return if disabled.
* src/complain.c: Simplify.
Just as the yacc.c skeleton, the lalr1.cc skeleton should reject
invalid values for parse.lac.
* data/skeletons/lalr1.cc: check validity of parse.lac
* tests/input.at: new test cases
Just as the yacc.c skeleton, the lalr1.cc skeleton should reject
invalid values for parse.lac.
* data/skeletons/lalr1.cc: check validity of parse.lac
* tests/input.at: new test cases
Some users would like to avoid having to "parse" the *.y file to find
the strings to translate. Let's issue the translatable tokens with N_
to allow "parsing" the generated parsers instead.
See
https://lists.gnu.org/archive/html/bison-patches/2019-01/msg00015.html
* src/output.c (prepare_symbol_names): Issue symbol_names with N_()
markup.
In addition to
%token NUM "number"
accept
%token NUM _("number")
in which case the token will be translated in error messages.
Do not use _() in the output if there are no translatable tokens.
* src/symtab.h, src/symtab.c (symbol): Add a 'translatable' member.
* src/parse-gram.y (TSTRING): New token.
(string_as_id.opt): Replace with...
(alias): this.
Use it.
* src/scan-gram.l (SC_ESCAPED_TSTRING): New start conditions, to match
TSTRINGs.
* src/output.c (prepare_symbols): Define b4_translatable if there are
translatable strings.
* data/skeletons/glr.c, data/skeletons/lalr1.cc,
* data/skeletons/yacc.c (yytnamerr): Receive b4_translatable, and use it.
* src/output.c (escape_trigraphs, xescape_trigraphs): New.
(prepare_symbol_names): Use it.
* tests/regression.at: Check the handling of trigraphs with
parse.error = detailed.
"detailed" error messages are almost like "verbose", except that we
don't double escape them, they don't get inner quotes, we don't use
yytnamerr, and we hide the table.
"custom" is exposed with the "detailed" tokens, not the "verbose"
ones: they are not double-quoted.
Because there's a risk that some people use yytname even without
"verbose", let's keep yytname (instead of yys_name) in "simple"
parse.error.
* src/output.c (prepare_symbol_names): Be ready to output symbol names
unquoted.
(prepare_symbol_names): Output both the old tname table, and the new
symbol_names one.
* data/skeletons/bison.m4: Accept 'detailed'.
* data/skeletons/yacc.c: When parse.error is 'detailed', don't emit
yytname and yytnamerr, just yysymbol_name with the table inside.
* tests/calc.at: Adjust.
Only parse.error verbose and simple will get the original yytname: the
other options will rely on a different table. So let's move on top of
the yysymbol_name function.
* data/skeletons/c.m4 (yy_symbol_print): Use yysymbol_name.
* data/skeletons/glr.c (yytokenName): Rename as...
(yysymbol_name): this.
The change of naming scheme is unfortunate, but it's definitely glr.c
which is "wrong".
Currently yy_symbol_print is defined before yytokenName, although it
should use it instead of read yytname directly. Move blocks around to
avoid this.
* data/skeletons/glr.c (yy_symbol_print): Move its definition after
that of yytokenName.
See https://github.com/nemequ/icc-travis/issues/15.
Thanks to Jeff Hammond and Evan Nemerson for their help.
* configure.ac (warn_common): Disable dubious warnings.
* .travis.yml: Use ICC again.
Be robust to newer versions of Autoconf where the package URL defaults
to https instead of http.
* configure.ac (AC_INIT): Use https.
* tests/report.at: Adjust expected output s/http/https/
to match updated URL.
Currently we get warnings with GCC 4.8 when running the
maintainer-check-g++ tests:
143. skeletons.at:85: testing Installed skeleton file names ...
../../tests/skeletons.at:120: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret --skeleton=yacc.c -o input-cmd-line.c input-cmd-line.y
../../tests/skeletons.at:121: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o input-cmd-line input-cmd-line.c $LIBS
stderr:
input-cmd-line.c: In function 'int yysyntax_error(long int*, char**, const yyparse_context_t*)':
input-cmd-line.c:977:52: error: conversion to 'int' from 'long int' may alter its value [-Werror=conversion]
YYSIZEOF (yyarg) / YYSIZEOF (*yyarg));
^
cc1plus: all warnings being treated as errors
stdout:
../../tests/skeletons.at:121: exit code was 1, expected 0
and
429. calc.at:823: testing Calculator parse.error=custom %locations api.prefix={calc} ...
../../tests/calc.at:823: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -Wno-deprecated -o calc.c calc.y
../../tests/calc.at:823: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o calc calc.c $LIBS
stderr:
calc.y: In function 'int yyreport_syntax_error(const yyparse_context_t*)':
calc.y:157:58: error: conversion to 'int' from 'long unsigned int' may alter its value [-Werror=conversion]
int n = yysyntax_error_arguments (ctx, arg, sizeof arg / sizeof *arg);
^
cc1plus: all warnings being treated as errors
stdout:
../../tests/calc.at:823: exit code was 1, expected 0
We could use a cast to avoid the warning, but it becomes too
cluttered. We can also use YYPTRDIFF_T, but that forces the user to
use YYPTRDIFF_T too, although this is an array of tokens, which is
limited by YYNTOKENS, an int. So let's completely avoid this warning.
* data/skeletons/yacc.c, tests/local.at (yyreport_syntax_error): Avoid
relying on sizeof to compute the array capacity.
Enhance the calculator tests: show that passing arguments to yyerror
works.
* tests/calc.at: Add a new parse-param, nerrs, which counts the number
of syntax errors in a run.
* tests/local.at: Adjust to handle the new 'nerrs' argument, when
present.
The custom error reporting function show sees the user's additional
arguments. Let's experiment with passing them as arguments to
yyreport_syntax_error, but maybe storing them in the context would be
a bettter alternative.
* data/skeletons/yacc.c (yyreport_syntax_error): Handle the
parse-params.
* tests/calc.at, tests/local.at: Adjust.
Currently the parse-params are tested in calc.at by checking that the
global variable and the parse-params have the same value. But it does
not check that value, that could remain being 0 just as well.
* tests/calc.at: Don't define the params when they are not used.
Check the final value of result and count.
Also, do count the number of line of logs.
Provide users with a means to query for the currently allowed tokens.
Could be used for autocompletion for instance.
* data/skeletons/yacc.c (yyexpected_tokens): New, extracted from
yysyntax_error_arguments.
* examples/c/calc/calc.y (PRINT_EXPECTED_TOKENS): New.
Use it.
We use a different format to check parse.error custom. Compute the
"verbose" one from it instead of forcing the test author to provide
the various formats of expected error messages.
* tests/calc.at (_AT_CHECK_CALC_ERROR): Handle this transformation
when needed.
Simplify callers.
* tests/local.at (AT_ERROR_CUSTOM_IF, AT_ERROR_VERBOSE_IF)
(AT_ERROR_SIMPLE_IF): New.
(AT_YYERROR_DEFINE(c)): Generate yyreport_syntax_error.
* tests/calc.at (_AT_CHECK_CALC_ERROR): Accept custom error messages
as additional test case.
Use it.
Add a new test case for %define parse.error custom.
When parse.error is custom, let users define a yyreport_syntax_error
function, and use it.
* data/skeletons/bison.m4 (b4_error_verbose_if): Accept 'custom'.
* data/skeletons/yacc.c: Implement it.
* examples/c/calc/calc.y: Experiment with it.
That allows users to cover more cases, such as easily filtering some
arguments they don't want to expose. But they now have to call
yysymbol_name explicitly.
* data/skeletons/yacc.c (yysyntax_error_arguments, yysyntax_error):
Deal with symbol numbers instead of symbol names.
Isolate a function that returns the list of expected and unexpected
tokens. It will be exposed to users willing to customize their error
messages.
* data/skeletons/yacc.c (yyparse_context_t): New.
(yyerror_message_arguments): New, extracted from yysyntax_error.
When not empty, AT_PARSE_PARAMS was guaranteed to end with a comma.
Remove the trailing comma, so that we can use AT_PARSE_PARAMS at the
end of the arguments, not only at the beginning.
* tests/local.at: here.
Unfortunately, m4_append relies on the macro not being defined whereas
we would have preferred it to check for emptiness. So use
m4_define/m4_undefine instead of m4_pushdef/m4_popdef.
Let's have C be the reference, and match it elsewhere. Maybe C is too
verbose and some adjustments are needed, but then that would be done
in another batch of patches.
* data/skeletons/lalr1.cc: Print the stack once we popped after
YYERROR, and before emptying the stack at the end of parsing.
Currently the C and C++ parse traces differ in the order in which the
stack is displayed: bottom up in C, top down in C++. Let's stick to
the C order.
* data/skeletons/stack.hh (stack::iterator, stack::const_iterator)
(begin, end): Be forward, not backward.
Provide the users with a public API to get the name of the tokens. A
thin wrapper around yytname.
* data/skeletons/yacc.c (yysymbol_name): New.
Use it.
I was hoping it would help us catch warnings when char is
unsigned (see 78bb152a63), but it does
not seem to help. It's a pity that the compiler is the same all over
the place, I would have preferred testing others.
* .travis.yml: here.
Supporting YYERROR_VERBOSE via cpp is a nuisance: m4 is in charge of
handling alternatives. When adding more options for %define
parse.error, supporting both CPP and M4 is too complex. Anyway,
YYERROR_VERBOSE was deprecated long ago.
* data/skeletons/yacc.c: Use m4 only to handle verbose/simple error
messages.
I would like to offer new ways to build the error message. As a first
step, let's simplify yysyntax_error whose first loop does two things
at the same time: (i) collect the tokens to be reported in the error
message, and (ii) accumulate their sizes and possibly return
"overflow". Let's pull (ii) in a second step.
Then test 525 (regression.at:1193: parse.error=verbose overflow)
failed. This test checks that we correctly report "memory overflow"
when the error message is too large. However the test is mistaken: it
is triggered in a place where there are five (large) expected tokens,
so anyway we would not display them, so there is no (memory) overflow
here! Transform this test to (i) check that indeed there is no
overflow, and (ii) create syntax_error3 which does check the intended
behavior, but with four expected tokens.
* data/skeletons/yacc.c (yysyntax_error): First compute the list of
arguments, then compute yysize.
* tests/regression.at (parse.error=verbose overflow): Enhance and fix.
GCC's -Wchar-subscripts may report issues on platforms where char is
unsigned. Unfortunately the current CI does not reproduce the
problem. But that would allow contributors to report issues if the
warning appears somewhere.
See 139d065594.
Problem reported by Andy Fiddaman in:
https://lists.gnu.org/r/bug-bison/2019-12/msg00021.html
* configure.ac (warn_common): Add -Wchar-subscripts.
Problem reported by Andy Fiddaman in:
https://lists.gnu.org/r/bug-bison/2019-12/msg00021.html
* data/skeletons/yacc.c (yy_reduce_print, yy_lac, yysyntax_error)
(yyreturn): If I might be a char, write a[+I] instead of a[I],
so that ‘gcc -Wchar-subscripts’ does not complain.
Another breakage revealed by vcsn.
* data/skeletons/c++.m4 (yytranslate_): Do not hard code "yy" and
"parser", both can be changed by the user.
Actually, since we are in the parser itself, there's really no need to
qualify the type.
* data/skeletons/glr.c (YYASSERT): Rename as...
(YY_ASSERT): this, for consistency with yacc.c, and also to emphasize
the fact that this is not for the end user (YY_ prefix).
* tests/glr-regression.at: Define parse.assert.
Now that we use small integral types, possibly unsigned (e.g.,
unsigned char), to store state numbers, using -1 to denote an empty
state (i.e., a state that stores no semantical value) is very
dangerous: it will be confused with state 255, which might be
non-empty.
Rather than allocating a larger range of state numbers to keep the
empty-state apart, let's use the number of a state known to store no
value. The initial state, numbered 0, seems to fit perfectly the job.
Reported by Frank Heckenbach.
https://lists.gnu.org/archive/html/bug-bison/2019-11/msg00016.html
* data/skeletons/lalr1.cc (empty_state): Be 0.
It is not used. And its implementation was wrong when api.token.raw
was defined, as it was still mapping to the external token numbers,
instead of the internal ones. Besides it was provided only when
api.token.constructor is defined, yet always declared.
* data/skeletons/c++.m4 (by_type::token): Remove, useless.
Reported by Frank Heckenbach.
https://lists.gnu.org/archive/html/bug-bison/2019-11/msg00016.html
The cast is needed when yytranslate_'s argument type is token_type,
i.e., when api.token.constructor is defined.
373. types.at:138: testing lalr1.cc api.value.type=variant api.token.constructor ...
======== Testing with C++ standard flags: ''
../../tests/types.at:138: bison --color=no -fno-caret -o test.cc test.y
../../tests/types.at:138: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o test test.cc $LIBS
stderr:
test.cc:966:16: error: result of comparison of constant 257 with
expression of type 'yy::parser::token_type'
(aka 'yy::parser::token::yytokentype') is always true
[-Werror,-Wtautological-constant-out-of-range-compare]
else if (t <= user_token_number_max_)
~ ^ ~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
It is because it is expected that when api.token.constructor is
defined, only symbol constructors will be used, that yytranslate_ then
takes a token_type. But it is wrong: we still allow literal
characters in this case, as demonstrated by test 373 for instance.
%define api.value.type variant
%define api.token.constructor
%token <std::pair<int, int>> '1' '2';
[...]
static yy::parser::symbol_type yylex ()
{
static char const input[] = "12";
int res = input[toknum++];
typedef yy::parser::symbol_type symbol;
if (res)
return symbol (res, std::make_pair (res - '0', res - '0' + 1));
else
return symbol (res);
}
So let yytranslate_ always take an int, which makes the cast truly
useless.
* data/skeletons/c++.m4, data/skeletons/lalr1.cc (yytranslate_): here.
The C++ implementation of LAC did not skip the $undefined token,
probably because it was not exposed. Expose it, and use clearer
names.
* data/skeletons/c++.m4: Don't define undef_token_ in yytranslate_,
but...
* data/skeletons/lalr1.cc (yy_undef_token_): here.
Use a more precise type to define yy_undef_token_ and yy_error_token_.
Unfortunately we move from a compile-time value defined via an enum to
a static const member. Eventually we should make it constexpr.
Make LAC implementation more alike yacc.c's one.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java: Don't expose
yyuser_token_number_max_ and yyundef_token_. Do as in C++: scope them
into yytranslate_, and only when api.token.raw is not defined.
(yyterror_): Rename as...
(yy_error_token_): this.
* data/skeletons/lalr1.d (token_number_type): New.
Use it.
Can't be done in the Java backend, as Java does not have type aliases.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java (yytoken_number_):
Remove, useless.
Was used in ancient C skeletons to support YYPRINT, long obsoleted by
%printer.
It is not used at all. We will remove it also from yacc.c, but
later (see TODO).
* data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java (yyerrcode_):
Remove.
Reported by Frank Heckenbach.
https://lists.gnu.org/archive/html/bug-bison/2019-11/msg00016.html
* data/skeletons/c++.m4 (b4_yytranslate_define): Don't use yyeof_ as
if it had two different types.
It is used once against the input argument, which is the value
returned by yylex, which is an "external token number", typically an
int. It is also used as output type, an "internal symbol number".
It turns out that in both cases we mean "0", but let's keep yyeof_
only for the case "internal symbol number", i.e., _after_ conversion
by yytranslate.
This frees us from one cast.
The current code for yysyntax_error for %define parse.error verbose is
fishy (given that YYEMPTY is -2, invalid argument for yytname[]):
static int
yysyntax_error ([...])
{
YYPTRDIFF_T yysize0 = yytnamerr (YY_NULLPTR, yytname[yytoken]);
[...]
if (yytoken != YYEMPTY)
A nearby comment reports
The only way there can be no lookahead present (in yychar) is if
this state is a consistent state with a default action. Thus,
detecting the absence of a lookahead is sufficient to determine
that there is no unexpected or expected token to report. In that
case, just report a simple "syntax error".
So it _is_ possible to call yysyntax_error with yytoken == YYEMPTY,
albeit quite difficult when meaning to, so virtually impossible by
accident (after all, there was never a bug report about this).
I failed to produce a test case, but Joel E. Denny provided me with
one (added to the test suite below). The yacc.c skeleton fails on
this, and once fixed dies on a second problem. The glr.c skeleton was
also dying, but immediately of this second problem.
Indeed we were not allocating space for the error message's final \0.
This was hidden by the fact that we only had error messages with at
least an unexpected token displayed, so with at least one "%s" in the
format string, whose size (2) was included (incorrectly) in the final
size of the message (where the %s have been replaced by the actual
content).
* data/skeletons/glr.c, data/skeletons/yacc.c (yysyntax_error):
Do not invoke yytnamerr on YYEMPTY.
Clarify the computation of the length of the _final_ error message,
with the NUL terminator but without the '%s's.
* tests/conflicts.at (Syntax error in consistent error state):
New, contributed by Joel E. Denny.
Having a file named "exception" is risky: the compiler might use that
file in #include.
Reported by 马俊 <[email protected]>.
* tests/local.at (AT_SKIP_IF_EXCEPTION_SUPPORT_IS_POOR): Generate
'exceptions', not 'exception'.
String literals as tokens serve two distinct purposes: freeing from
having to implement the keyword matching in the scanner, and improving
error messages. Most of the time both can be achieved at the same
time, but on occasions, it does not work so well.
We promote their use for error messages. We will also still support
the former case, but it is _not_ the recommended approach.
* doc/bison.texi (Tokens from Literals): Clearly state that we don't
recommend looking up the token types in the list of token names.
String literals, which allow for better error messages, are (too)
liberally accepted by Bison, which might result in silent errors. For
instance
%type <exVal> cond "condition"
does not define “condition” as a string alias to 'cond' (nonterminal
symbols do not have string aliases). It is rather equivalent to
%nterm <exVal> cond
%token <exVal> "condition"
i.e., it gives the type 'exVal' to the "condition" token, which was
clearly not the intention.
Introduce -Wdangling-alias to catch this.
* src/complain.h, src/complain.c: Add support for -Wdangling-alias.
(argmatch_warning_args): Sort.
* src/symtab.c (symbol_check_defined): Complain about dangling
aliases.
* doc/bison.texi: Document it.
* tests/input.at (Dangling aliases): New test.
On
%token TOKEN1
%type <ival> TOKEN1 TOKEN2 't'
%token TOKEN2
%%
expr:
bison -Wyacc gives
input.y:2.15-20: warning: POSIX yacc reserves %type to nonterminals [-Wyacc]
2 | %type <ival> TOKEN1 TOKEN2 't'
| ^~~~~~
input.y:2.29-31: warning: POSIX yacc reserves %type to nonterminals [-Wyacc]
2 | %type <ival> TOKEN1 TOKEN2 't'
| ^~~
input.y:2.22-27: warning: POSIX yacc reserves %type to nonterminals [-Wyacc]
2 | %type <ival> TOKEN1 TOKEN2 't'
| ^~~~~~
The messages appear to be out of order, but they are emitted when the
error is found.
* src/symtab.h (symbol_class): Add pct_type_sym, used to denote
symbols appearing in %type.
* src/symtab.c (complain_pct_type_on_token): New.
(symbol_class_set): Check that %type is not applied to tokens.
(symbol_check_defined): pct_type_sym also means undefined.
* src/parse-gram.y (symbol_decl.1): Set the class to pct_type_sym.
* src/reader.c (grammar_current_rule_begin): pct_type_sym also means
undefined.
* tests/input.at (Yacc's %type): New.
As an extension to POSIX Yacc, Bison's %type accepts tokens.
Unfortunately with string literals as implicit tokens, this is
misleading, and led some users to write
%type <exVal> cond "condition"
believing that "condition" would be associated to the 'cond'
nonterminal (see https://github.com/apache/httpd/pull/72).
* doc/bison.texi: Promote %nterm rather than %type to declare the type
of nonterminals.
* README: A few fixes.
Explain how to install color support.
* README-hacking: Rename as...
* README-hacking.md: this, and convert to Markdown.
Improve typography.
Improve explanations about update-test.
* src/gram.c (grammar_dump): Print terminals likewise non terminals.
* tests/sets.at (Reduced Grammar): Update test case to catch up the
change and add a test case where prec and assoc are used.
* tests/diagnostics.at (Locations from M4, Tabulations and multibyte
characters from M4): These tests are actually checking a message
coming from C, not from M4. Replace with...
(Complaints from M4): This.
We still have a few old C casts in lalr1.cc, let's get rid of them.
Reported by Frank Heckenbach.
Actually, let's monitor all our casts using easy to grep macros.
Let's use these macros to use the C++ standard casts when we are in
C++.
* data/skeletons/c.m4 (b4_cast_define): New.
* data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, data/skeletons/stack.hh,
* data/skeletons/yacc.c:
Use it and/or its casts.
* tests/actions.at, tests/cxx-type.at,
* tests/glr-regression.at, tests/headers.at, tests/torture.at,
* tests/types.at:
Use YY_CAST instead of C casts.
* configure.ac (warn_cxx): Add -Wold-style-cast.
* doc/bison.texi: Disable it.
* data/skeletons/location.cc: Remove the u (for unsigned) suffix from
the initial line and column.
* NEWS: AFAICT, only C++ backends have their location types changed.
We have too many global variables, adding structure would help. For a
start, let's hide some of the variables closer to their usage.
* src/getargs.c, src/files.h (current_file): Move to...
* src/scan-gram.c: here.
* src/scan-gram.h (gram_in, gram__flex_debug): Remove, make them
private to the scanner.
* src/reader.h, src/reader.c (reader): Take a grammar file as argument.
Move the handling of scanner variables to...
* src/scan-gram.l (gram_scanner_open, gram_scanner_close): here.
(gram_scanner_initialize): Remove, replaced by gram_scanner_open.
* src/main.c: Adjust.
* src/parse-gram (%initial-action): here.
(handle_skeleton): Don't depend on the current file name to look for
"local" skeletons (subject to changes coming from "#lines"): depend
only on the initial file name, the one given on the command line.
Currently there are two globals denoting the input file: grammar_file
is the one from the command line, and current_file which might change
because of #line. Use only the former.
* src/complain.c (error_message): here.
* tests/diagnostics.at: Adjust.
This skeleton uses a single stack of state structures, so it is less
likely to benefit from a stack size reduction than yacc.c (which uses
several stacks: state number, value and location). But it will reduce
the size of the LAC stack.
This skeleton was already using int for state numbers, so, contrary to
yacc.c, this brings nothing for large automata.
Overall, it is still nicer to make the skeletons alike.
* data/skeletons/lalr1.cc (state_type): Here.
Reported by Thomas Petazzoni.
https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00000.html
* examples/c/reccalc/local.mk: Complete dependencies, including for
earlier versions of Automake (for sake of our CI, on top of Ubuntu
Xenial/Bionic, which feature only Automake 1.15).
(%D%/scan.c %D%/scan.h): Upgrade to the full version provided in
Automake's documentation.
See https://lists.gnu.org/archive/html/bug-bison/2019-10/msg00061.html
to https://lists.gnu.org/archive/html/bug-bison/2019-10/msg00073.html.
Paul Eggert's changes in gnulib do fix the issue for modern GCCs (7,
8, 9) on macOS. Unfortunately these warnings are back on the
CI (GNU/Linux) with GCC 4.6, 4.7, (not 4.8) and 4.9.
Disable the warning locally.
* configure.ac (warn_common, warn_tests): Remove -Wtype-limits.
* src/system.h (IGNORE_TYPE_LIMITS_BEGIN, IGNORE_TYPE_LIMITS_END): New.
* src/InadequacyList.c, src/parse-gram.c, src/parse-gram.y,
* src/symtab.c: Use it.
This is really weird: GCC points to the LHS of the assignment...
260. headers.at:184: testing Sane headers: api.pure api.push-pull=both ...
tests/headers.at:184: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -d -o input.c input.y
tests/headers.at:184: $CC $CFLAGS $CPPFLAGS -c -o input.o input.c
stderr:
input.c: In function 'yyparse':
input.c:1276:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1276 | yylval = *yypushed_val;
| ~~~~~~~^~~~~~~~~~~~~~~
input.c: In function 'yypull_parse':
input.c:1276:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1276 | yylval = *yypushed_val;
| ~~~~~~~^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
stdout:
tests/headers.at:184: exit code was 1, expected 0
See also d87c8ac79a
and 9645a2b20e.
* tests/headers.at (Several parsers, Several parsers): Disable these
warnings when in push parser.
The documentation for Oracle Solaris Studio 12.3 (Sun C++ 5.12
2011/11/16) says it supports C++03. This compiler rejects the
location.cc use of std::max for some reason; I don’t know why
since I don’t use C++ as a rule. The simplest workaround is to
open-code ‘max’.
* data/skeletons/location.cc (add_):
Do max by hand rather than relying on std::max.
Don’t include <algorithm.h>; no longer needed.
As documented in the Autoconf manual, Solaris 10 sed rejects
script labels contianing more than 7 characters. POSIX requires
support for at least 8 characters, but we might as well be portable
to Solaris 10 which is still supported.
* tests/local.at (AT_SETS_CHECK): Use only the first 7 characters
in sed labels.
* src/symtab.c: Include intprops.h
(symbol_user_token_number_set): Don’t allow user_token_number ==
INT_MAX because too much other code adds 1 to the user token number.
(symbols_token_translations_init): Complain on integer overflow
instead of indulging in undefined behavior.
* src/scan-gram.l: Include errno.h, for errno.
(scan_integer, handle_syncline): Check for integer overflow.
* tests/input.at (too-large.y): Adjust to match new diagnostics.
* src/parse-gram.y: Include intprops.h.
(handle_require): Don’t indulge in undefined behavior if the major
or minor number is out of range. Instead, check that the
resulting value is nonnegative, fits in int, and that the minor
number is less than 100. Also, check that a number was parsed.
Sun C 5.12 defines __SUNPRO_C to 0x5120 but diagnoses
‘__attribute__ ((__unused__))’. Change the ifdefs to use
the same method as Gnulib in this area.
* data/skeletons/c.m4 (YY_ATTRIBUTE): Remove, since
not all attributes were added in the same compiler version.
(YY_ATTRIBUTE_PURE, YY_ATTRIBUTE_UNUSED):
Use specific GCC version for each attribute.
Pay no attention to __SUNPRO_C.
* tests/headers.at (Several parsers): Tighten tests accordingly.
Oracle Solaris Studio 12.3 (Sun C 5.12 2011/11/16) by default does
not conform to C99; it defines __STDC_VERSION__ to be 199409L, so
the Bison code does not include <stdint.h> (not required by C89
amendment 1) even though this compiler does have <stdint.h>. On
this platform <limits.h> defines INT_LEAST8_MAX (POSIX allows
this) so the skeleton got confused and thought that <stdint.h> had
been included even though it wasn’t.
* data/skeletons/c.m4 (b4_c99_int_type_define) [!__PTRDIFF_MAX__]:
Always include <limits.h>.
(YY_STDINT_H): Define when <stdint.h> was included.
All uses of expressions like ‘defined INT_LEAST8_MAX’ changed to
‘defined YY_STDINT_H’, since Sun C 5.12 <limits.h> defines macros
like INT_LEAST8_MAX but does not declare types like int_least8_t.
Let's make a difference between places where Perl is required for the
test (AT_PERL_REQUIRE), and the places where it's used to run the
test, but it's not not to run the test (AT_PERL_CHECK).
* tests/local.at (AT_REQUIRE): New.
(AT_PERL_CHECK, AT_PERL_REQUIRE): New.
Use them where appropriate.
* tests/local.mk ($(TESTSUITE)): Beware not to start the line with
'-pi' if Perl is empty, as Make understands this as "it's ok to fail".
Which it is not.
My previous tests (with ./configure PERL=false) have been fooled by
configure, that managed to find perl anyway. This time, I ran this on
a Fedora in Docker, without Perl.
* tests/calc.at, tests/diagnostics.at, tests/headers.at,
* tests/input.at, tests/local.at, tests/named-refs.at,
* tests/output.at, tests/regression.at, tests/skeletons.at,
* tests/synclines.at, tests/torture.at: Don't require Perl.
* data/skeletons/c.m4 (b4_c99_int_type_define): Reorder to put the
signed types first, since they’re simpler and this keeps similar
code closer. For signed types, don’t bother checking whether the
type promotes to int since the type must be signed anyway. For
unsigned types, protect a test like ‘UCHAR_MAX <= INT_MAX’ with
‘!defined __UINT_LEAST8_MAX__’, as otherwise the logic is wrong
for oddball platforms; and once we do that, there should no need
for ‘defined INT_MAX’ so remove that.
Currently we face test suite failures in different environments,
because of a conflict between the definitions of isnan by gnulib, and
by the C++ library:
262. headers.at:186: testing Sane headers: %locations %debug c++ ...
./headers.at:186: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -d -o input.cc input.y
./headers.at:186: $CXX $CXXFLAGS $CPPFLAGS -c -o input.o input.cc
stderr:
In file included from /usr/include/c++/4.8.2/cmath:44:0,
from /usr/include/c++/4.8.2/random:38,
from /usr/include/c++/4.8.2/bits/stl_algo.h:65,
from /usr/include/c++/4.8.2/algorithm:62,
from location.hh:41,
from input.hh:90,
from input.cc:50:
/u/cs/fac/eggert/src/gnu/bison/lib/math.h: In function 'bool isnan(double)':
/u/cs/fac/eggert/src/gnu/bison/lib/math.h:2849:1: error: new declaration 'bool isnan(double)'
_GL_MATH_CXX_REAL_FLOATING_DECL_2 (isnan, isnan, bool)
^
In file included from /usr/include/features.h:375:0,
from /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/os_defines.h:39,
from /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/c++config.h:2097,
from /usr/include/c++/4.8.2/cstdlib:41,
from input.hh:48,
from input.cc:50:
/usr/include/bits/mathcalls.h:235:1: error: ambiguates old declaration 'int isnan(double)'
__MATHDECL_1 (int,isnan,, (_Mdouble_ __value)) __attribute__ ((__const__));
^
There might be something to do in gnulib about this, but I believe
that gnulib should not be used in the test suite in the first place.
The test suite should work with other compilers than the one used to
compile the package. For a start, Bison sources are more
demanding (C99) than the generated parsers. Last time I tried, tcc
for example, was not able to compile Bison, yet our generated parsers
should compile cleanly with it.
Besides the problem at hand is with the C++ compiler, with is not the
one used to set up gnulib at configuration-time (config.h is mainly
built from probing the C compiler).
We should really not depend on gnulib in tests.
This was introduced in 2001 to check whether including
stdlib.h/string.h is safe thanks to STDC_HEADERS
(2ce1014469). Today, we assume at least
a C90 compiler, it should be safe enough.
* tests/local.at, tests/testsuite.h: Do not include config.h.
* tests/atlocal.in (conftest.cc): Likewise.
(CPPFLAGS): Do not expose lib/, as because of this we might picked up
gnulib replacement headers for system headers.
* tests/input.at: Use int instead of ptrdiff_t, for easier portability
(some machine on the CI did not find ptrdiff_t).
* tests/c++.at: Add missing include for getchar.
A number of portability issues with GCC 4.6 .. 4.9 (inclusive):
input.c:184:7: error: "UCHAR_MAX" is not defined [-Werror=undef]
#elif UCHAR_MAX <= INT_MAX
^
input.c:184:20: error: "INT_MAX" is not defined [-Werror=undef]
#elif UCHAR_MAX <= INT_MAX
^
input.c:202:7: error: "USHRT_MAX" is not defined [-Werror=undef]
#elif USHRT_MAX <= INT_MAX
^
input.c:202:20: error: "INT_MAX" is not defined [-Werror=undef]
#elif USHRT_MAX <= INT_MAX
^
* data/skeletons/c.m4 (b4_c99_int_type_define): Don't assume they are
defined.
Flex should not be required to build Bison or run the test suite (of
course it is needed for maintaining Bison). Yet the Automake
conditional FLEX_WORKS does not work.
* m4/flex.m4 (_AC_PROG_LEX_YYTEXT_DECL): Since this is called
conditionally, don't define LEX_IS_FLEX here, but rather...
(AC_PROG_LEX): here.
* configure.ac: Be more cautious about possibly undefined variables.
That way, glr.c can use it too.
* data/skeletons/c.m4 (b4_int_type):
Do not special-case ‘char’; it’s not worth the trouble,
as clang complains about char subscripts.
(b4_c99_int_type, b4_c99_int_type_define): New macros,
taken from yacc.c.
* data/skeletons/glr.c: Use b4_int_type_define.
* data/skeletons/yacc.c (b4_int_type): Remove, since there’s
no longer any need to redefine it.
Use b4_c99_int_type_define rather than its body.
This changes the Yacc skeleton to use “least” integer types to
keep tables smaller on some platforms, which should lessen cache
pressure. Since Bison uses the Yacc skeleton, it follows suit.
* data/skeletons/yacc.c: Include limits.h and stdint.h if this
seems to be needed.
(yytype_uint8, yytype_int8, yytype_uint16, yytype_int16):
If available, use GCC predefined macros __INT_MAX__ etc. to select
a “least” type, as this avoids namespace hassles. Otherwise, if
available fall back on selecting a “least” type via the C99 macros
INT_MAX, INT_LEAST8_MAX, etc. Otherwise, fall further back on one of
the builtin C99 types signed char, short, and int. Make sure that
any selected type promotes to int. Ignore any macros YYTYPE_INT16,
YYTYPE_INT8, YYTYPE_UINT16, YYTYPE_UINT8 defined by the user.
(ptrdiff_t, PTRDIFF_MAX): Simplify in the light of the above.
(yytype_uint8, yytype_uint16): Do not assume that unsigned char
and unsigned short promote to int, as this isn’t true on some
platforms (e.g., TI TMS320C55x).
* src/parse-gram.y (YYTYPE_INT16, YYTYPE_INT8, YYTYPE_UINT16)
(YYTYPE_UINT8): Remove, as these are no longer effective.
* data/skeletons/yacc.c (YYPTRDIFF_T, YYPTRDIFF_MAXIMUM):
Default to long, not int.
(yy_lac_stack_realloc, yy_lac, yytnamerr, yyparse):
Avoid casts to YYPTRDIFF_T that were masking the problem.
Because the checking of the grammar is made by phases after the whole
grammar was read, we sometimes have diagnostics that look weird. In
some case, within one type of checking, the entities are not checked
in the order in which they appear in the file. For instance, checking
symbols is done on the list of symbols sorted by tag:
foo.y:1.20-22: warning: symbol BAR is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
foo.y:1.16-18: warning: symbol QUX is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
Let's sort them by location instead:
foo.y:1.16-18: warning: symbol 'QUX' is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
foo.y:1.20-22: warning: symbol 'BAR' is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
* src/location.h (location_cmp): Be robust to empty file names.
* src/symtab.c (symbol_cmp): Sort by location.
* tests/input.at: Adjust expectations.
From
input.y:1.17-19: warning: symbol baz is used, but is not defined as a token and has no rules [-Wother]
1 | %printer {} foo baz
| ^~~
to
input.y:1.17-19: warning: symbol 'baz' is used, but is not defined as a token and has no rules; did you mean 'bar'? [-Wother]
1 | %printer {} foo baz
| ^~~
| bar
* bootstrap.conf: We need fstrcmp.
* src/symtab.c (symbol_from_uniqstr_fuzzy): New.
(complain_symbol_undeclared): Use it.
* tests/diagnostics.at (Suggestions): New.
* data/bison-default.css (insertion): Rename as...
(fixit-insert): this, as this is what GCC uses.
* src/symtab.c (complain_symbol_undeclared): New.
Use it.
Use quote on the guilty symbol (like GCC does, and we also do
elsewhere).
* tests/input.at: Adjust.
Both are stored in a hash, and back in the days, we used to iterate
over these tables using hash_do_for_each. However, the order of
traversal was not deterministic, which was a nuisance for
deterministic output (and therefore also a problem for tests). So at
some point (83b60c97ee) we generated a
sorted list of these symbols, and symbols_do actually iterated on that
list. But we kept the constraints of using hash_do_for_each, which
requires a lot of ceremonial code, and makes it hard/unnatural to
preserve data between iterations (see the next commit).
Alas, this is C, not C++.
Let's remove this abstraction, and directly iterate on the sorted
tables.
* src/symtab.c (symbols_do): Remove.
Adjust callers to use a simple for-loop instead.
(table_sort): New.
(symbols_check_defined): Use it.
(symbol_check_defined_processor, symbol_pack_processor)
(semantic_type_check_defined_processor, symbol_translation_processor):
Remove.
Simplify the corresponding functions (that no longer need to return a
bool).
This commit adds the suggestion in green, on the line below the
caret-and-tildes.
foo.y:1.1-14: warning: deprecated directive: '%error-verbose', use '%define parse.error verbose' [-Wdeprecated]
1 | %error-verbose
| ^~~~~~~~~~~~~~
| %define parse.error verbose
The current approach, with location_caret_suggestion, is fragile:
there's a protocol of calls to the complain functions which is strict.
We should rather have a richer structure describing the diagnostics,
including with submessages such as the suggestions, passed in the end
to the routines in charge of formatting and printing them.
* src/location.h, src/location.c (location_caret_suggestion): New.
* src/complain.c (deprecated_directive): Use it.
* tests/diagnostics.at, tests/input.at: Adjust expectations.
* src/location.c (caret_set_file): New.
Store the current line's length in caret_info.line_len.
Pay attention to fseek's return value.
Extracted from...
(location_caret): here.
I no longer agree with that item, there are indeed two things to
report: lack of definition, and being useless. We could have either
one without the other, they are not directly related.
input.c: In function 'int yyparse()':
input.c: error: conversion to 'long int' from 'long unsigned int'
may change the sign of the result [-Werror=sign-conversion]
yyes_capacity = sizeof yyesa / sizeof *yyes;
^
cc1plus: all warnings being treated as errors
* data/skeletons/yacc.c: here.
GCC 4.8 reports:
input.y:57:33: error: conversion to 'int' from 'long unsigned int'
may alter its value [-Werror=conversion]
int input_elts = sizeof input / sizeof input[0];
^
* tests/local.at (AT_YYLEX_DEFINE(c)): Add a cast (sorry, Paul!).
* src/location.c (caret_set_file):
* src/scan-code.l (contains_dot_or_dash):
Do not quietly convert pointer to bool, as Oracle Developer Studio
12.6 complains and it is arguably confusing style anyway.
For instance with GCC 4.9 and --enable-gcc-warnings:
25. input.at:1201: testing Torturing the Scanner ...
../../tests/input.at:1344: $CC $CFLAGS $CPPFLAGS -c -o input.o input.c
stderr:
input.c:239:18: error: "__STDC_VERSION__" is not defined [-Werror=undef]
# elif 199901 <= __STDC_VERSION__
^
input.c:256:18: error: "__STDC_VERSION__" is not defined [-Werror=undef]
# elif 199901 <= __STDC_VERSION__
^
* data/skeletons/yacc.c: Check that __STDC_VERSION__ is defined before
using it.
* doc/bison.texi (Mfcalc Symbol Table, Mfcalc Lexer):
Don’t abort on memory allocation failure or integer overflow.
Instead, comment that these things aren’t checked for.
On the CI with GCC 6:
examples/c++/calc++/parser.cc:845:5: error: 'ptrdiff_t' was not declared in this scope
ptrdiff_t yycount = 0;
^~~~~~~~~
examples/c++/calc++/parser.cc:845:5: note: suggested alternatives:
/usr/include/x86_64-linux-gnu/c++/6/bits/c++config.h:202:28: note: 'std::ptrdiff_t'
typedef __PTRDIFF_TYPE__ ptrdiff_t;
^~~~~~~~~
* data/skeletons/lalr1.cc: Qualify ptrdiff_t and size_t with std::.
* tests/torture.at (Big horizontal): Adjust to recent changes with
integers. If there are states 0..256, Bison now uses a signed
rather than an unsigned 16-bit integer.
This patch contains more fixes to prefer signed to unsigned
integer types, as modern tools like 'gcc -fsanitize=undefined'
can check for signed integer overflow but not unsigned overflow.
* NEWS: Document the API change.
* boostrap.conf (gnulib_modules): Add intprops.
* data/skeletons/glr.c: Include stddef.h and stdint.h,
since this skeleton can assume C99 or later.
(YYSIZEMAX): Now signed, and the minimum of SIZE_MAX and PTRDIFF_MAX.
(yybool) [!__cplusplus]: Now signed (which is how bool behaves).
(YYTRANSLATE): Avoid use of unsigned, and make the macro
safe even for values greater than UINT_MAX.
(yytnamerr, struct yyGLRState, struct yyGLRStateSet, struct yyGLRStack)
(yyaddDeferredAction, yyinitStateSet, yyinitGLRStack)
(yyexpandGLRStack, yymarkStackDeleted, yyremoveDeletes)
(yyglrShift, yyglrShiftDefer, yy_reduce_print, yydoAction)
(yyglrReduce, yysplitStack, yyreportTree, yycompressStack)
(yyprocessOneStack, yyreportSyntaxError, yyrecoverSyntaxError)
(yyparse, yy_yypstack, yypstack, yypdumpstack):
* tests/input.at (Torturing the Scanner):
Prefer ptrdiff_t to size_t.
* data/skeletons/c++.m4 (b4_yytranslate_define):
* src/AnnotationList.c (AnnotationList__computePredecessorAnnotations):
* src/AnnotationList.h (AnnotationIndex):
* src/InadequacyList.h (InadequacyListNodeCount):
* src/closure.c (closure_new):
* src/complain.c (error_message, complains, complain_indent)
(complain_args, duplicate_directive, duplicate_rule_directive):
* src/gram.c (nritems, ritem_print, grammar_dump):
* src/ielr.c (ielr_compute_ritem_sees_lookahead_set)
(ielr_item_has_lookahead, ielr_compute_annotation_lists)
(ielr_compute_lookaheads):
* src/location.c (columns, boundary_print, location_print):
* src/muscle-tab.c (muscle_percent_define_insert)
(muscle_percent_define_check_values):
* src/output.c (prepare_rules, prepare_actions):
* src/parse-gram.y (id, handle_require):
* src/reader.c (record_merge_function_type, packgram):
* src/reduce.c (nuseless_productions, nuseless_nonterminals)
(inaccessable_symbols):
* src/relation.c (relation_print):
* src/scan-code.l (variant, variant_table_size, variant_count)
(variant_add, get_at_spec, show_sub_message, show_sub_messages)
(parse_ref):
* src/scan-gram.l (<SC_ESCAPED_STRING,SC_ESCAPED_CHARACTER>)
(scan_integer, convert_ucn_to_byte, handle_syncline):
* src/scan-skel.l (at_complain):
* src/symtab.c (complain_symbol_redeclared)
(complain_semantic_type_redeclared, complain_class_redeclared)
(symbol_class_set, complain_user_token_number_redeclared):
* src/tables.c (conflict_tos, conflrow, conflict_table)
(conflict_list, save_row, pack_vector):
* tests/local.at (AT_YYLEX_DEFINE(c)):
Prefer signed to unsigned integer.
* data/skeletons/lalr1.cc (yy_lac_check_):
* tests/actions.at (_AT_CHECK_PRINTER_AND_DESTRUCTOR):
* tests/local.at (AT_YYLEX_DEFINE(c)):
Omit now-unnecessary casts.
* data/skeletons/location.cc (b4_location_define):
* doc/bison.texi (Mfcalc Lexer, C++ position, C++ location):
Prefer int to unsigned for line and column numbers.
Change example to abort explicitly on memory exhaustion,
and fix an off-by-one bug that led to undefined behavior.
* data/skeletons/stack.hh (stack::operator[]):
Also allow ptrdiff_t indexes.
(stack::pop, slice::slice, slice::operator[]):
Index arg is now ptrdiff_t, not int.
(stack::ssize): New method.
(slice::range_): Now ptrdiff_t, not int.
* data/skeletons/yacc.c (b4_state_num_type): Remove.
All uses replaced by b4_int_type.
(YY_CONVERT_INT_BEGIN, YY_CONVERT_INT_END): New macros.
(yylac, yyparse): Use them around conversions that -Wconversion
would give false alarms about. Omit unnecessary casts.
(yy_stack_print): Use int rather than unsigned, and omit
a cast that doesn’t seem to be needed here any more.
* examples/c++/variant.yy (yylex):
* examples/c++/variant-11.yy (yylex):
Omit no-longer-needed conversions to unsigned.
* src/InadequacyList.c (InadequacyList__new_conflict):
Don’t assume *node_count is unsigned.
* src/output.c (muscle_insert_unsigned_table):
Remove; no longer used.
* NEWS: Mention this.
* data/skeletons/c.m4 (b4_int_type):
Prefer char if it will do, and prefer signed types to unsigned if
either will do.
* data/skeletons/glr.c (yy_reduce_print): No need to
convert rule line to unsigned long.
(yyrecoverSyntaxError): Put action into an int to
avoid GCC warning of using a char subscript.
* data/skeletons/lalr1.cc (yy_lac_check_, yysyntax_error_):
Prefer ptrdiff_t to size_t.
* data/skeletons/yacc.c (b4_int_type):
Prefer signed types to unsigned if either will do.
* data/skeletons/yacc.c (b4_declare_parser_state_variables):
(YYSTACK_RELOCATE, YYCOPY, yy_lac_stack_realloc, yy_lac)
(yytnamerr, yysyntax_error, yyparse): Prefer ptrdiff_t to size_t.
(YYPTRDIFF_T, YYPTRDIFF_MAXIMUM): New macros.
(YYSIZE_T): Fix "! defined YYSIZE_T" typo.
(YYSIZE_MAXIMUM): Take the minimum of PTRDIFF_MAX and SIZE_MAX.
(YYSIZEOF): New macro.
(YYSTACK_GAP_MAXIMUM, YYSTACK_BYTES, YYSTACK_RELOCATE)
(yy_lac_stack_realloc, yyparse): Use it.
(YYCOPY, yy_lac_stack_realloc): Cast to YYSIZE_T to pacify GCC.
(yy_reduce_print): Use int instead of unsigned long when int
will do.
(yy_lac_stack_realloc): Prefer long to unsigned long when
either will do.
* tests/regression.at: Adjust to these changes.
Currently we properly use the "best" integral type for tables,
including those storing state numbers. However the variables for
state numbers used in yyparse (and its dependencies such as
yy_stack_print) still use int16_t invariably. As a consequence, very
large models overflow these variables.
Let's use the "best" type for these variables too. It turns out that
we can still use 16 bits for twice larger automata: stick to unsigned
types.
However using 'unsigned' when 16 bits are not enough is troublesome
and generates tons of warnings about signedness issues. Instead,
let's use 'int'.
Reported by Tom Kramer.
https://lists.gnu.org/archive/html/bug-bison/2019-09/msg00018.html
* data/skeletons/yacc.c (b4_state_num_type): New.
(yy_state_num): Be computed from YYNSTATES.
* tests/linear: New.
* tests/torture.at (State number type): New.
Use it.
Instead of
#define YYPACT_NINF -130
#define yypact_value_is_default(Yystate) \
(!!((Yystate) == (-130)))
generate
#define YYPACT_NINF (-130)
#define yypact_value_is_default(Yyn) \
((Yyn) == YYPACT_NINF)
* data/skeletons/c.m4 (b4_table_value_equals): Add support for $4.
* data/skeletons/glr.c, data/skeletons/yacc.c: Use it.
Also, use shorter macro argument names, the name of the macro is clear
enough.
* upstream/maint:
c++: add copy ctors for compatibility with the IAR compiler
CI: show git status
CI: disable ICC
tests: pass -jN from Make to the test suite
quotearg: avoid leaks
maint: post-release administrivia
Reported by Andreas Damm.
https://savannah.gnu.org/support/?110032
* data/skeletons/lalr1.cc (stack_symbol_type::operator=): New
overload, const, to please the IAR C++ compiler (version ca 2013).
* bootstrap.conf: We need winsz-ioctl and winsz-termios.
* src/location.c (columns): Use winsize to get the number of
columns.
Code taken from the GNU Coreutils.
* src/location.h, src/location.c (caret_init): New.
* src/complain.c (complain_init): Call it.
* tests/bison.in: Export COLUMNS so that users of tests/bison can
enjoy proper line truncation.
* src/location.c (min_int, columns): New.
(location_caret): Compute the line width. Based on it, compute how
many columns must be skipped before the quoted location and truncated
after, to fit the sceen width.
* tests/local.at (AT_QUELL_VALGRIND): Transform into...
(AT_SET_ENV_IF, AT_SET_ENV): these.
Define COLUMNS to protect the test suite from the user's environment.
So far diagnostics were cheating: in addition to the 'column' field of
locations (based on actual screen width per multibyte characters and
on tabulation expansion), the scanner sets the 'byte' field.
Diagnostics used this byte count to decide where to insert (color)
style.
We want to be able to truncate the quoted lines when there are too
wide to fit the screen. This requires that the diagnostics learn how
to count columns, the byte-in-boundary trick no longer works.
Bytes are still used for fix-its.
* bootstrap.conf: We need mbfile for mbf_getc.
* src/location.c (caret_info): We need an mbfile.
(caret_set_file): Initialize it.
(caret_getc): Convert to mbfile.
(location_caret): Instead of relying on the byte position to decide
where to insert the color style, count the current column using
boundary_compute.
* src/location.c (caret_info): Replace file and line with pos, a
boundary. This will allow us to use features of the boundary type,
such as boundary_compute.
The handling of the contributions of the tabulations in the columns is
burried inside location_compute. We will soon be willing to use the
boundary part of the computation (to compute the current column number
each time we read a multibyte char).
* src/location.c (boundary_compute): New, extracted from...
(location_compute): here.
This changes the traces from
Reading a token:
Now at end of input.
to
Reading a token:
Next token is token $end (7FFEE56E6474)
which is ok. Actually it is even better, as it gives the location
when locations are enabled, and is clearer when rules explicitly use
the EOF token.
* data/skeletons/lalr1.d (yytranslate_): Handle eof here, as is done
in lalr1.cc.
Bison used to feature %raw, documented as follows:
@item %raw
The output file @file{@var{name}.h} normally defines the tokens with
Yacc-compatible token numbers. If this option is specified, the
internal Bison numbers are used instead. (Yacc-compatible numbers start
at 257 except for single character tokens; Bison assigns token numbers
sequentially for all tokens starting at 3.)
Unfortunately, as far as I can tell, it never worked: token numbers
are indeed changed in the generated tables (from external token number
to internal), yet the code was still applying the mapping from
external token numbers to internal token numbers.
This commit reintroduces the feature as it was expected to be.
* data/skeletons/bison.m4 (b4_token_format): When api.token.raw is
enabled, use the internal token number.
* data/skeletons/yacc.c (yytranslate): Don't emit if api.token.raw is
enabled.
(YYTRANSLATE): Adjust.
We used to treat lone CRs (\r, aka ^M) as regular NLs (\n), probably
to please Classic MacOS. As of today, it makes more sense to treat \r
like a plain white space character.
https://lists.gnu.org/archive/html/bison-patches/2019-09/msg00027.html
* src/scan-gram.l (no_cr_read): Remove. Instead, use...
(eol): this new abbreviation denoting end-of-line.
* src/location.c (caret_getc): New.
(location_caret): Use it.
* tests/diagnostics.at (Carriage return): Adjust expectations.
(CR NL): New.
When the input file contains lone CRs (aka, ^M, \r), the locations see
a new line. Diagnostics look only at \n as end-of-line, so sometimes
there is an offset in diagnostics. Worse yet: sometimes we loop
endlessly waiting for \n to come from a continuous stream of EOF.
Fix that:
- check for EOF
- beware not to call end_use_class if begin_use_class was not
called (which would abort). This could happen if the actual
line is shorter that the expected one.
Prompted by a (private) report from Marc Schönefeld.
* src/location.c (location_caret): here.
* tests/diagnostics.at (Carriage return): New.
When the input file contains lone CRs (aka, ^M, \r), the locations see
a new line. Diagnostics look only at \n as end-of-line, so sometimes
there is an offset in diagnostics. Worse yet: sometimes we loop
endlessly waiting for \n to come from a continuous stream of EOF.
Fix that:
- check for EOF
- beware not to call end_use_class if begin_use_class was not
called (which would abort). This could happen if the actual
line is shorter that the expected one.
Prompted by a (private) report from Marc Schönefeld.
* src/location.c (location_caret): here.
* tests/diagnostics.at (Carriage return): New.
With
%token EOF 0 EOF 0
we get
input.y:3.14-16: warning: symbol EOF redeclared [-Wother]
3 | %token EOF 0 EOF 0
| ^~~
input.y:3.8-10: previous declaration
3 | %token EOF 0 EOF 0
| ^~~
Assertion failed: (nsyms == ntokens + nvars), function check_and_convert_grammar,
file /Users/akim/src/gnu/bison/src/reader.c, line 839.
Reported by Marc Schönefeld.
* src/symtab.c (symbol_user_token_number_set): Register only the
first definition of the end of input token.
* tests/input.at (Symbol redeclared): Check that case.
The CI, with CC='gcc-7 -fsanitize=undefined,address
-fno-omit-frame-pointer', reports:
calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
../../tests/calc.at:867: cat stderr
--- expout 2019-09-05 20:30:37.887257545 +0000
+++ /home/travis/build/bison-3.4.1.72-79a1-dirty/_build/tests/testsuite.dir/at-groups/438/stdout 2019-09-05 20:30:37.887257545 +0000
@@ -1 +1,2 @@
syntax error
+calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
438. calc.at:867: 438. Calculator glr.cc (calc.at:867): FAILED (calc.at:867)
The problem is that yylookaheadNeeds is not initialized in
yyinitStateSet, and when it is copied, the value is not 0 or 1.
* data/skeletons/glr.c (yylookaheadNeeds): Initialize yylookaheadNeeds.
hash_initialize returns NULL when out of memory. Check for it, and
die cleanly instead of crashing.
Reported by 江 祖铭 (Zu-Ming Jiang).
https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00015.html
* src/muscle-tab.c, src/state.c, src/symtab.c, src/uniqstr.c:
Check the value returned by hash_initialize.
https://lists.gnu.org/archive/html/bison-patches/2019-08/msg00007.html
When Bison is started with a flag that suppresses warning messages, the
error_message() function can produce a few gigabytes of indentation
because of a dangling pointer.
* src/complain.c (error_message): Don't reset indent_ptr here, but...
(complain_indent): here.
* tests/diagnostics.at (Indentation with message suppression): Check
this case.
* .travis.yml: Bionic is now available, with GCC8.
GCC7 sanitizers work, but they are too longer: cover only part 1.
Redefine part 1 and part 2 so that part 1 is really the core of the
tests: not playing with POSIX and C++ compiler for C code.
Travis answered favorably to my suggestion to provide a means to
disable git clone on some jobs (issue 7542). See
https://docs.travis-ci.com/user/customizing-the-build/#disabling-git-clone.
* .travis.yml: Disable git globally, enable it for i. the compile job,
and ii. the test job on ICC which needs the install-icc.sh script.
With
%token EOF 0 EOF 0
we get
input.y:3.14-16: warning: symbol EOF redeclared [-Wother]
3 | %token EOF 0 EOF 0
| ^~~
input.y:3.8-10: previous declaration
3 | %token EOF 0 EOF 0
| ^~~
Assertion failed: (nsyms == ntokens + nvars), function check_and_convert_grammar,
file /Users/akim/src/gnu/bison/src/reader.c, line 839.
Reported by Marc Schönefeld.
* src/symtab.c (symbol_user_token_number_set): Register only the
first definition of the end of input token.
* tests/input.at (Symbol redeclared): Check that case.
The CI, with CC='gcc-7 -fsanitize=undefined,address
-fno-omit-frame-pointer', reports:
calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
../../tests/calc.at:867: cat stderr
--- expout 2019-09-05 20:30:37.887257545 +0000
+++ /home/travis/build/bison-3.4.1.72-79a1-dirty/_build/tests/testsuite.dir/at-groups/438/stdout 2019-09-05 20:30:37.887257545 +0000
@@ -1 +1,2 @@
syntax error
+calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
438. calc.at:867: 438. Calculator glr.cc (calc.at:867): FAILED (calc.at:867)
The problem is that yylookaheadNeeds is not initialized in
yyinitStateSet, and when it is copied, the value is not 0 or 1.
* data/skeletons/glr.c (yylookaheadNeeds): Initialize yylookaheadNeeds.
* .travis.yml: Bionic is now available, with GCC8.
GCC7 sanitizers work, but they are too longer: cover only part 1.
Redefine part 1 and part 2 so that part 1 is really the core of the
tests: not playing with POSIX and C++ compiler for C code.
hash_initialize returns NULL when out of memory. Check for it, and
die cleanly instead of crashing.
Reported by 江 祖铭 (Zu-Ming Jiang).
https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00015.html
* src/muscle-tab.c, src/state.c, src/symtab.c, src/uniqstr.c:
Check the value returned by hash_initialize.
https://lists.gnu.org/archive/html/bison-patches/2019-08/msg00007.html
When Bison is started with a flag that suppresses warning messages, the
error_message() function can produce a few gigabytes of indentation
because of a dangling pointer.
* src/complain.c (error_message): Don't reset indent_ptr here, but...
(complain_indent): here.
* tests/diagnostics.at (Indentation with message suppression): Check
this case.
Implement lookahead correction (LAC) for the C++ skeleton. LAC is a
mechanism to make sure that we report the correct list of expected
tokens if a syntax error occurs. So far, LAC was only supported for
the C skeleton "yacc.c".
* data/skeletons/lalr1.cc: Add LAC support.
* doc/bison.texi: Update.
* data/skeletons/yacc.c (yysyntax_error): Change the nesting of `m4`
conditions slightly to make it more readable.
The generated C code stays unchanged.
* tests/regression.at: Use %expect to avoid warnings.
Set the keywords to facilitate running specific tests.
Use macros such as AT_YYLEX_DECLARE to facilitate tests for other
languages.
Likewise for AT_FULL_COMPILE.
In Texinfo. empty lines in multitable rows generate empty lines in the
output. Avoid them altogether.
With help from Gavin Smith.
https://lists.gnu.org/archive/html/bug-texinfo/2019-07/msg00000.html
* build-aux/cross-options.pl: Separate rows with empty lines.
So, to be more readable, generate a single line for each row.
Use Perl format to this end.
See the previous commit. This option should be removed, -o suffices.
* src/getargs.c (FIXED_OUTPUT_FILES): New.
Add support for it.
(getargs): Define loc, and use it.
This is safer when we need to pass a pointer to a location.
The name fixed-output-files is pretty clear: generate y.tab.c, as Yacc
does. So let's detach this from %yacc which does more: it requires
POSIX Yacc behavior.
This directive is obsolete since December 29th 2001
8c9a50bee1. It does not show in the
doc. I don't want to spend more time on improving its diagnostics, it
could be removed just as well as far as I'm concerned.
* src/scan-gram.l, src/parse-gram.y (%fixed-output-files): Detach from
%yacc.
Years ago we moved from 'look-ahead' to 'lookahead', and that alias
was kept for backward compatibility. But now that we use argmatch to
generate the documentation, that value clutters the doc.
* src/getargs.c (argmatch_report_args): Remove the
--report=look-aheads alias.
The doc says that -Dfoo=bar is the same as %define foo "bar". It is
not: the quotes are not added (and it makes a difference).
* doc/bison.texi (Tuning the Parser): Fix the definition of -D/-F
* src/getargs.c (usage): Likewise.
Let's clarify --help: use clearer "section" names, as in the doc.
Move --yacc to where it belongs.
* src/getargs.c (usage): Rename "Parser" as "Tuning the Parser", as in
the doc.
Rename "Output" as "Output Files"
Move --yacc to "Tuning the Parser".
* doc/bison.texi: Likewise.
It can now generate the usage message.
* src/complain.h (feature_fixit_parsable): Rename as...
(feature_fixit): this, for column economy.
Adjust dependencies.
(warning_usage): New.
Use it.
* src/complain.h, src/complain.c, src/getargs.h, src/getargs.c:
Use ARGMATCH_DEFINE_GROUP instead of the older interface.
The code is inconsistent: sometimes we pass by value, sometimes by
reference. Let's stick to the last, more conventional for large
values in C.
* src/scan-code.l: Pass locations by reference.
* configure.ac (DCFLAGS): Pass -g.
* data/skeletons/d.m4 (b4_locations_if): Remove, let bison.m4's one do
its job.
* data/skeletons/lalr1.d (position): Leave filename empty by default.
(position::toString): Don't print empty file names.
(location::this): New ctor.
(location::toString): Match the implementations of C/C++.
(yy_semantic_null): Leave undefined, the previous implementation does
not compile.
* tests/calc.at: Improve the implementation for D.
Enable more checks, in particular using locations.
* tests/local.at (AT_YYERROR_DEFINE(d)): Fix its implementation.
Sadly enough, AFAIK, there were never answers to the "More user
feedback will help to stabilize it" sentences. Remove them.
* src/getargs.c: IELR, canonical LR and XML output are here to stay,
and they are no more experimental than some other features.
* doc/bison.texi: Likewise.
Also remove "experimental" warning for Java, LAC, LR tuning options,
and named references.
* data/skeletons/lalr1.java: Use more conventional function names for
Java.
Prefer < and <= to => and >.
Use the same approach for m4 quotation as in the other skeletons.
Fix indentation issues.
* tests/calc.at, tests/java.at, tests/javapush.at: Fix quotation style.
(main): Use 'args', not 'argv', the former seems more conventional and
is used elsewhere in Bison.
Prefer character literals to integers to denote characters.
* examples/java/Calc.y: Likewise.
Travis answered favorably to my suggestion to provide a means to
disable git clone on some jobs (issue 7542). See
https://docs.travis-ci.com/user/customizing-the-build/#disabling-git-clone.
* .travis.yml: Disable git globally, enable it for i. the compile job,
and ii. the test job on ICC which needs the install-icc.sh script.
This is an experiment. Maybe more styles will be used (in which case
a short-hand function will be useful), maybe it will be just reverted.
* data/bison-default.css (.traces0): New.
* src/lalr.c (lalr): Use it.
See e3fdc37049: in C++ we generate
explicitly the code for the default action instead of simply copying
blindly the semantic value buffer. This is important when copying
raw memory is not enough, as exemplified by move-only types.
This is currently tested by examples/c++/variant.yy and variant-11.yy.
But it is safer to also have a test in the main test suite.
* tests/local.at (AT_REQUIRE_CXX_STD): Fix.
(AT_BISON_OPTION_PUSHDEFS, AT_BISON_OPTION_POPDEFS): Define/undefine
AT_BISON_OPTIONS.
* tests/c++.at (Default action): New.
It will be convenient to check sets elsewhere.
* tests/sets.at (AT_EXTRACT_SETS): Transform into...
* tests/local.at (AT_SETS_CHECK): this.
* tests/sets.at: Adjust.
The final gain is small: 2h2min instead 2h9min. But that is still an
improvement.
* .travis.yml (git.depth): Make the clone very shallow.
(git.submodules): Don't clone gnulib in test jobs.
(jobs.include.compile.script): Do it here.
* data/diagnostics.css: Rename as...
* data/bison-default.css: this.
Add the GPL header.
This is the convention followed by Bruno Haible in gettext.
Adjust dependencies.
* src/complain.c (complain_init_color): Use BISON_STYLE instead of
BISON_DIAGNOSTICS_STYLE.
Build the tarball in one job, check it in many.
Unfortunately no real gain in overall duration.
With help from Clément Démoulins.
* .travis.yml: here.
Remove all the tricks that were used to be able to boostrap on old
distros.
(before_install): Merge into 'script', because before_install applies
to all the jobs, and we don't want to run it for the 'compile' job.
In preparation for Bison 3.4, revert to the version before this
commit:
commit 03752516b21091cf3c4beea7e8b9bcad462d50ed
Author: John Darrington <[email protected]>
Date: Sun May 12 00:42:36 2019 +0200
version-etc: Ease translation.
* lib/version-etc.c (version_etc_arn, emit_bug_reporting_address):
Move URLs and formatting newlines out of translatable string.
because it changes the messages for --version, translated in
gnulib.po. These changes are not yet available on the translation
project, so we would have a regression in the set of translated
strings.
It works only in implicit rules (or with GNU Make, but not with
Solaris Make).
Reported by Bruno Haible.
http://lists.gnu.org/archive/html/bug-bison/2019-05/msg00009.html
Diagnosed thanks to Kiyoshi Kanazawa.
* examples/c/reccalc/local.mk, examples/d/local.mk,
* examples/java/local.mk: Don't use $< in non implicit rules.
It is more consistent with --color=html, --color=test, etc.
* src/getargs.h, src/getargs.c (style_debug): Rename as...
(color_debug): this.
(getargs_colors): Rename --style=debug as --color=debug.
Adjust dependencies.
An experimental commit introduced a fix-it hint that changes comments
such as "/* empty */" into %empty. But in some case, because
diagnostics are not necessarily emitted in order, the fixits also come
in disorder, which must never happen, as the fixes are installed in
one pass.
* src/fixits.c (fixits_register): Insert them in order.
Currently we remove the rhs to install %empty instead.
* src/reader.c (grammar_rule_check_and_complete): Insert the missing
%empty in front of the rhs, not in replacement thereof.
* tests/actions.at (Add missing %empty): Check that.
Some members are called foo_location, others are foo_loc. Stick to
the latter.
* src/gram.h, src/location.h, src/location.c, src/output.c,
* src/parse-gram.y, src/reader.h, src/reader.c, src/reduce.c,
* src/scan-gram.l, src/symlist.h, src/symlist.c, src/symtab.h,
* src/symtab.c:
Use _loc consistently, not _location.
symbol_list features a 'location' and a 'sym_loc' member. The former
is expected to be set only for symbol_lists that denote a symbol (not
a type name), and the latter should only denote the location of the
symbol/type name. Yet both are set, and the name "location" is too
unprecise.
* src/symlist.h, src/symlist.c (symbol_list::location): Rename as
rhs_loc for clarity. Move it to the "section" of data valid only
for rules.
* src/reader.c, src/scan-code.l: Adjust.
Revert "build: also generate the graph
reports" (4ec413da32). The problem is
Automake's ylwrap which does not rename y.dot with the appropriate
name. We should completely stop using Automake's support for Yacc,
which is not something I will do right now. So step back.
* Makefile.am (AM_YFLAGS_WITH_LINES): Don't pass --graph.
Because some of our examples use
%C%_reccalc_SOURCES = %D%/parse.y
Automake ships parse.y and parse.c, and possibly parse.h when it
"understands" that there is one. This is not what we want: ship only
parser.y. Yet we still want to use Automake to compile the sources
from parser.y. The easiest seems to use
nodist_%C%_reccalc_SOURCES = %D%/parse.y
together with
dist_reccalc_DATA = %D%/parse.y %D%/scan.l %D%/Makefile %D%/README.md
which guarantees that parse.y is indeed shipped.
* examples/c/calc/local.mk, examples/c/lexcalc/local.mk,
* examples/c/reccalc/local.mk: Always use nodist_*SOURCES for parsers,
let the dist_*_DATA rules do their job.
* cfg.mk: Disable checks where needed (e.g., we do want to check the
behavior with tabs).
(sc_at_parser_check): Remove. Unfortunately since
a11c144609 we no longer use the './'
prefix to run programs in the current directory. That was so that we
could run Java programs like the other, although they are no run with
the `./` prefix (see 967a59d2c0).
As a consequence this sc check no longer makes sense.
However, since now AT_PARSER_CHECK passes the `./` prefix itself, this
sc-check was superfluous.
* examples/c/reccalc/scan.l: Use memcpy, not strncpy.
* src/ielr.c, src/reader.c: Obfuscate "lr(0)" so that the sc-check for
"space before paren" does not fire.
* tests/diagnostics.at: Avoid space-tab, use tab-tab.
This makes reading the trace slightly easier. It would be very nice
to highlight the "big steps", especially reductions. But this is a
private experiment: do not use it.
* data/diagnostics.css (value): New.
* src/parse-gram.y: Use no delimiters and no c quotation for strings
to facilitate debugging.
(tron, troff, TRACE): New.
Not very elegant, but until there is support for printf-formats in
libtextstyle, it shall be enough.
Currently we pass only the columns based on the screen-width, which is
important for the carets. But we don't pass the bytes-based columns,
which is important for the colors. Pass both.
* src/muscle-tab.c (muscle_boundary_grow): Also pass the byte-based column.
* src/location.c (location_caret): Clarify.
(boundary_set_from_string): Adjust to the new format.
* tests/diagnostics.at (Tabulations and multibyte characters from M4): New.
Locations issued from M4 need the byte-based column for the
diagnostics to work properly. Currently they were unassigned, which
typically resulted in partially non-colored diagnostics.
* src/location.c (boundary_set_from_string): Fix the parsed location.
* src/muscle-tab.c (muscle_percent_define_default): Set the byte values.
* tests/diagnostics.at (Locations from M4): New.
This is meant for developers, not end users, that's why I attached it
to --trace.
* src/getargs.h, src/getargs.c (trace_locations): New.
* src/location.c (location_print): Use it.
We were using the gnulib's gettext module with tricks in
bootstrap.conf to avoid useless files. Instead, use gnulib's
gettext-h module.
* .travis.yml: Force Gettext 0.18.3 on Trusty.
* bootstrap.conf: Use gettext-h instead of gettext.
(excluded_files): Remove.
* configure.ac (AM_GNU_GETTEXT_VERSION): Bump to 0.19.
Reported by Balázs Scheidler.
* data/skeletons/c.m4 (b4_location_type_define): Use api.location.type
if defined.
* doc/bison.texi: Document it.
* tests/local.at (AT_C_IF, AT_LANG_CASE): New.
Support Span in C.
* tests/calc.at (Span): Convert it to be usable in C and C++.
Check api.location.type with yacc.c and glr.c.
The "identifier and colon" of a rule is implemented as a single token,
but whose location is only that of the identifier (so that messages
about the lhs of a rule are accurate). When reducing empty rules, the
default location is the single point location on the end of the
previous symbol. As a consequence, when Bison parses a grammar, the
location of the right-hand side of an empty rule is based on the
lhs, *independently of the position of the colon*. And the colon can
be way farther, separated by comments, white spaces, including empty
lines.
As a result, some messages look really bad. For instance:
$ cat foo.y
%%
foo : /* empty */
bar
: /* empty */
gives
$ bison -Wall foo.y
foo.y:2.4: warning: empty rule without %empty [-Wempty-rule]
2 | foo : /* empty */
| ^
foo.y:3.4: warning: empty rule without %empty [-Wempty-rule]
3 | bar
| ^
The carets are not at the right column, not even the right line.
This commit passes the colon "again" after the "id colon" token, which
gives more accurate locations for these messages:
$ bison -Wall foo.y
foo.y:2.10: warning: empty rule without %empty [-Wempty-rule]
2 | foo : /* empty */
| ^
foo.y:4.2: warning: empty rule without %empty [-Wempty-rule]
4 | : /* empty */
| ^
* src/scan-gram.l (SC_AFTER_IDENTIFIER): Rollback the colon, so that
we scan it again afterwards.
(INITIAL): Scan colons.
* src/parse-gram.y (COLON): New.
(rules): Parse the colon after the rule's id_colon (and possible
named reference).
* tests/actions.at, tests/conflicts.at, tests/diagnostics.at,
* tests/existing.at: Adjust.
Because the fix-its were ready the character-based columns, but were
applied on byte-based columns, the result with multibyte characters or
tabs could be "interesting". For instance
%fixed-output_files
%fixed_output-files
%fixed-output-files
%define api.prefix {foo}
%no-default-prec
would give
%fixed-%fixed-output-files %fixed_output-files
%fixed-orefix= "foo"
o_default-prec
* src/fixits.c (fixit_print, fixits_run): Work on byte-base columns.
* tests/input.at: Check it.
Currently, when we quote the source file, we indent it with one space,
and preserve tabulations, so there is a discrepancy and the visual
rendering is bad. One way out is to indent with a tab instead of a
space, but then this space can be used for more information. This is
what GCC9 does. Let's play copy cats.
See
https://lists.gnu.org/archive/html/bison-patches/2019-04/msg00025.htmlhttps://developers.redhat.com/blog/2019/03/08/usability-improvements-in-gcc-9/https://gcc.gnu.org/onlinedocs/gccint/Guidelines-for-Diagnostics.html#Guidelines-for-Diagnostics
* src/location.c (location_caret): Prefix quoted lines with the line
number and a pipe, fitting 8 columns.
* tests/actions.at, tests/c++.at, tests/conflicts.at,
* tests/diagnostics.at, tests/input.at, tests/java.at,
* tests/named-refs.at, tests/reduce.at, tests/regression.at,
* tests/sets.at: Adjust expectations.
Partly by "./build-aux/update-test tests/testsuite.dir/*/testsuite.log"
repeatedly, and partly by hand.
This is a pity: efforts were invested in computing correctly the
number of screen columns consumed by multibyte characters, but the
routines that do that were fed by single-byte inputs...
As a consequence Bison never displayed correctly locations when there
are multibyte characters.
* src/scan-gram.l (mbchar): New.
Use it instead of . in the catch-all clause.
* tests/diagnostics.at (Tabulations): Enhance into...
(Tabulations and multibyte characters): this.
Single point locations (equal boundaries) are troublesome, and we were
incorrectly ending the style in their case. Which results in an abort
in libtextstyle.
There is also a confusion between columns as displayed on the
screen (which take into account multibyte characters and tabulations),
and the number of bytes. Counting the screen-column
incrementally (character by character) is uneasy (because of multibyte
characters), and I don't want to maintain a buffer of the current line
when displaying the diagnostic. So I believe the simplest solution is
to track the byte number in addition to the screen column.
* src/location.h, src/location.c (boundary): Add the byte-column.
Adjust dependencies.
* src/getargs.c, src/scan-gram.l: Adjust.
* tests/diagnostics.at: Check zero-width locations.
Enable checking of styles even when libtextstyle is not installed.
* src/getargs.h, src/getargs.c (style_debug): New.
(getargs_colors): Set it when --style=debug.
* src/complain.c (begin_use_class, end_use_class): Use it.
* tests/diagnostics.at: New.
* src/lalr.c: Move logs to a better place to understand the chronology
of events.
* src/symlist.c (symbol_list_syms_print): Don't dump core on type
elements.
The 'regen' commit in Bison's history are a nuisance. They are
especially big because of the #lines. Let's generate our parse
without these lines in the repository, but generate them in the
tarball.
* Makefile.am (AM_YFLAGS_WITH_LINES): New.
(AM_YFLAGS): Use it.
(dist-hook): Regenerate the parser with #lines.
Currently, with --no-lines, instead of "#line file line\n", we emit
"\n". Let's emit nothing.
* data/skeletons/bison.m4 (b4_syncline): Emit at end-of-line when enabled.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, src/output.c: Use dnl after b4_syncline to
avoid spurious empty lines.
* tests/synclines.at (Sync Lines): Make sure that --no-lines is like
grep -v #line.
* tests/calc.at: Make sure that a rich grammar file behaves properly
with %no-lines.
Currently we use the syncline to report errors about a symbol's
destructor/printer. This is not accurate (only file and line), and
this is incorrect: the file name is double quotes (a recent change,
needed to make sure we escape properly double quotes in it). And
worst of all: with --no-line, b4_syncline expands to nothing.
Rather, push the locations into the backend, and use them.
* src/muscle-tab.h, src/muscle-tab.c (muscle_location_grow): Make it
public.
* src/output.c (prepare_symbol_definitions): Use it to pubish the
location of the printer and destructor.
* data/skeletons/lalr1.java: Use complain_at instead of complain.
* tests/java.at (Java invalid directives): Adjust expectations.
* data/skeletons/bison.m4 (b4_symbol_action_location): Remove.
We should not use b4_syncline this way.
Fatal errors are inconvenient, and should be reserved to cases where
we cannot continue. Here, it could even be warnings actually: these
directives will simply be ignored.
* data/skeletons/lalr1.java: Prefer error (b4_complain) to fatal
errors (b4_fatal).
* tests/java.at (Java invalid directives): New.
I never understood why we book ngotos+1 slots for relations between
gotos: there are at most ngotos images, not ngotos+1 (and "includes"
does have cases where a goto is in relation with itself, so it's not
ngotos-1).
Maybe bbf37f2534 explains the +1: a bug
left us register a goto several times on occasion, and the +1 might
have been a means to avoid this problem in most cases. Now that this
bug is addressed, we should no longer overbook memory, if only for the
clarity of the code ("why ngotos+1 instead of ngotos?").
* src/lalr.c: A goto has at most ngotos images, not ngotos+1.
While at it, avoid useless repeated call to map_goto introduced in
bbf37f2534.
Reported by wcventure.
http://lists.gnu.org/archive/html/bug-bison/2019-03/msg00008.html
* src/symtab.c (complain_class_redeclared): Don't print empty
locations.
There can only be empty locations for predefined symbols. And the
only symbol that is lexically available is the error token. So this
appears to be the only possible way to have an error involving an
empty location.
* tests/input.at (Symbol class redefinition): Check it.
The "includes" relation [DeRemer 1982] is between gotos, so of course,
for a given goto, there cannot be more that ngotos (number of gotos)
images. But we manipulate the set of images of a goto as a list,
without checking that an image was not already introduced. So we can
"register" way more images than ngotos, leading to a crash (heap
buffer overflow).
Reported by wcventure.
http://lists.gnu.org/archive/html/bug-bison/2019-03/msg00007.html
For the records, this bug is present in the first committed version of
Bison.
* src/lalr.c (build_relations): Don't insert the same goto several
times.
* tests/sets.at (Build Relations): New.
It seems that not many people read these logs: the error was
introduced in 2001 (3067fbef53),
* src/gram.c (grammar_dump): Fix the headers of the table: remove
duplicate display of "Ritem Range".
While at it, remove duplicate display of the rule number (and remove
an incorrect comment about it: these numbers _are_ equal).
* tests/sets.at (Reduced Grammar): Use useless rule, nterm and token
in the example.
When we update some output format, too many adjustements must be made
by hand. This script updates most tests based on the actual output
made during the tests.
* build-aux/update-test: New.
Suggested by Yuri at
<http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00000.html>.
The improvement is marginal for most grammars, but notable for large
grammars (e.g., PosgreSQL's postgre.y), and very large for the
sample.y grammar submitted by Yuri in
http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00012.html.
Measured with --trace=time -fsyntax-only.
parser action tables postgre.y sample.y
Before 0,129 (44%) 37,095 (99%)
After 0,117 (42%) 5,046 (93%)
* src/tables.c (pos): Replace this set of integer coded as an unsorted
array or integers with...
(pos_set): this bitset.
* src/reader.c (grammar_rule_check_and_complete): When 'p' and 'lhs'
are aliases, prefer the latter, for clarity and consistency.
(grammar_current_rule_begin): Avoid 'p', current_rule suffices.
* src/gram.h, src/gram.c: Comment changes.
ptdr# calc.tab.c
Bruno Haible just added a default implementation of libtextstyle's
interface when the library is not available.
https://lists.gnu.org/archive/html/bison-patches/2019-03/msg00025.html
* gnulib: Update.
* bootstrap.conf: Replace libtextstyle with libtextstyle-optional.
* src/complain.c, src/getargs.c: Remove now useless cpp guards.
The timevar and bitset modules now use the c99 module which causes
$CXX to now include -std=gnu++11 when possible. Unfortunately, G++
4.7 does not implement [[noreturn]] in C++11 mode, so our tests of
glr.cc (which uses _Noreturn) fail with
input.cc:954:1: error: expected unqualified-id before '[' token
right before [[noreturn]]. 4.8 works fine.
* data/skeletons/c.m4 (b4_attribute_define): Do not use [[noreturn]]
with GCC 4.7.
See f8408562f8.
* tests/calc.at: Stop imitating the C API.
Prepare more tests to run in the future.
%verbose works as expected (what a surprise, it's unrelated to the
skeleton...).
The variable spec_defines_file denotes the name of the generated
header. Its name is derived from --defines/%defines, whose name in
turn is derived from the fact that the header, in Yacc, contained the
Not only does the header now contain a lot more than just the token
definitions, but we no longer even generate macros, but an enum...
Let's modernize our vocabulary.
* src/files.h, src/files.c (spec_defines_file): Rename as...
(spec_header_file): this.
Currently when --defines is used, we generate a header, and paste an
exact copy of it into the generated parser implementation file. Let's
provide a means to #include it instead.
We don't do it by default because of the Autotools' ylwrap. This
program wraps invocations of yacc (that uses a fixed output name:
y.tab.c, y.tab.h, y.output) to support a more modern naming
scheme (dir/foo.y -> dir/foo.tab.c, dir/foo.tab.h, etc.). It does
that by renaming the generated files, and then by running sed to
propagate these renamings inside the files themselves.
Unfortunately Automake's Makefiles uses Bison as if it were Yacc (with
--yacc or with -o y.tab.c) and invoke bison via ylwrap. As a
consequence, as far as Bison is concerned, the output files are
y.tab.c and y.tab.h, so it emits '#include "y.tab.h"'. So far, so
good. But now ylwrap processes this '#include "y.tab.h"' into
'#include "dir/foo.tab.h"', which is not guaranteed to always work.
So, let's do the Right Thing when the output file is not y.tab.c, in
which case the user should %define api.header.include. Binding this
behavior to --yacc is tempting, but we recently told people to stop
using --yacc (as it also enables the Yacc warnings), but rather to use
-o y.tab.c.
Yacc.c is the only skeleton concerned: all the others do include their
header.
* data/skeletons/yacc.c (b4_header_include_if): New.
(api.header.include): Provide a default value when the output is not
y.tab.c.
* src/parse-gram.y (api.header.include): Define.
Running with CC='gcc-mp-8 -fsanitize=undefined' revealed Undefined
Behaviors.
https://lists.gnu.org/archive/html/bison-patches/2019-03/msg00008.html
* src/state.c (errs_new): Don't call memcpy with NULL as source.
* src/location.c (add_column_width): Don't assume that the column
argument is nonnegative: the scanner sometimes "backtracks" (e.g., see
ROLLBACK_CURRENT_TOKEN and DEPRECATED) in which case we can have
negative column numbers (temporarily).
Found in test 3 (Invalid inputs).
* data/skeletons/yacc.c: here.
This is more logical for the time stamps, but it's also required by
following patches: the shared declarations are also in charge of
handling api.value.type=union. So far, they are run in the
implementation file in both cases (with or without header). But if we
run them only in the header, then the implementation file is emited
with incorrect support for api.value.type=union.
Arguably we should not have such dependencies. This is because we
have side-effects in our backend (redefining the symbols' type and
type_tag). In the future we should find a better solution for this,
without sacrificing the independence of the backend from bison
itself (i.e., I don't think we should handle api.value.type=union in
bison, leave it to m4).
Currently we generate things like:
#line 683 "src/parse-gram.y" /* yacc.c:316 */
The first part is of course very important: compilers point the users
to their grammar file rather than into the generated parser. The
second part points to the place in the skeletons that generated this
piece of code.
This dependency on the Bison skeletons generates lots of useless 'git
diff'. This location is useless for the regular user (who does not
care about the skeletons) and is actually not useful for Bison
developpers too (I never used this to locate the code in skeletons
that generated output). So disable it completely. If someone thinks
this was actually useful, a %define variable should be provided to
control the level of verbosity of '#line', in replacement of
--no-lines.
So now, generate:
#line 683 "src/parse-gram.y"
* data/skeletons/bison.m4 (b4_sync_end): Emit nothing.
* src/lalr.c (follows_print): Just print the symbol tag.
Take and print a title.
Indent the output.
Use it to print the various steps of the computation.
(lookahead_tokens_print): Fix a lie: the number displayed is not the
number of tokens.
Don't display states that don't even have reductions.
* src/relation.h, src/relation.c (relation_print): Accept and use a
title.
Don't print empty rows.
Indent the output.
Adjust dependencies.
* src/lalr.c (initialize_goto_follows): Print 'reads' in traces.
* configure.ac (DCFLAGS): Define.
* tests/atlocal.in: Receive it.
* data/skeletons/d.m4 (api.parser.class): Remove spurious YY.
* data/skeletons/lalr1.d (yylex): Return an int instead of a
YYTokenType, so that we can use characters as tokens.
* examples/d/calc.y: Adjust.
* tests/local.at: Initial support for D.
(AT_D_IF, AT_DATA_GRAMMAR(D), AT_YYERROR_DECLARE(d))
(AT_YYERROR_DECLARE_EXTERN(d), AT_YYERROR_DEFINE(d))
(AT_MAIN_DEFINE(d), AT_COMPILE_D, AT_LANG_COMPILE(d), AT_LANG_EXT(d)):
New.
* tests/calc.at: Initial support for D.
* tests/headers.at
* examples/d/calc.y: Exit with failure on errors.
Remove useless operators (=, !) meant for the test suite.
Add unary + for symmetry.
* examples/d/calc.test: Adjust expectations.
Commit db34f79889 renames the variable F
as goto_follows, but forgot to rename this function.
* src/lalr.c (initialize_F): Rename as...
(initialize_goto_follows): this.
I need to be able to read includes and goto_follows.
* src/relation.h, src/relation.c (relation_print): Provide a means to
pretty-print the nodes of the relation.
* src/lalr.c (goto_print, follows_print): New.
(set_goto_map): Use goto_print.
(build_relations): Show INCLUDES.
(compute_FOLLOWS): Rename as...
(compute_follows): this.
Show FOLLOWS.
* src/lalr.c (build_relations): Reduce the scopes.
Instead of keeping rp alive in two different loops, clarify the second
one by having an index on the path we traverse (i.e., use that index
to compute the source state _and_ the symbol that labels the
transition).
This allows to turn an obscure 'while'-loop in a clearer (IMHO)
'for'-loop. We also consume more variables (by introducing p instead
of making more side effects on length), but we're in 2019, I don't
think this matters. What does matter is that (IMHO again), this is
now clearer.
Also, use clearer names.
* tests/local.at (AT_MAIN_DEFINE(java)): Exit failure on failure.
(AT_PARSER_CHECK): If in Java, run AT_JAVA_PARSER_CHECK.
* tests/conflicts.at (AT_CONSISTENT_ERRORS_CHECK): Simplify.
Currently the caller must specify the ./ prefix to its command. Let's
avoid that: it will be nicer to read, make it easier to have a version
that works for Java and C/C++.
* tests/local.at (AT_PARSER_CHECK): Prefix the command with ./.
Adjust callers.
* tests/conflicts.at (AT_YYLEX_PROTOTYPE): Don't define it, leave that
task to AT_DATA_GRAMMAR.
But be honest: tell AT_BISON_OPTION_PUSHDEFS all the options we use.
Reported by Hans Åberg.
https://lists.gnu.org/archive/html/help-bison/2019-02/msg00064.html
* src/files.c (spec_graph_file): Use `*.gv` when 3.4 or better,
otherwise `*.dot`.
* src/parse-gram.y (handle_require): Pretend we are already 3.4.
* doc/bison.texi: Adjust.
* tests/local.at, tests/output.at: Exercise this.
Currently, we have errors:
./doc/bison.texi:13005: node `History' lacks menu item for `Yacc' despite being its Up target
* doc/bison.texi (History): Add the local menu.
While at it, add a few index entries.
* examples/c/lexcalc/parse.y: Formatting/comment changes.
(line): Don't return a value.
Print the result here, which avoids printing a value for lines with an
error.
(yyerror): Be sure to increment the pointed, not the pointer...
* examples/c/lexcalc/lexcalc.test: Check errors.
* examples/c/lexcalc/local.mk: Fix a dependency.
While hacking on the computation of the automaton, I had yystate being
equal to -1, and the parser loops. Let's catch this when
parser.assert is enabled.
* data/skeletons/yacc.c (YY_ASSERT): New.
Use it.
Not using the name YYASSERT, to make it clear that this is private.
glr.c should probably move to YY_ASSERT too.
Also, while at it, report 'Entering state...' even before growing the
stacks.
* examples/c/calc/local.mk, examples/c/lexcalc/local.mk,
* examples/c/mfcalc/local.mk, examples/c/rpcalc/local.mk:
Regenerate the files if dependencies have changed.
I don't think this style:
/* If buffer is full, make it bigger. */
if (i == length)
{
length *= 2;
symbuf = (char *) realloc (symbuf, length + 1);
}
/* Add this character to the buffer. */
symbuf[i++] = c;
/* Get another character. */
c = getchar ();
or more readable than
/* If buffer is full, make it bigger. */
if (i == length)
{
length *= 2;
symbuf = (char *) realloc (symbuf, length + 1);
}
/* Add this character to the buffer. */
symbuf[i++] = c;
/* Get another character. */
c = getchar ();
Actually, I think the latter is more readable, and helps with width
issues in the PDF. As a matter of fact, I would happily move to //
only for single line comments.
* doc/bison.texi: A single space before closing comments.
* doc/bison.texi: Prefer 'fun' to 'fnct'.
Reduce local variable scopes.
Prefer strdup to malloc + strcpy.
Avoid gratuitous casts.
Use simpler names (e.g., 'name' instead of 'fname').
Avoid uses of 0 for NULL.
Avoid using NULL when possible (e.g., 'p' instead of 'p != NULL').
Prefer union names to casts (e.g. 'yylval.VAR = s' instead of
'*((symrec**) &yylval) = s').
Give arguments a name in fun declarations.
Use our typedefs instead of duplicating them (func_t).
Stop promoting an explicit $$ = $1;, it should be implicit (Bison
might be able to eliminate useless chain rules).
Help a bit Texinfo by making smaller groups.
Rely on the C compiler to call function pointers (prefer
'$1->value.fun ($3)' to (*($1->value.fnctptr))($3)').
Currently we have no simple example: rpcalc in reverse Polish, mfcalc
has functions, and lexcalc is using lex.
* examples/c/calc/Makefile, examples/c/calc/calc.y,
* examples/c/calc/calc.test, examples/c/calc/local.mk: New.
* examples/extexi: Since we issue #lines only at the beginning of
@example, leave empty line when removing content (such as @comment
lines), otherwise the lines that follow have incorrect source line
location. This leaves ugly empty lines, but they are removed when you
tidy the output for the end user: sequences of \n are mapped to at
most two sucessive \n.
The format is inconsistent. For instance most sections are
indented (including "Terminals unused in grammar" for instance), but
the sections "Terminals, with rules where they appear" and
"Nonterminals, with rules where they appear" are not. Let's indent
them. Also, these two sections try to wrap the output to avoid lines
too long. Yet we don't do that in the rest of the file, for instance
when listing the lookaheads of an item.
For instance in the case of Bison's parse-gram.output we go from:
Terminals, with rules where they appear
"end of file" (0) 0
error (256) 28 88
"string" <char*> (258) 9 13 16 17 20 23 24 109 116
[...]
Nonterminals, with rules where they appear
$accept (58)
on left: 0
input (59)
on left: 1, on right: 0
prologue_declarations (60)
on left: 2 3, on right: 1 3
prologue_declaration (61)
on left: 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24
25 26 27 28 29, on right: 3
[...]
to
Terminals, with rules where they appear
"end of file" (0) 0
error (256) 28 88
"string" <char*> (258) 9 13 16 17 20 23 24 109 116
[...]
Nonterminals, with rules where they appear
$accept (58)
on left: 0
input (59)
on left: 1
on right: 0
prologue_declarations (60)
on left: 2 3
on right: 1 3
prologue_declaration (61)
on left: 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24 25 26 27 28 29
on right: 3
[...]
* src/print.c (END_TEST): Remove.
(print_terminal_symbols): Don't try to wrap the output.
(print_nonterminal_symbols): Likewise.
Make two different lines for occurrences on the left, and occurrence
on the rhs of the rules.
Indent by 4 and 8, not 3.
* src/reduce.c (reduce_output): Indent by 4, not 3.
* tests/conflicts.at, tests/existing.at, tests/reduce.at,
* tests/regression.at, tests/report.at:
Adjust.
This should not be used to generate parsers. My point is actually to
facilitate debugging (when tweaking the generation of the LR(0)
automaton for instance, not carying -yet- about lookaheads).
* src/reader.c (prepare_percent_define_front_end_variables): Add lr(0).
* src/conflicts.c (set_conflicts): Be robust to reds not having
lookaheads at all.
* src/ielr.c (LrType, lr_type_get): Adjust.
(ielr): Implement support for LR(0).
* src/lalr.c (lalr_free): Don't free LA when it's not computed.
* maint:
maint: post-release administrivia
version 3.3.2
style: minor fixes
NEWS: named constructors are preferable to symbol_type ctors
gram: fix handling of nterms in actions when some are unused
style: rename local variable
CI: update the ICC serial number for travis-ci.org
Since Bison 3.3, semantic values in rule actions (i.e., '$...') are
passed to the m4 backend as the symbol number. Unfortunately, when
there are unused symbols, the symbols are renumbered _after_ the
numbers were used in the rule actions. As a result, the evaluation of
the skeleton failed because it used non existing symbol numbers.
Which is the happy scenario: we could use numbers of other existing
symbols...
Reported by Balázs Scheidler.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00044.html
Translating the rule actions after the symbol renumbering moves too
many parts in bison. Relying on the symbol identifiers is more
troublesome than it might first seem: some don't have an
identifier (tokens with only a literal string), some might have a
complex one (tokens with a literal string with characters special for
M4). Well, these are tokens, but nterms also have issues: "dummy"
nterms (for midrule actions) are named $@32 etc. which is risky for
M4.
Instead, let's simply give M4 the mapping between the old numbers and
the new ones. To avoid confusion between old and new numbers, always
emit pre-renumbering numbers as "orig NUM".
* data/README: Give details about "orig NUM".
* data/skeletons/bison.m4 (__b4_symbol, _b4_symbol): Resolve the
"orig NUM".
* src/output.c (prepare_symbol_definitions): Pass nterm_map to m4.
* src/reduce.h, src/reduce.c (nterm_map): Extract it from
nonterminals_reduce, to make it public.
(reduce_free): Free it.
* src/scan-code.l (handle_action_dollar): When referring to a nterm,
use "orig NUM".
* tests/reduce.at (Useless Parts): New, based Balázs Scheidler's
report.
A rule is a useless chain iff it's a chain (aka unit, or injection)
rule (i.e., the RHS has length 1), and it's useless (it has no used
defined semantic action).
* src/gram.h, src/gram.c (rule_useless_chain_p): New.
(grammar_dump): Report useless chain rules.
* tests/sets.at: Check the traces.
There are several places where we need to recover the rule from an
item, let's factor that into item_rule. We also want to print items
in a nice way: we do it when generating the *output file, but it is
also useful in debug messages.
* src/gram.h, src/gram.c (item_rule, item_print): New.
* src/print.c (print_core): Use them.
* src/state.h, src/state.c: Propagate constness.
The memory allocated by 'closure' (and some data such as 'fderives')
is used to computed a state's full itemset from its core. This is
needed during the construction of the LR(0) automaton, and the memory
is reclaimed immediately afterwards.
Unfortunately the reports (graph, text, xml) also need this
information when describing the states with their full itemsets. As a
consequence the memory was allocated again, fderives computed again
too, and more --trace reports are generated which only duplicate what
was already reported.
Stop that. It does mean that we release the memory later (hence the
peak memory usage is higher now), but I don't think that's a problem
today.
* src/lr0.c (generate_states): Don't call closure_free.
* src/state.c (states_free): Do it here.
(for symmetry with closure_new which is called in generate_states).
* src/print-graph.c, src/print-xml.c, src/print.c: You can now expect
the closure module to be functional.
This is more consistent with the other files.
* closure.h, closure.c (new_closure, free_closure): Rename as...
(closure_new, closure_free): this.
Adjust dependencies.
This will make it easier to add new elements (that might already be
part of shift_symbol) without having to worry about the size of
shift_symbol (which is currently a fixed size vector).
I could not measure any significant differences in performances in the
generation of LR(0) automaton (benched on gramamrs of Ruby, C, and C++).
* src/lr0.c (shift_symbol): Make it a bitset.
When debugging Bison itself, this is very handy, especially when
tweaking the frontend badly enough to break the backends. It can also
be used to check a grammar.
* src/getargs.h, src/getargs.c (feature_syntax_only): New.
(feature_args, feature_types): Adjust.
* src/main.c (main): Use it.
* src/symtab.h (status): Rename as...
(declaration_status): this, to avoid colliding with status, the
argument of 'usage'.
'status' seems a tad too general to be used only here.
* maint:
maint: post-release administrivia
version 3.3.1
yacc: issue warnings, not errors, for Bison extensions
style: formatting changes in NEWS and complain.c
tests: don't depend on the user's definition of SHELL
* src/gram.c (grammar_dump): Print the effective number first instead
of last. And fix it (remove the incorrect "+1").
Use t/f for Booleans.
* src/reduce.c: When asked, always print the reduced grammar, even if
there was nothing useless.
* tests/sets.at (Reduced Grammar): Check that.
It is inconvenient that we also generate the output files when we
update the grammar file, and it's somewhat unexpected. Let's not do
that.
* src/main.c (main): Skip generation when --update is passed.
* src/getargs.c (usage): Update the help message.
* doc/bison.texi (Bison Options): Likewise.
* tests/input.at: Check that we don't generate the output.
After all, this is clearly harmless.
* src/muscle-tab.c (muscle_percent_define_insert): Let equal
definitions of a %define variable be only a warning.
Adjust test cases.
* tests/input.at ("%define" backward compatibility): Don't define
twice "api.namespace", so that we don't get an error, which stops the
process too soon to see an error about the value given to
'lr.keep-unreachable-state'.
Don't repeat the name of the warning in the sub messages. E.g.,
remove the second "[-Wother]" in the following message
foo.y:2.1-27: warning: %define variable 'parse.error' redefined [-Wother]
%define parse.error verbose
^~~~~~~~~~~~~~~~~~~~~~~~~~~
foo.y:1.1-27: previous definition [-Wother]
%define parse.error verbose
^~~~~~~~~~~~~~~~~~~~~~~~~~~
* src/complain.c (error_message): Don't print the warning type when
it's indented.
Adjust test cases.
Reported by Derek Clegg
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00021.html
aux/parser-internal.h:429:12: error: 'syntax_error' has no out-of-line virtual
method definitions; its vtable will be emitted in every translation unit
[-Werror,-Wweak-vtables]
struct syntax_error : std::runtime_error
To avoid this warning, we need syntax_error to have a virtual function
defined in a compilation unit. Let it be the destructor. To comply
with C++98, this dtor should be 'throw()'. Merely making YY_NOEXCEPT
be 'throw()' in C++98 triggers
errors (http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00022.html),
so let's introduce YY_NOTHROW and flag only ~syntax_error with it.
Also, since we now have an explicit dtor, we need to provide an copy
ctor.
* configure.ac (warn_cxx): Add -Wweak-vtables.
* data/skeletons/c++.m4 (YY_NOTHROW): New.
(syntax_error): Declare the dtor, and define the copy ctor.
* data/skeletons/glr.cc, data/skeletons/lalr1.cc (~syntax_error):
Define.
Some reported by syntax-check.
* po/POTFILES.in: Add fixits.cc.
* src/muscle-tab.c: Don't cast for free.
* src/files.c: Reduce scopes.
* cfg.mk: We need the cast for free in muscle_percent_define_insert.
* doc/bison.texi: Use @kbd where appropriate.
Update ^~~~ marks for caret-errors.
* build-aux/cross-options.pl: Do not add quotes to %define's argument.
"handle_" is the prefix used in scan-code.l for instance.
* src/parse-gram.y (do_error_verbose, do_name_prefix, do_require)
(do_skeleton, do_yacc):
Rename as...
(handle_error_verbose, handle_name_prefix, handle_require)
(handle_skeleton, handle_yacc):
these.
We should use -ffixit and --update to clean files with duplicate
directives. And we should complain only once about duplicate obsolete
directives: keep only the "duplicate" warning. Let's start with %yacc.
For instance on:
%fixed-output_files
%fixed-output-files
%yacc
%%
exp:
This run of bison:
$ bison /tmp/foo.y -u
foo.y:1.1-19: warning: deprecated directive, use '%fixed-output-files' [-Wdeprecated]
%fixed-output_files
^~~~~~~~~~~~~~~~~~~
foo.y:2.1-19: warning: duplicate directive [-Wother]
%fixed-output-files
^~~~~~~~~~~~~~~~~~~
foo.y:1.1-19: previous declaration
%fixed-output_files
^~~~~~~~~~~~~~~~~~~
foo.y:3.1-5: warning: duplicate directive [-Wother]
%yacc
^~~~~
foo.y:1.1-19: previous declaration
%fixed-output_files
^~~~~~~~~~~~~~~~~~~
bison: file 'foo.y' was updated (backup: 'foo.y~')
gives:
%fixed-output-files
%%
exp:
* src/location.h, src/location.c (location_empty): New.
* src/complain.h, src/complain.c (duplicate_directive): New.
* src/getargs.h, src/getargs.c (yacc_flag): Instead of a Boolean, be
the location of the definition.
Update dependencies.
* src/scan-gram.l (%yacc, %fixed-output-files): Move the handling of
its warnings to...
* src/parse-gram.y (do_yacc): This new function.
* tests/input.at (Deprecated Directives): Adjust expectations.
* src/fixits.h, src/fixits.c (fixits_empty): New.
* src/complain.c (deprecated_directive): Register the Wdeprecated
fixits only if -Wdeprecated was enabled, so that we don't apply
updates if the user didn't ask for them.
* src/main.c (main): If there were fixits, issue a warning suggesting
running with --update.
Free uniqstrs after the fixits, since the latter use the former.
* tests/headers.at, tests/input.at: Update expectations.
* src/fixits.c (fixits_run): If erase the content of a line, also
erase the following \n.
* tests/input.at (Deprecated directives): Update expectations.
Reported by Derek Clegg.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00004.html
* configure.ac (warn_common): Add -Wimplicit-fallthrough.
This does trigger failures in the test suite.
* data/skeletons/glr.c, data/skeletons/lalr1.cc,
* data/skeletons/yacc.c, tests/c++.at:
Make fall-throws explicit.
Reported by Derek Clegg.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00006.html
Clang does not like this:
template <typename D>
struct basic_symbol : D
{
basic_symbol();
};
struct by_type {};
struct symbol_type : basic_symbol<by_type>
{
symbol_type(){}
};
It gives:
$ clang++-mp-7.0 -Wundefined-func-template foo.cc -c
foo.cc:11:3: warning: instantiation of function 'basic_symbol<by_type>::basic_symbol'
required here, but no definition is available [-Wundefined-func-template]
symbol_type(){}
^
foo.cc:4:3: note: forward declaration of template entity is here
basic_symbol();
^
foo.cc:11:3: note: add an explicit instantiation declaration to suppress this warning
if 'basic_symbol<by_type>::basic_symbol' is explicitly instantiated in
another translation unit
symbol_type(){}
^
1 warning generated.
The same applies for the basic_symbol's destructor and `clear()`.
* configure.ac (warn_cxx): Add -Wundefined-func-template.
This triggered one failure in the test suite:
* tests/headers.at (Sane headers): here, where we check that we can
compile the generated headers in other compilation units than the
parser's.
Add a variant type to make sure that basic_symbol and symbol_type are
properly generated in this case.
* data/skeletons/c++.m4 (basic_symbol): Inline the definitions of the
destructor and of `clear` in the class definition.
Avoid duplicate warnings about %error-verbose, once for deprecation,
another for duplicate. Keep only the duplicate warning for the second
occurrence of %error-verbose.
This will help removal fixits.
* src/scan-gram.l (%error-verbose): Return as a PERCENT_ERROR_VERBOSE
token.
* src/parse-gram.y (do_error_verbose): New.
Use it.
* src/muscle-tab.c (muscle_percent_variable_update): Handle pseudo
variables such as %error-verbose.
Currently, on
%define parser_class_name "Parser"
%define parser_class_name "Parser"
%%
exp:;
we issue:
foo.y:1.9-25: warning: deprecated directive, use '%define api.parser.class {Parser}' [-Wdeprecated]
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
foo.y:2.9-25: warning: deprecated directive, use '%define api.parser.class {Parser}' [-Wdeprecated]
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
foo.y:2.9-25: error: %define variable 'api.parser.class' redefined
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
foo.y:1.9-25: previous definition
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
Let's get rid of the second warning about the deprecated variable
parser_class_name. This is noise, but it will also be a problem with
fixits for removing duplicates, as we will first generate the update,
and then it's too late to remove it: fixits do not edit the result of
previous fixits.
So generate this instead:
foo.y:1.1-34: warning: deprecated directive, use '%define api.parser.class {Parser}' [-Wdeprecated]
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
foo.y:2.1-34: error: %define variable 'api.parser.class' redefined
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
foo.y:1.1-34: previous definition
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* src/muscle-tab.c (muscle_percent_variable_update): Pass the warning
to the caller, instead of issuing it.
(muscle_percent_define_insert): Issue this warning only if we don't
have to complain about a duplicate definition.
* tests/input.at: Adjust expectations.
* src/parse-gram.y: Use the location of the whole definition to record
the location of a %define variable, instead of just the name of the
variable.
Adjust tests.
Introduce proper support for fixits, instead of just printing them on
demand.
* bootstrap.conf: We need gnulib's xlists.
* src/fixits.h, src/fixits.c: New.
* src/complain.c (deprecated_directive): Use fixits_register.
* src/main.c (main): Use fixits_free.
Currently the diagnostics for %name-prefix are not precise enough. In
particular, they does not show that braces must be used instead of
quotes.
Before:
foo.y:3.1-14: warning: deprecated directive, use '%define api.prefix' [-Wdeprecated]
%name-prefix = "foo"
^^^^^^^^^^^^^^
After:
foo.y:3.1-20: warning: deprecated directive, use '%define api.prefix {foo}' [-Wdeprecated]
%name-prefix = "foo"
^^^^^^^^^^^^^^^^^^^^
To do this we need the value passed to %name-prefix, so move the
warning from the scanner to the parser.
Accuracy will be very important for the forthcoming changes.
* src/parse-gram.y (do_name_prefix): New.
(PERCENT_NAME_PREFIX): Have a semantic value: the raw source, with
possibly underscores, equal sign, and spaces. This is used to provide
a more accurate message. It does not take comments into account,
but...
* src/scan-gram.l (%name-prefix): Delegate the warnings to the parser.
* tests/headers.at, tests/input.at: Adjust expectations.
* src/muscle-tab.c (muscle_percent_variable_update): Avoid allocating
memory when it is not needed, which should be most of the time (when
there's no update to perform).
Adjust callers.
This line:
slice<stack_symbol_type, stack_type> slice (yystack_, yylen);
triggers warnings:
parse.h:1790:11: note: shadowed declaration is here
Reported by Frank Heckenbach.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00002.html
* configure.ac (warn_c): Move -Wshadow to...
(warn_common): here.
* data/skeletons/stack.hh (slice): Define as an inner class of stack.
* data/skeletons/lalr1.cc: Adjust.
Rename the variable as 'range' instead of 'slice'.
There are many macros that are defined and used just
once (b4_public_if, b4_abstract_if, etc.). That's overkill. Rather,
let's define a macro to build the "public class YYParser" line.
It appears that the same syntax with "extends", "abstract", etc. is
implemented in the D parser, which looks very fishy...
* data/skeletons/d.m4, data/skeletons/java.m4 (b4_public_if)
(b4_abstract_if, b4_final_if, b4_strictfp_if): Replace with
(b4_parser_class_declaration): this.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java: Adjust.
Commit 90a8537e62 was right, but issued
two error messages. Commit 80ef7e7639
tried to address that by mapping yychar and yytoken to empty, but that
completely breaks the invariants of glr.c. In particular, yygetToken
can be called repeatedly and is expected to return the latest result,
unless yytoken is YYEMPTY. Since the previous attempt was "recording"
that the token was coming from an exception by setting it to YYEMPTY,
instead of getting again the faulty token, we fetched another one.
Rather, revert to the first approach: map yytoken to "invalid token",
but record in yychar the fact that we come from an exception thrown in
the scanner.
* data/skeletons/glr.c (YYFAULTYTOK): New.
(yygetToken): Use it to record syntax errors from the scanner.
* tests/c++.at (Syntax error as exception): In addition to checking
syntax_error with error recovery, make sure it also behaves as
expected without.
The previous name was historical and inconsistent.
* src/muscle-tab.c (define_directive): Use the proper value passing
syntax, based on the muscle kind.
(muscle_percent_variable_update): Use the right value passing syntax.
Migrate from parser_class_name to api.parser.class.
* data/skeletons: Migrate from parser_class_name to api.parser.class.
* doc/bison.texi (%define Summary): Document both parser_class_name
and api.parser.class.
Promote the latter over the former.
This is very debatable. This function is not pure at all, so it could
stick to returning void: that's a common coding style to tell the
difference between "real" (pure) functions and side-effecting
subroutines. However, we already have this style elsewhere (e.g.,
yylex), and I feel the callers are somewhat nice to read this way.
* data/skeletons/glr.c (yygetLRActions): Return the action rather than
passing by pointer.
While at it, fix type of yytoken.
Adjust callers.
Reported by Askar Safin.
https://lists.gnu.org/archive/html/bison-patches/2019-01/msg00000.html
* data/skeletons/glr.c (yygetToken): Return YYEMPTY when an exception
is thrown.
* data/skeletons/lalr1.cc: Log when an exception is caught.
* tests/c++.at (Syntax error as exception): Be sure to recover from
error before triggering another error.
This way, it is easier to make sure its implementation is available in
glr.cc too, which is not the case currently.
* data/skeletons/c++.m4 (b4_public_types_define): Move the
implementation of syntax_error...
(b4_public_types_declare): here.
* tests/local.at (AT_LOCATION_TYPE_IF): Turn into...
(AT_LOCATION_TYPE_SPAN_IF): this.
Adjust dependencies.
* tests/headers.at (Several parsers): Add another C++ parser,
which uses the first C++ parser's locations.
We used to create a short definition of yy::parser with all the
implementations of its member functions outside. But yy::parser is no
longer short and simple to read. Maintaining each function twice is
painful: a lot of redundancy but different indentation levels, output
which depends on whether we are in a header or not (see
d132c2d545), etc.
Let's simplify this and put the implementations into the class
definition itself.
Discussed in this monologue:
https://lists.gnu.org/archive/html/bison-patches/2018-12/msg00058.html.
* data/skeletons/c++.m4, data/skeletons/lalr1.cc,
* data/skeletons/variant.hh (b4_basic_symbol_constructor_define)
(_b4_token_constructor_declare, b4_token_constructor_declare)
Merge into...
(b4_basic_symbol_constructor_define, _b4_token_constructor_define)
(b4_token_constructor_define): these.
Commit 112ccb5ed7 moved the skeletons
from dist_pkgdata_DATA to dist_skeletons_DATA, hence broke the dependencies.
* Makefile.am (dependencies): New.
Use it where appropriate.
We used to define such auxiliary structures outside the class, mainly
as a matter of style to keep the definition of yy::parser short and
simple. However, now there's a lot more code generated inside the
class definition (e.g., all the token constructors), so the
readability no longer applies.
However, if we move stack (and slice) inside yy::parser, then it
should no longer be needed to change the namespace to have multiple
parsers: changing the class name should suffice.
One common argument against inner classes is that they code bloat. It
hardly applies here, since typically different parsers will have
different semantic value types, hence different actual stack types.
* data/skeletons/lalr1.cc: Invoke b4_stack_define inside yy::parser.
Suggested by David Barto
https://lists.gnu.org/archive/html/help-bison/2015-02/msg00004.html
and Victor Zverovich.
https://lists.gnu.org/archive/html/bison-patches/2018-10/msg00121.html
This is very easy to do, thanks to work by Bruno Haible in gnulib.
See "Supporting Relocation" in gnulib's documentation.
* bootstrap.conf: We need relocatable-prog and relocatable-script (for yacc).
* src/yacc.in: New.
* configure.ac, src/local.mk: Instantiate it.
* src/main.c, src/output.c (main, pkgdatadir): Use relocatable2.
* doc/bison.texi (FAQ): Document it.
Suggested by Wolfgang Thaller.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00081.html
* data/c++.m4 (basic_symbol, by_type): Instead of provide either move
or copy constructor, always provide the copy one.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Check it.
Currently the following piece of code crashes (with parse.assert),
because we don't record that s was moved-from, and we invoke its dtor.
{
auto s = parser::make_INT (42);
auto s2 = std::move (s);
}
Reported by Wolfgang Thaller.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00077.html
* data/c++.m4 (by_type): Provide a move-ctor.
(basic_symbol): Be sure not to read a moved-from value.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Check this case.
* tests/c++.at: Use AT_YYLEX_PROTOTYPE etc.
Which requires that we use the same argument names (lvalp, etc.).
* tests/local.at (AT_NAME_PREFIX): Fix regex.
Instead of introducing make_symbol (whose name, btw, somewhat
infringes on the user's "name space", if she defines a token named
"symbol"), let's make the construction of symbol_type safer, using
assertions.
For instance with:
%token ':' <std::string> ID <int> INT;
generate:
symbol_type (int token, const std::string&);
symbol_type (int token, const int&);
symbol_type (int token);
It does mean that now named token constructors (make_ID, make_INT,
etc.) go through a useless assert, but I think we can ignore this: I
assume any decent compiler will inline the symbol_type ctor inside the
make_TOKEN functions, which will show that the assert is trivially
verified, hence I expect no code will be emitted for it. And anyway,
that's an assert, NDEBUG controls it.
* data/c++.m4 (symbol_type): Turn into a subclass of
basic_symbol<by_type>.
Declare symbol constructors when variants are enabled.
* data/variant.hh (_b4_type_constructor_declare)
(_b4_type_constructor_define): Replace with...
(_b4_symbol_constructor_declare, _b4_symbol_constructor_def): these.
Generate symbol_type constructors.
* doc/bison.texi (Complete Symbols): Document.
* tests/types.at: Check.
On
%token <int> FOO BAR
we currently generate make_FOO(int) and make_BAR(int). However, in
order to factor their scanners, some users would also like to have
make_symbol(tok, int), where tok is FOO or BAR. To ensure type
safety, add assertions that do check that value type and token type
match. Bind this assertion to the parse.assert %define variable.
Suggested by Frank Heckenbach.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00034.html
Should also match expectations from Аскар Сафин.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00023.html
* data/variant.hh: Use b4_token_visible_if where applicable.
(_b4_type_constructor_declare, _b4_type_constructor_define): New.
Use them.
Bitten by macros, again.
See 680b715518.
* data/variant.hh (_b4_symbol_constructor_declare)
(_b4_symbol_constructor_define): Do not use user types, which can
include commas as in `std::pair<int, int>`, to macros.
* tests/local.at: Adjust the lex related macros to support the
case of token constructors.
* tests/types.at: Also check token constructors on types with commas.
Prompted by Rici Lake.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00000.html
We have four classes of directives that declare symbols: %nterm,
%type, %token, and the family of %left etc. Currently not all of them
support the possibility to have several type tags (`<type>`), and not
all of them support the fact of not having any type tag at all
(%type). Let's unify this.
- %type
POSIX Yacc specifies that %type is for nonterminals only. However,
some Bison users want to use it for both tokens and nterms
(actually, Bison's own grammar does this in several places, e.g.,
CHAR). So it should accept char/string literals.
As a consequence cannot be used to declare tokens with their alias:
`%type foo "foo"` would be ambiguous (are we defining foo = "foo",
or are these two different symbols?)
POSIX specifies that it is OK to use %type without a type tag. I'm
not sure what it means, but we support it.
- %token
Accept token declarations with number and string literal:
(ID|CHAR) NUM? STRING?.
- %left, etc.
They cannot be the same as %token, because we accept to declare the
symbol with %token, and to then qualify its precedence with %left.
Then `%left foo "foo"` would also be ambiguous: foo="foo", or two
symbols.
They cannot be simply a list of identifiers, but POSIX Yacc says we
can declare token numbers here. I personally think this is a bad
idea, precedence management is tricky in itself and should not be
cluttered with token declaration issues.
We used to accept declaring a token number on a string literal here
(e.g., `%left "token" 1`). This is abnormal. Either the feature is
useful, and then it should be supported in %token, or it's useless
and we should not support it in corner cases.
- %nterm
Obviously cannot accept tokens, nor char/string literals. Does not
exist in POSIX Yacc, but since %type also works for terminals, it is
a nice option to have.
* src/parse-gram.y: Avoid relying on side effects. For instance, get
rid of current_type, rather, build the list of symbols and iterate
over it to assign the type.
It's not always possible/convenient. For instance, we still use
current_class.
Prefer "decl" to "def", since in the rest of the implementation we
actually "declare" symbols, we don't "define" them.
(token_decls, token_decls_for_prec, symbol_decls, nterm_decls): New.
Use them for %token, %left, %type and %nterm.
* src/symlist.h, src/symlist.c (symbol_list_type_set): New.
* tests/regression.at b/tests/regression.at
(Token number in precedence declaration): We no longer accept
to give a number to string literals.
After having spent quite some time on cleaning the handling of symbol
declarations in the grammar files, I believe we should keep it.
It looks like it's a duplicate of %type, but it is not. While POSIX
Yacc requires %type to apply only to nonterminal symbols, it appears
that both byacc and bison accept it for tokens too. And some
experienced users do actually expect this feature to group
symbols (terminal or not) by type ("On the other hand, it is generally
more useful IMHO to group terminals and non-terminals with the same
type tag together",
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00000.html).
Even Bison's own parser does this today (see CHAR).
Basically reverts 7928c3e6fb.
* src/scan-gram.l (%nterm): Dedeprecate, but issue a Wyacc warning.
* tests/input.at: Adjust expectations.
(Yacc warnings on symbols): New.
* src/symtab.c (symbol_class_set): Fix error introduced in
20b0746793.
I personally prefer 'non terminal', or 'non-terminal', but
'nonterminal' is the common spelling.
* data/glr.c, src/parse-gram.y, src/symtab.c, src/symtab.h,
* tests/input.at, doc/refcard.tex: here.
Currently our error messages include both "symbol redeclared" and
"symbol redefined", and they mean something different. This is
obscure, let's make this clearer.
I think the idea between 'definition' vs. 'declaration' is that in the
case of the nonterminals, the actual definition is its set of rules,
so %nterm would be about declaration. The case of %token is less
clear.
* src/symtab.c (complain_class_redefined): New.
(symbol_class_set): Use it.
Simplify the logic of this function to clearly skip its body when the
preconditions are not met.
* tests/input.at (Symbol class redefinition): New.
Convert some of the READMEs to Markdown, which is now more common, and
nicely displayed in some git hosting services.
Add missing READMEs and Makefiles. Generate XML, HTML and Dot files. Be
sure to ship the test files. Complete CLEANFILES to remove all generated
files.
* examples/calc++: Move into...
* examples/c++: here.
* examples/mfcalc, examples/rpcalc: Move into...
* examples/c: here.
* examples/README.md, examples/c++/calc++/Makefile, examples/c/local.mk,
* examples/c/mfcalc/Makefile, examples/c/rpcalc/Makefile,
* examples/d/README.md, examples/java/README.md:
New files.
* examples/test (medir): Be robust to deeper directory nesting.
Revamping the handling of the symbols is the grammar is much more
delicate than I anticipated. Let's first move things around for
clarity.
* src/symtab.c (symbol_make_alias): Don't accept to alias
non-terminals.
(symbol_user_token_number_set): Don't accept user token numbers
for non-terminals.
Don't do anything in case of redefinition, instead of trying to
update. The flow is eaier to follow this way.
When we extract the examples from the documentation, %require
"@value{VERSION}" is replaced with the current version. If we change
the git branch, without changing the documentation, the generated
examples will %require a version of Bison that differs from the actual
version.
* examples/local.mk (extracted.stamp): Depend on doc/version.texi.
* data/README: Convert to Markdown.
Start documenting some of the macros used in all our skeletons.
Simplify and fix the documentation of the macros in the skeletons.
Currently it is the front end that passes the symbol types to the
backend. For instance:
%token <ival> NUM
%type <ival> exp1 exp2
exp1: NUM { $$ = $1; }
exp2: NUM { $<ival>$ = $<ival>1; }
In both cases, $$ and $1 are passed to the backend as having type
'ival' resulting in code like `val.ival`. This is troublesome in the
case of api.value.type=union, since in that the case the code this:
%define api.value.type union
%token <int> NUM
%type <int> exp1 exp2
exp1: NUM { $$ = $1; }
exp2: NUM { $<int>$ = $<int>1; }
because in this case, since the backend does not know the symbol being
processed, it is forced to generate casts in both cases: *(int*)(&val)`.
This is unfortunate in the first case (exp1) where there is no reason
at all to use a cast instead of `val.NUM` and `val.exp1`.
So instead delegate the computation of the actual value type to the
backend: pass $<ival>$ as `symbol-number, ival` and $$ as
`symbol-number, MULL`, instead of passing `ival` before.
* src/scan-code.l (handle_action_dollar): Find the symbol the action
is about, not just its tyye. Pass both symbol-number, and explicit
type tag ($<tag>n when there is one) to b4_lhs_value and b4_rhs_value.
* data/bison.m4 (b4_symbol_action): adjust to the new signature to
b4_dollar_pushdef.
* data/c-like.m4 (_b4_dollar_dollar, b4_dollar_pushdef): Accept the
symbol-number as new argument.
* data/c.m4 (b4_symbol_value): Accept the symbol-number as new
argument, and use it.
(b4_symbol_value_union): Accept the symbol-number as new
argument, and use it to prefer ready a union member rather than
casting the union.
* data/yacc.c (b4_lhs_value, b4_rhs_value): Accept the new
symbol-number argument.
Adjust uses of b4_dollar_pushdef.
* data/glr.c (b4_lhs_value, b4_rhs_value): Adjust.
* data/lalr1.cc (b4_symbol_value_template, b4_lhs_value): Adjust
to the new symbol-number argument.
* data/variant.hh (b4_symbol_value, b4_symbol_value_template): Accept
the new symbol-number argument.
* data/java.m4 (b4_symbol_value, b4_rhs_data): New.
(b4_rhs_value): Use them.
* data/lalr1.java: Adjust to b4_dollar_pushdef, and use b4_rhs_data.
* data/bison.m4, data/c++.m4, data/glr.c, data/java.m4, data/lalr1.cc,
* data/yacc.c, src/scan-code.l:
Fix comments.
Prefer POS to denote the position of a symbol in a rule, since NUM
is also used to denote symbol numbers.
Instead of defining yy::variant<S> and then alias
yy::parser::semantic_type to variant<sizeof (union_type)>, directly
define yy::parser::semantic_type.
This model is more appropriate if we want to sit the storage on top of
unions in C++11.
* data/variant.hh (b4_variant_define): Specialize and inline the
definition into...
(b4_value_type_declare): Here.
Define union_type here.
* data/lalr1.cc: Adjust.
* examples/java/Calc.y: New, based on test 495: "Calculator
parse.error=verbose %locations".
* examples/java/Calc.test, examples/java/local.mk: New.
* configure.ac (ENABLE_JAVA): New.
* examples/test (prog): Be ready to run Java programs.
This was demanded several times. See for instance:
- David M. Warme
https://lists.gnu.org/archive/html/help-bison/2011-04/msg00003.html
- box12009
http://lists.gnu.org/archive/html/bug-bison/2016-10/msg00001.html
Basically, this reverts:
- commit 3d3bc1fe30
Get rid of (yy)rhs and (yy)prhs
- commit d333175f63
Avoid compiler warning.
Note that since these tables are not needed in the generated parsers,
no skeleton requests them. This change only brings back their
definition to M4, making it possible to user-defined skeletons to use
these tables.
* src/output.c (muscle_insert_item_number_table): Define.
(prepare_rules): Generate the rhs and prhs tables.
Reported by Rici Lake.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00000.html
* src/complain.h: Formatting change.
* src/parse-gram.y (id): Reject character literals used in a context
for non-terminals.
* tests/input.at (Invalid %nterm uses): Check that.
Reported by Uxio Prego.
https://lists.gnu.org/archive/html/help-bison/2018-11/msg00031.html
We also need to move the unreachable 'goto' to a reachable place,
otherwise clang complains about the code being unreachable anyway.
See also https://bugs.llvm.org/show_bug.cgi?id=39736.
Interestingly, we don't have to apply that trick to
`#define YYCDEBUG if (false) std::cerr`, clang does not warn when the
code comes from macro expansion.
* configure.ac: Use -Wunreachable-code when supported.
* data/lalr1.cc, data/yacc.c: Pacify clang's warning about `if (0)`
by using a macro.
Another possibility was to move this statement to a reachable place.
* tests/actions.at, tests/c++.at: Avoid generating unreachable code.
We should probably introduce some struct and functions to deal with
stack management, rather than open coding it. yyparse would be much
nicer to read, and a better model for possible other skeletons.
* data/yacc.c (yyparse::yysetstate): Avoid generating code when
neither yyoverflow nor YYSTACK_RELOCATE is defined.
Currently on a grammar such as
exp : a '1' | a '2' | a '3' | b '1' | b '2' | b '3'
a:
b:
we count only one rr-conflict on the `b:` rule, i.e., we expect:
b: %expect-rr 1
although there are 3 conflicts in total. That's because in the
conflicted state we count only a single conflict, not three (one for
each of the lookaheads: '1', '2', '3').
State 0
0 $accept: . exp $end
1 exp: . a '1'
2 | . a '2'
3 | . a '3'
4 | . b '1'
5 | . b '2'
6 | . b '3'
7 a: . %empty ['1', '2', '3']
8 b: . %empty ['1', '2', '3']
'1' reduce using rule 7 (a)
'1' [reduce using rule 8 (b)]
'2' reduce using rule 7 (a)
'2' [reduce using rule 8 (b)]
'3' reduce using rule 7 (a)
'3' [reduce using rule 8 (b)]
$default reduce using rule 7 (a)
exp go to state 1
a go to state 2
b go to state 3
See https://lists.gnu.org/archive/html/bison-patches/2013-02/msg00106.html.
* src/conflicts.c (rule_has_state_rr_conflicts): Rename as...
(count_rule_state_sr_conflicts): this.
DWIM.
(count_rule_rr_conflicts): Adjust.
* tests/conflicts.at (%expect-rr in grammar rules)
(%expect-rr too much in grammar rules)
(%expect-rr not enough in grammar rules): New.
On a grammar such as
exp: "num" | "num" | "num"
we currently report only one RR conflict, instead of two.
This bug is present since the origins of Bison
commit 08089d5d35
Author: David MacKenzie <[email protected]>
Date: Tue Apr 20 05:42:52 1993 +0000
Initial revision
and was preserved in
commit 676385e29c
Author: Paul Hilfinger <[email protected]>
Date: Fri Jun 28 02:26:44 2002 +0000
Initial check-in introducing experimental GLR parsing. See entry in
ChangeLog dated 2002-06-27 from Paul Hilfinger for details.
See
https://lists.gnu.org/archive/html/bison-patches/2018-11/msg00011.html
* src/conflicts.h, src/conflicts.c (count_state_rr_conflicts)
(count_rr_conflicts): Use only the correct count of conflicts.
* tests/glr-regression.at: Fix expectations.
Currently on a grammar such as
exp: "number" | exp "+" exp | exp "*" exp
we count only one sr-conflict for both binary rules, i.e., we expect:
exp: "number" | exp "+" exp %expect 1 | exp "*" exp %expect 1
although there are 4 conflicts in total. That's because in the states
in conflict, for instance that for the "+" rule:
State 6
2 exp: exp . "+" exp
2 | exp "+" exp . [$end, "+", "*"]
3 | exp . "*" exp
"+" shift, and go to state 4
"*" shift, and go to state 5
"+" [reduce using rule 2 (exp)]
"*" [reduce using rule 2 (exp)]
$default reduce using rule 2 (exp)
we count only a single conflict, although there are two (one on "+"
and another with "*").
See https://lists.gnu.org/archive/html/bison-patches/2013-02/msg00106.html.
* src/conflicts.c (rule_has_state_sr_conflicts): Rename as...
(count_rule_state_sr_conflicts): this.
DWIM.
(count_rule_sr_conflicts): Adjust.
* tests/conflicts.at (%expect in grammar rules): New.
This change allows one to document (and check) which rules participate
in shift/reduce and reduce/reduce conflicts. This is particularly
important GLR parsers, where conflicts are a normal occurrence. For
example,
%glr-parser
%expect 1
%%
...
argument_list:
arguments %expect 1
| arguments ','
| %empty
;
arguments:
expression
| argument_list ',' expression
;
...
Looking at the output from -v, one can see that the shift-reduce
conflict here is due to the fact that the parser does not know whether
to reduce arguments to argument_list until it sees the token AFTER the
following ','. By marking the rule with %expect 1 (because there is a
conflict in one state), we document the source of the 1 overall shift-
reduce conflict.
In GLR parsers, we can use %expect-rr in a rule for reduce/reduce
conflicts. In this case, we mark each of the conflicting rules. For
example,
%glr-parser
%expect-rr 1
%%
stmt:
target_list '=' expr ';'
| expr_list ';'
;
target_list:
target
| target ',' target_list
;
target:
ID %expect-rr 1
;
expr_list:
expr
| expr ',' expr_list
;
expr:
ID %expect-rr 1
| ...
;
In a statement such as
x, y = 3, 4;
the parser must reduce x to a target or an expr, but does not know
which until it sees the '='. So we notate the two possible reductions
to indicate that each conflicts in one rule.
See https://lists.gnu.org/archive/html/bison-patches/2013-02/msg00105.html.
* doc/bison.texi (Suppressing Conflict Warnings): Document %expect,
%expect-rr in grammar rules.
* src/conflicts.c (count_state_rr_conflicts): Adjust comment.
(rule_has_state_sr_conflicts): New static function.
(count_rule_sr_conflicts): New static function.
(rule_nast_state_rr_conflicts): New static function.
(count_rule_rr_conflicts): New static function.
(rule_conflicts_print): New static function.
(conflicts_print): Also use rule_conflicts_print to report on individual
rules.
* src/gram.h (struct rule): Add new fields expected_sr_conflicts,
expected_rr_conflicts.
* src/reader.c (grammar_midrule_action): Transfer expected_sr_conflicts,
expected_rr_conflicts to new rule, and turn off in current_rule.
(grammar_current_rule_expect_sr): New function.
(grammar_current_rule_expect_rr): New function.
(packgram): Transfer expected_sr_conflicts, expected_rr_conflicts
to new rule.
* src/reader.h (grammar_current_rule_expect_sr): New function.
(grammar_current_rule_expect_rr): New function.
* src/symlist.c (symbol_list_sym_new): Initialize expected_sr_conflicts,
expected_rr_conflicts.
* src/symlist.h (struct symbol_list): Add new fields expected_sr_conflicts,
expected_rr_conflicts.
* tests/conflicts.at: Add tests "%expect in grammar rule not enough",
"%expect in grammar rule right.", "%expect in grammar rule too much."
We also lack a consistent naming for directive implementations.
`directive_skeleton` is too long, `percent_skeleton` is not very nice
looking, `process_skeleton` looks ambiguous, `do_skeleton` is somewhat
ambiguous too, but seems a better track.
* src/parse-gram.y (version_check): Rename as...
(do_require): this.
(do_skeleton): New.
Use it.
We may generate code such as
basic_symbol (typename Base::kind_type t, YY_RVREF (std::pair<int,int>) v);
which, of course, breaks, because YY_RVREF sees two arguments. Let's
not play tricks with _VA_ARGS__, I'm unsure about it portability.
Anyway, I plan to change more things in this area.
Reported by Sébastien Villemot.
http://lists.gnu.org/archive/html/bug-bison/2018-11/msg00014.html
* data/variant.hh (b4_basic_symbol_constructor_declare)
(b4_basic_symbol_constructor_define): Don't use macro on user types.
* tests/types.at: Check that we support pairs.
In C++, [[noreturn]] must not be between "static" and the rest of the
function signature, it must precede it. C's _Noreturn does not seem
to have such a constraint, but it is therefore compatible with the C++
constraint. Since we #define _Noreturn as [[noreturn]] is modern C++,
be sure to push the _Noreturn first.
Unfortunately this was not caught by the test suite, because it always
loads config.h first, and config.h contains another definition of
_Noreturn that does not use [[noreturn]], and hides ours. That's
probably a sign we should avoid always loading config.h.
* data/glr.c (yyFail, yyMemoryExhausted): here.
* tests/local.at (AT_LANG_FOR_EACH_STD): New.
(AT_REQUIRE_CXX_VERSION): Rename as...
(AT_REQUIRE_CXX_STD): this.
Accept an argument for what to do when the requirement is not met.
* tests/types.at (api.value.type): Check all the C++ stds.
The following commit introduce even more compilations/runs than
before, and with ASAN on, we go beyond to 50min credit from Travis.
* .travis.yml (Clang 7 libc++ and ASAN): Split in two.
It is unfortunate that %error_verbose was properly diagnosed as
obsoleted by "%define parse.error verbose", but %error-verbose was
not.
* src/parse-gram.y (%error-verbose): Remove support.
* src/scan-gram.l: Do it here instead, with a warning.
* tests/input.at (Deprecated directives): Check it.
These tests are skipped with GCC:
"\"".c:1:5: error: function declaration isn't a prototype [-Werror=strict-prototypes]
int main() { return 0; }
^~~~
* tests/synclines.at: Stop writing C++ in C.
* tests/local.at: Formatting changes.
* src/parse-gram.y (api.value.type): Set to union.
Replace occurrences of %union with explicit %types.
* src/scan-gram.l: Adjust yylval's field names.
(RETURN_VALUE): No longer needs the Field argument.
Use it more.
Support for DJGPP was announced to be removed in the NEWS of Bison
3.1 (2018-08-27) unless someone expressed interest. There was no answer.
* djgpp: Remove.
* NEWS, Makefile.am, cfg.mk, po/POTFILES.in: Adjust.
Reported by Andre da Costa Barros.
https://savannah.gnu.org/patch/?9716
* examples/calc++/local.mk: We no longer generate position.hh and
stack.hh. Leaving them here triggers their concurrent generation,
which fails.
(%C%_calc___CPPFLAGS): Fix the extracted headers in the source tree.
* examples/mfcalc/local.mk (%C%_mfcalc_CPPFLAGS): Ditto.
Currently, the examples are extracted on the user's side.
Unfortunately, that requires that the user has Perl, which is
otherwise not needed for Bison. Let's ship the examples instead.
The examples were handled this way so that we could depend on
configure flags: if --enable-gcc-warnings is passed, it is understood
as "I'm a maintainer", so the examples are generated with `#line`s.
Regular users should not see them, so they are now unconditionally
removed when rolling a tarball.
Reported by Mike Frysinger.
https://lists.gnu.org/archive/html/bison-patches/2015-04/msg00000.html
* examples/local.mk: Ship all the extracted files.
(examples-unline): New.
Make sure that the generated tarballs do not contain the #lines.
Also, make sure that `make dist` generates a correct tarball even if
the C++ compiler does not work.
Reported by Nelson H. F. Beebe.
* m4/cxx.m4 (BISON_CXX_WORKS): Define to true/false instead of
true/exit 77. The latter is too dangerous to use (it directly quits).
(ENABLE_CXX): New name for the Automake conditional, for consistency
with ENABLE_CXX11 etc.
* tests/local.at (AT_COMPILE, AT_COMPILE_CXX): Adjust to the new
semantics of BISON_CXX_WORKS.
* examples/c++/local.mk: Skip the variant test if C++ does not work.
* examples/calc++/local.mk: Likewise.
On some systems (x86_64-pc-solaris2.11), with Developer Studio 12.5's
CC, we get:
".../include/CC/Cstd/vector.cc", line 127: Error: Cannot assign const yy::parser::stack_symbol_type to yy::parser::stack_symbol_type without "yy::parser::stack_symbol_type::operator=(const yy::parser::stack_symbol_type&)";.
".../include/CC/Cstd/vector", line 475: Where: While instantiating "std::vector<yy::parser::stack_symbol_type>::__insert_aux(yy::parser::stack_symbol_type*, const yy::parser::stack_symbol_type&)".
".../include/CC/Cstd/vector", line 475: Where: Instantiated from non-template code.
1 Error(s) detected.
Don't expect __cplusplus to be always defined. If it's not, consider
this is C++98.
Reported by Nelson H. F. Beebe.
* data/c++.m4, data/lalr1.cc, examples/c++/variant.yy, tests/local.at,
* tests/testsuite.h:
An undefined __cplusplus means pre C++11.
On some systems, we don't use our getopt. As a consequence the error
messages vary:
$ bison --skeleton
bison: option requires an argument -- skeleton
Try 'bison --help' for more information.
instead of
bison: option '--skeleton' requires an argument
Try 'bison --help' for more information.
Reported by Jannick and Nelson H. F. Beebe.
https://lists.gnu.org/archive/html/bison-patches/2018-10/msg00140.html
* tests/input.at (Invalid number of arguments): work around getopt
portability issues.
On Solaris, GCC 7.3 defines:
-std=c++14 -std=c++17
__cplusplus 201402L 201703L
__STDC_VERSION__ 199901L 201112L
So the current #definition of _Noreturn sees that 201112 <=
__STDC_VERSION__, i.e., that C11 is supported, so it expects _Noreturn
to be supported. Apparently it is not.
This is only for C++, the test suite works for C. However, the test
suite does not try several C standards, maybe we should...
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00064.html
* data/c.m4 (b4_attribute_define): Define _Noreturn as [[noreturn]] in
modern C++.
Does not work on Solaris 11.3 x86/64:
479. c++.at:1293: testing C++ GLR parser identifier shadowing ...
======== Testing with C++ standard flags: '-std=c++17'
./c++.at:1332: $BISON_CXX_WORKS
stderr:
stdout:
./c++.at:1332: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o input input.cc $LIBS
stderr:
input.cc:837:8: error: '_Noreturn' does not name a type
static _Noreturn void
^~~~~~~~~
input.cc:845:8: error: '_Noreturn' does not name a type
static _Noreturn void
^~~~~~~~~
Reported by Kiyoshi Kanazawa.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00051.html
* data/c.m4 (b4_attribute_define): Use the snippet which is currently
in gnulib's m4/gnulib-common.m4 (which seems a little more advanced
than lib/_Noreturn.h).
The "Report translation bugs to..." part of --help is issued only on
glibc systems. So if the tarball is not wrapped on such a system, and
used on such a system (or the converse), then bison.help will differ
on the user's system, and help2man will be called to update bison.1.
But help2man should not be a requirement.
Reported by Alexandre Duret-Lutz.
* doc/local.mk (doc/bison.help): Remove the possible doc about
translation bugs.
Pass LC_ALL=C, as reported in src/getargs.c's usage().
(doc/cross-options.texi): Use bison.help instead of calling bison
--help.
Some users rely on this sentence to know that the file can be ignored.
Reported by Alexandre Duret-Lutz.
* data/bison.m4 (b4_generated_by): New.
(b4_copyright): Use it.
* data/location.cc, data/stack.hh: Use it too, for the stub files
(position.hh and stack.hh).
* doc/bison.texi (Calc++ Scanner): Show how exception can be thrown
from auxiliary functions.
Clarify the meaning of the various flex %options we use.
Get rid of a warning.
(Calc++ Parsing Driver): Use the parser as a functor.
* doc/bison.texi: Simplify wording.
Fix Texinfo error.
(Complete Symbols): Handle the token EOF.
(Calc++ Parser): In the modern C++ world, prefer assignment to swap.
(Strings are Destroyed): Prefer an explicit 'continue' to a comment.
The CI is using Flex 2.5.35. And ICC is too picky for it. Let's stop
making these warnings errors. I wish I could disable them in the
source files using the ICC version and the Flex version, but ICC's
pragma support is unclear, and I'm tired of fighting it.
* configure.ac (FLEX_SCANNER_CXXFLAGS): Make warnings warnings.
* examples/c++/local.mk: Comment changes.
Suggested by Victor Khomenko.
http://lists.gnu.org/archive/html/bug-bison/2018-08/msg00037.html
* doc/bison.texi (A Simple C++ Example): New.
* examples/c++/local.mk, examples/c++/simple.test: New.
Extract, check, and install this new example.
* examples/local.mk: Adjust.
* examples/test: Adjust to the case where the dirname differs
from the test name.
* build-aux/local.mk, cfg.mk, examples/calc++/local.mk,
* examples/local.mk, examples/mfcalc/local.mk,
* examples/rpcalc/local.mk, lib/local.mk, src/local.mk,
* tests/local.mk:
Use Automake comments so that we don't get a copy of each in the
generated Makefile.
On the CI, both GCC and Clang report:
src/parse-gram.c: In function 'yy_lac':
src/parse-gram.c:1479:29: error: format '%hd' expects argument of type 'int',
but argument 3 has type 'yytype_int16 {aka long int}' [-Werror=format=]
YYDPRINTF ((stderr, " G%hd", yystate));
^
Although yytype_int16 is supposed to be a short int, not a long int.
This must be explored.
* data/yacc.c (yy_lac): Work around typing issue.
* data/glr.c: Don't use `foo |= bar` with foo and bar being yybool:
the result appears to be an int, not a yybool.
Use yybool where appropriate.
Add casts where needed.
Reported by Derek Clegg.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00018.html
Rather than adding casts, we should be more careful with types. For
instance yystate should be a yytype_int16. But currently we can't: it
is also used sometimes for storing other things that state numbers.
* data/yacc.c (yyparse): Add missing casts.
The yacc.c skeleton is old, and was using many tricks to save
registers. Today's register allocators can do this themselves. Let's
keep the code simpler to read and let compilers do their job.
* data/yacc.c: Avoid using yystate for different types of content.
An inline function would be better, but doing this portably will be
a problem.
* data/lalr1.cc: Fix oldish comment.
* data/stack.hh: Prefer typename for type names.
Use size() instead of duplicating it.
* examples/variant-11.yy, examples/variant.yy (yylex): Use int,
as this is the type of the semantic value.
Currently we emit useless code for places where we might issue user
content, but there is none. This commit avoids this. Besides, some
of the comments looked like implementation details ("Copy the first
part of user declarations"), rather than made for the reader of the
result ("First part of user prologue").
On Bison's parse-gram.c we get:
@@ -76,10 +76,6 @@
#define yynerrs gram_nerrs
-/* Copy the first part of user declarations. */
-
-#line 82 "src/parse-gram.c" /* yacc.c:339 */
-
* data/bison.m4 (b4_define_user_code): Accept a comment to document
the section.
Do not emit any code if the content is empty.
Adjust callers to not emit the comment.
Do not
* data/glr.c, data/glr.cc, data/lalr1.cc, data/lalr1.java, data/yacc.c:
Adjust.
The previous name is too obscure, and the other macros for C++ use
CXX, not CC.
* tests/local.at (AT_SKEL_CC_IF, AT_SKEL_JAVA_IF): Rename as...
(AT_CXX_IF, AT_JAVA_IF): these.
Adjust callers.
When we introduced variants in Bison, C++ did not have the 'emplace'
functions, and we chose 'build'. Let's align with modern C++ and
promote 'emplace' rather than 'build'.
* data/lalr1.cc, data/variant.hh (emplace): New.
(build): Deprecate in favor of emplace.
* doc/bison.texi: Adjust.
As a matter of fact, I think it is wrong to gitignore generated files
that belong to the build tree. There should be the strict minimum,
and it's up to people that build in place to adjust their own
~/.gitignore.
* doc/.gitignore: here.
Remove files we no longer produce (thanks to texi2dvi).
In the previous commit we fixed a problem when the C++ stack was
resized. The test was using ints. Let's add a test with someone
quite touchy: unique_ptr
* examples/variant-11.yy: Accept an argument, which is the number of
numbers to send to the parser.
* examples/variant-11.test: Check with many numbers.
In some casing, once we moved a stack symbol, we forget to mark the
source stack symbol as emptied. As a consequence, it may be destroyed
a second time.
This happens when the stack has to be resized.
* data/lalr1.cc (stack_symbol_type::stack_symbol_type): Record that
the source was emptied.
(stack_symbol_type::operator=): Likewise.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Force the stack
to be resized. Check its content.
The files stack.hh and position.hh are deprecated. Rather than
devoting specify %define variables to discard them (api.position.file
and api.stack.file), and rather than having to use special rules when
api.location.file is used, let's simply decide that from %require
"3.2" onwards, these files will not be generated.
The only noticeable thing here is that, in order to be able to check
the behavior of %require "3.2", to have this version (which is still
3.1-*) to accept %require "3.2".
* src/gram.h, src/gram.c (required_version): New.
* src/parse-gram.y (version_check): Set it.
* src/output.c (prepare): Pass it m4.
* data/bison.m4 (b4_required_version_if): Receive it and use it.
* data/location.cc, data/stack.hh: Replace the api.*.file with only
required version comparison.
* tests/input.at: No longer check api.stack.file and api.position.file.
* NEWS, doc/bison.texi: Don't mention them.
Document the %require 3.2 behavior.
* tests/output.at: Use %require 3.2 instead.
Currently, in C, the default semantic action is implemented by being
always run before running the actual user semantic action. As a
consequence, when the user action is run, $$ is already set as $1.
In C++ with variants, we don't do that, since we cannot manipulate the
semantic value without knowing its exact type. When variants are
enabled, the only guarantee is that $$ is default contructed and ready
to the used.
Some users still would like the default action to be run with
variants. Frank Heckenbach's parser in
C++17 (http://lists.gnu.org/archive/html/bug-bison/2018-04/msg00011.html)
provides this feature, but relying on std::variant's dynamic typing,
which we forbid in lalr1.cc.
The simplest seems to be actually generating the default semantic
action (in all languages/skeletons). This makes the pre-action (that
sets $$ to $1) useless. But... maybe some users depend on this, in
spite of the comments that clearly warn againt this. So let's not
turn this off just yet.
* src/reader.c (grammar_rule_check_and_complete): Rename as...
(grammar_rule_check_and_complete): this.
Install the default semantic action when applicable.
* examples/variant-11.yy, examples/variant.yy, tests/calc.at:
Exercise the default semantic action, even with variants.
Currently we use "<dir><api.location.file>" as \file argument, and as
base for the CPP guard. This is not nice when <dir> is absolute, in
which case it is expected that the user will use api.location.include
to get something nicer. If defined, use that name instead.
* data/location.cc (b4_location_path): New.
Use it.
* tests/c++.at (Shared locations): Check the guard and Doxygen doc.
Now that distcheck no longer fails (see previous commit), let's
address the shortcomings.
* Makefile.am (CLEANDIRS, clean-local): New.
* doc/local.mk, examples/calc++/local.mk, examples/local.mk,
* examples/mfcalc/local.mk, examples/rpcalc/local.mk,
* src/local.mk
(CLEANDIRS): Get rid of Apple's *.dSYM directories.
(CLEANFILES): Get rid of *.output files.
* examples/variant-11.yy, examples/variant.yy: Don't generate
any of the auxiliary files (location.hh and the like).
`make update-po` runs:
package_gnu="$(PACKAGE_GNU)"; \
test -n "$$package_gnu" || { \
if { if (LC_ALL=C find --version) 2>/dev/null | grep GNU >/dev/null; then \
LC_ALL=C find -L $(top_srcdir) -maxdepth 1 -type f \
-size -10000000c -exec grep 'GNU @PACKAGE@' \
/dev/null '{}' ';' 2>/dev/null; \
else \
LC_ALL=C grep 'GNU @PACKAGE@' $(top_srcdir)/* 2>/dev/null; \
fi; \
} | grep -v 'libtool:' >/dev/null; then \
package_gnu=yes; \
else \
package_gnu=no; \
fi; \
}; \
and based on the result, put GNU or not in the following line from
bison.pot:
# This file is distributed under the same license as the GNU bison package.
It turns out that in my environment some log files had the 'GNU bison'
string (note the lower case), but in distcheck, these files are no
longer visible, so the generate bison.pot was different, and distcheck
failed because we try to update bison.pot, which is read only in
distcheck.
The heuristics should look accept 'GNU Bison', not just 'GNU bison'.
But let's please it to make sure we have our 'GNU'.
* README: Mention 'GNU bison'.
Users may want to generate the location file elsewhere, say
$top_srcdir/include/ast/location.hh. Yet, we should not generate
`#include "$top_srcdir/include/ast/location.hh"` but probably
something like `#include <ast/location.hh>`, or `#include
"ast/location.hh", or `#include <location.hh>`. It entirely depends
on the compiler flags (-I/-isystem) that are used. Bison cannot guess
what is expected, so let's give the user a means to tell how the
location file should be included.
* data/location.cc (b4_location_file): New.
* data/glr.cc, data/lalr1.cc: Use it.
In the case a user wants to create location.hh elsewhere, it can be
helpful to define api.location.file to some possibly absolute path
such as -Dapi.location.file='"$(top_srcdir)/include/ast/location.hh"'.
Currently this does not work with `-o foo/parser.cc`, as we join foo/
and $(top_srcdir) together, the latter starting with slash.
We should not try to do that in m4, manipulating file names is quite
complex when you through Windows file name in. Let m4 delegate this
to gnulib.
* src/scan-skel.l (at_output): Accept up to two arguments.
* data/bison.m4 (b4_output): Adjust.
* tests/skeletons.at (Fatal errors but M4 continues producing output):
Adjust to keep the error.
* data/location.cc, data/stack.hh: Leave the concatenation to @output.
* tests/output.at: Exercise api.location.file with an absolute path.
Make it easier to have fewer files.
* data/stack.hh: Don't generate stack.hh when api.location.file is
specified.
* tests/calc++.at, tests/output.at: Adjust tests.
Let's put the definition of position into location.hh, there's no real
value in keeping them separate: they are small, and share the same
requirements.
To help users transition to this new model, still generate position.hh
by default, but as a simple include to location.hh.
* data/location.cc (api.position.file): Accept only 'none' as possible
value.
(position.hh): Make it a stub.
(location.hh): Adjust.
(b4_position_define): Merge into...
(b4_location_define): this.
* data/glr.cc, data/lalr1.cc, tests/input.at, tests/output.at: Adjust.
* data/location.cc: Sort includes.
(b4_position_file, b4_location_file): New.
When there's a file for locations but not for positions, include the
definition of position in the location file.
* data/lalr1.cc (b4_shared_declarations): Include the
position/location file when it exists.
Otherwise, define the class.
* data/glr.cc: Likewise.
* tests/input.at (%define file variables): Check them.
* tests/output.at (C++ output): Check various cases with
api.position.file and api.location.file.
It was not a good idea to generate the file stack.hh. It never was.
But now we have to deal with backward compatibility: if we stop
generating it, the build system of some build system will probably
break.
So offer the user a means to (i) decide what the name of the output
file should be, and (ii) not generate this file at all (its content
will be inline where the parser is defined).
* data/lalr1.cc (b4_percent_define_check_file_complain)
(b4_percent_define_check_file): New.
* data/stack.hh: Generate the file only if api.stack.file is not
empty.
In that case, use it as file name.
* data/lalr1.cc: Adjust to include the right file, or to include
the definition of stack.
* tests/calc.at, tests/output.at: Exercise api.stack.file.
They are not the same concept. It appears that we still consider that
api.prefix is the default for api.namespace. We should stop that.
* tests/local.at (AT_BISON_OPTION_PUSHDEFS, AT_BISON_OPTION_POPDEFS):
Separate namespace and prefix.
Adjust dependencies.
* src/getargs.c (getargs): Don't display any argv other that argv[0]
when reporting a missing argument.
* tests/bison.in: Neutralize path differences in stderr.
* tests/input.at (Invalid number of arguments): New.
* src/getargs.c (usage): Rely on setlocale (LC_MESSAGES, NULL)
trick only on glibc, as POSIX does not specify the output
of setlocale in this case, and the Gnulib localename module
source code indicates that the trick works only on glibc.
C11 no longer requires support for variable-length arrays, and
VS2015 does not have them. Redo UNIQSTR_CONCAT to use a method
that is simpler and better anyway.
* src/uniqstr.c (uniqstr_vsprintf): Remove; no longer needed.
* src/uniqstr.h (UNIQSTR_GEN_FORMAT, UNIQSTR_GEN_FORMAT_):
* src/uniqstr.c (uniqstr_concat): New function.
* src/uniqstr.h (UNIQSTR_CONCAT): Use it instead of using
uniqstr_vsprintf.
* doc/bison.texi: Minor fixes in typography.
It is no longer require to pass --defines for C++ (it was addressed
long ago).
No longer refer to the `variant` define variable, it was replaced by
`api.value.type variant`.
Prefer nullptr to 0 for the null pointer.
Use deftypeop for constructors.
(Complete Symbols): Give the expected signature of yylex.
Don't document the symbol_type constructors, as we want users to focus
on make_TOKEN.
Also show the case without locations.
Bison's test 464 (Syntax error as exception) fails on the CI.
Do not use clang with asan on Ubuntu's libc++.
https://bugs.llvm.org/show_bug.cgi?id=17379
* .travis.yml (Clang 7 libc++ and ASAN): New.
(Clang 6 -O3 and libc++): Really use libc++.
(Clang 5): Don't use libc++, nor asan (does not work either, same
reason).
* .travis.yml: Don't define CC/CXX, it does not work.
Use `[[...]]` instead of `[...]`.
Show the compiler versions.
(Clang 3.3, Clang 3.4): Specify the path to avoid using
/usr/local/clang-5.0.0/bin's clang.
The message on configure is misleading:
checking whether the linker accepts -std=c++11... yes
checking whether the linker accepts -std=c++14... yes
checking whether the linker accepts -std=c++17... no
It is the compiler that we check, not just the linker.
* m4/ax_check_link_flag.m4: Remove.
* m4/bison-check-compiler-flag.m4: New.
* m4/bison-cxx-std.m4: Use it.
This has been bugging me for while. I was hard to reproduce: it
worked only on GNU/Linux, probably because libc++ implements the small
string optimization, while libstdc++ did not and actually allocated on
the heap for this small string.
See https://lists.gnu.org/archive/html/bison-patches/2018-09/msg00110.html.
* tests/types.at (api.value.type): Do not provide a semantic value to
EOF.
The forthcoming automove feature, to be properly checked, will require
that we can rely on the value of a moved-from string, which is not
something the C++ standard guarantees. So introduce our own wrapper.
Suggested by Frank Heckenbach.
https://lists.gnu.org/archive/html/bison-patches/2018-09/msg00111.html
* tests/c++.at (Variants): Introduce and use a new 'string' class.
The 'Variants' tests are well suited to check support for move, and in
particular for the forthcoming automove feature. But the tests were
written to show the best practice in C++98, using swap:
list "," item { std::swap ($$, $1); $$.push_back ($3); }
This cannot work with std::move. So, make this example simpler, based
on regular assignment instead of swap, which is a regression for
C++98 (as the new traces show), but will be an improvement for modern
C++ with automove.
* tests/c++.at (Variants): Stop using swap.
We don't generate a header file, so remove the 'require' code section.
Adjust expectations.
Several types of failures. First, unable to pass the file name
properly to the linker.
./synclines.at:416: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o \"\\\"\" \"\\\"\".c $LIBS
stderr:
ld: cannot open output file "/"": No such file or directory
stdout:
Unable to save under such a file name.
./synclines.at:421: $CXX $CXXFLAGS $CPPFLAGS -c $LDFLAGS -o \"\\\"\" \"\\\"\".cc $LIBS
stderr:
error: can't open file "/"" for write
compilation aborted for "\"".cc (code 1)
Spurious output because of warning flags is failed to reject as an
error during configure:
./headers.at:343: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS c-only.o cxx-only.o -o c-and-cxx ||
exit 77
--- /dev/null 2018-09-18 21:21:37.745649000 +0000
+++ /home/travis/build/akimd/bison/tests/testsuite.dir/at-groups/222/stderr 2018-09-18 21:28:17.291919519 +0000
@@ -0,0 +1,7 @@
+icpc: command line warning #10006: ignoring unknown option '-Wcast-align'
+icpc: command line warning #10006: ignoring unknown option '-fparse-all-comments'
+icpc: command line warning #10006: ignoring unknown option '-Wdocumentation'
+icpc: command line warning #10006: ignoring unknown option '-Wnull-dereference'
+icpc: command line warning #10006: ignoring unknown option '-Wnoexcept'
+icpc: command line warning #10006: ignoring unknown option '-fno-color-diagnostics'
+icpc: command line warning #10006: ignoring unknown option '-Wno-keyword-macro'
stdout:
* tests/local.at (AT_SKIP_IF_CANNOT_LINK_C_AND_CXX): Also ignore
stderr, as with ICC we get
* tests/synclines.at (syncline escapes): Don't link the output.
The code was already using midrule only, never mid_rule. This is
simpler to remember, and matches a similar change we made from
look-ahead to lookahead.
* NEWS, doc/bison.texi, src/reader.c, src/scan-code.h, src/scan-code.l
* tests/actions.at, tests/c++.at, tests/existing.at: here.
We use both styles, let's stick to a single one. Autoconf uses the
prefix one, let's do the same.
* data/bison.m4, data/c++.m4, data/c-like.m4, data/lalr1.cc,
* data/variant.hh, data/yacc.c: Rename all the b4_*_ macros
as _b4_*.
On the CI, we had failures such as:
./c++.at:401: $PREPARSER ./list
stderr:
./list: error while loading shared libraries: libc++.so.1:
cannot open shared object file: No such file or directory
because we accepted `-std=c++ -stdlib=libc++` although libc++ is not
installed on the machine.
* m4/ax_check_compile_flag.m4 (AX_CHECK_COMPILE_FLAG): Rewrite as...
* m4/bison-check-compile-flag.m4 (BISON_CHECK_COMPILE_FLAG): this, so
that we use AC_LINK_IFELSE to check the compiler (and its std lib)
instead of AC_COMPILE_IFELSE.
The CI is littered with
# -*- compilation -*-
423. regression.at:907: testing Dancer %glr-parser ...
./regression.at:907: bison -fno-caret -o dancer.c dancer.y
./regression.at:907: $BISON_C_WORKS
stderr:
stdout:
./regression.at:907: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o dancer dancer.c $LIBS
stderr:
icc: command line warning #10006: ignoring unknown option '-Wcast-align'
icc: command line warning #10006: ignoring unknown option '-fparse-all-comments'
icc: command line warning #10006: ignoring unknown option '-Wdocumentation'
icc: command line warning #10006: ignoring unknown option '-Wnull-dereference'
icc: command line warning #10006: ignoring unknown option '-Wbad-function-cast'
icc: command line warning #10006: ignoring unknown option '-fno-color-diagnostics'
icc: command line warning #10006: ignoring unknown option '-Wno-keyword-macro'
dancer.c(755): error #1628: function declared with "noreturn" does return
}
^
dancer.c(761): error #1628: function declared with "noreturn" does return
}
^
compilation aborted for dancer.c (code 2)
ICC sees that `longjmp(buf, 1);` does not return, it sees that
`abort();` does not either, but fails to see it for
`longjmp(buf, 1); abort();`
* data/glr.c (YYLONGJMP): Be even clearer on the fact this does not
return.
Putting them in the env is useless. We don't want to pass
`CPPFLAGS="$CPPFLAGS"` to configure, as it means "set it to nothing"
when $CPPFLAGS is not set, which is not what we want.
This correctly started to use libc++, but it is not installed on the
Ubuntu. We will see later if we can use it.
* .travis.yml: Define CONFIGUREFLAGS, and pass it to configure.
* build-aux/install-icc.sh: New.
* .travis.yml (icc): New.
Use -k to get as many errors as possible from the start.
* src/complain.c (warnings_types): Use a more precise type.
On the CI, we have this spurious failure with clang 3.9 with
-std=c++17:
In file included from list.y:23:
In file included from /usr/include/c++/4.8/iostream:39:
In file included from /usr/include/c++/4.8/ostream:38:
In file included from /usr/include/c++/4.8/ios:42:
In file included from /usr/include/c++/4.8/bits/ios_base.h:41:
In file included from /usr/include/c++/4.8/bits/locale_classes.h:40:
In file included from /usr/include/c++/4.8/string:52:
In file included from /usr/include/c++/4.8/bits/basic_string.h:2815:
In file included from /usr/include/c++/4.8/ext/string_conversions.h:43:
/usr/include/c++/4.8/cstdio:120:11: error: no member named 'gets' in the global namespace
using ::gets;
~~^
This shows that our test, based on gl_WARN_ADD, is a joke. We have to
really check for at least a bit of C++.
* m4/ax_check_compile_flag.m4, m4/bison-cxx-std.m4: New.
* configure.ac: Use them to make sure the compiler actually works.
In C++ pre C++11 it is standard practice to use 0 for the null pointer.
But GCC pre 8 -std=c++98 with -Wzero-as-null-pointer-constant warns about
this.
So disable -Wzero-as-null-pointer-constant when compiling C++ pre 11.
Let's do this in AT_DATA_SOURCE_PROLOGUE (which is pasted on top of
all the test grammar files). Unfortunately, that shifts all the
locations in the expected error messages, which would be too noisy.
Instead, let's introduce testsuite.h, which can vary in length, and
include it in AT_DATA_SOURCE_PROLOGUE.
* tests/testsuite.h: New.
Disable -Wzero-as-null-pointer-constant's warning with GCC pre 8,
C++ pre 11.
* tests/local.at (AT_DATA_SOURCE_PROLOGUE): Use it.
* tests/atlocal.in (CPPFLAGS): Find it.
* tests/local.mk: Ship it.
* data/c.m4 (YY_NULLPTR): Prefer ((void*)0) to 0 in C.
For some reasons, the checkout on travis may not have any tags, so
`git describe` fails, so bootstrap fails.
* .travis.yml: If git describe fails, install some tag.
This is much of course more efficient than in the matrix of the CI (or
on our own machines), but a bit more tedious.
* configure.ac (CXX03_CXXFLAGS, CXX11_CXXFLAGS, CXX14_CXXFLAGS)
(CXX17_CXXFLAGS, CXX2A_CXXFLAGS, STDCXX_FLAGS): New.
* tests/atlocal.in: Receive them.
* tests/local.at (AT_FOR_EACH_CXX): New.
* tests/c++.at: Use AT_FOR_EACH_CXX.
Reported by Frank Heckenbach.
http://lists.gnu.org/archive/html/bug-bison/2018-03/msg00002.html
Actually the assignment operator should never be needed: the C++98
requirements for vector::push_back is CopyInsertable, which does not require
an assignment operator. However, libstdc++ shipped with GCC up to (and
including) 6 uses the assignment operator (which affects Clang on top of
libstdc++, but also ICC). So let's keep it for legacy C++.
See https://gcc.godbolt.org/z/q0XXmC.
* data/lalr1.cc (stack_symbol_type::operator=): Remove.
* data/c++.m4 (basic_symbol::operator=): Ditto.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Adjust.
Modern C++ (i.e., C++11 and later) introduced "move only" types: types such
as std::unique_ptr<T> that can never be duplicated. They must never be
copied (by assignments and constructors), they must be "moved". The
implementation of lalr1.cc used to copy symbols (including their semantic
values). This commit ensures that values are only moved in modern C++, yet
remain compatible with C++98/C++03.
Suggested by Frank Heckenbach, who provided a full implementation on
top of C++17's std::variant.
See http://lists.gnu.org/archive/html/bug-bison/2018-03/msg00002.html,
and https://lists.gnu.org/archive/html/bison-patches/2018-04/msg00002.html.
Symbols (terminal/non terminal) are handled by several functions that used
to take const-refs, which resulted eventually in a copy pushed on the stack.
With modern C++ (C++11 and later) the callers must use std::move, and the
callees must take their arguments as rvalue refs (foo&&). In order to avoid
duplicating these functions to support both legacy C++ and modern C++, let's
introduce macros (YY_MOVE, YY_RVREF, etc.) that rely on copy-semantics for
C++98/03, and move-semantics for modern C++.
That's easy for inner types, when the parser's functions pass arguments to
each other. Functions facing the user (make_NUMBER, make_STRING, etc.)
should support both rvalue-refs (for instance to support move-only types:
make_INT (std::make_unique<int> (1))), and lvalue-refs (so that we can pass
a variable: make_INT (my_int)). To avoid the multiplication of the
signatures (there is also the location), let's take the argument by value.
See:
https://lists.gnu.org/archive/html/bison-patches/2018-09/msg00024.html.
* data/c++.m4 (b4_cxx_portability): New.
(basic_symbol): In C++11, replace copy-ctors with move-ctors.
In C++11, replace copies with moves.
* data/lalr1.cc (stack_symbol_type, yypush_): Likewise.
Use YY_MOVE to avoid useless copies.
* data/variant.hh (variant): Support move-semantics.
(make_SYMBOL): In C++11, in order to support both read-only lvalues,
and rvalues, take the argument as a copy.
* data/stack.hh (yypush_): Use rvalue-refs in C++11.
* tests/c++.at: Use move semantics.
* tests/headers.at: Adjust to the new macros (YY_MOVE, etc.).
* configure.ac (CXX98_CXXFLAGS, CXX11_CXXFLAGS, CXX14_CXXFLAGS)
(CXX17_CXXFLAGS, ENABLE_CXX11): New.
* tests/atlocal.in: Receive them.
* examples/variant.yy: Don't define things in std.
* examples/variant-11.test, examples/variant-11.yy: New.
Check the support of move-only types.
* examples/README, examples/local.mk: Adjust.
This generates less code, which is nicer to read, but also takes less
chances with compilers such as G++ 4.8 that are too strict and check
"dead code" (templated code that is not instantiated).
* data/c++.m4 (b4_symbol_type_declare, b4_symbol_type_define): When
variants are used, don't generate code meant for non variants.
See ea0db44fed. We also need to disable
the warning in the examples (but don't want to clutter the
documentation with such details).
* doc/bison.texi (scanner.ll): Disable Clang's -Wdocumentation.
While at it, hide the other kludges.
That's the version on Ubuntu Precise.
See also 1dac131ec4.
* src/flex-scanner.h: Disable -Wdocumentation.
* doc/bison.texi: Turn off a warning triggered by Flex 2.6.4.
We have failures on Flex output, which are probably related to an old
release. Let's check.
In file included from src/scan-code-c.c:3:
src/scan-code.c:2198:21: error: empty paragraph passed to '@param' command
[-Werror,-Wdocumentation]
* @param line_number
~~~~~~~~~~~~~~~~~^
* .travis.yml: here.
Running all these tests might be overkill: it is very long, and don't
need full portability checks. Besides, some tests under Valgrind are
too slow and get killed by the CI (timeout of 10min without output).
* .travis.yml: here.
We have some maintainer-check-foo and some maintainer-foo-check. Keep
only the former.
* tests/local.mk (maintainer-push-check, maintainer-xml-check)
(maintainer-release-check): Rename as...
(maintainer-check-push, maintainer-check-xml)
(maintainer-check-release): these.
Most of our variables for C++ flags are named FOO_CXXFLAGS, not
CXXFLAGS_FOO.
* configure.ac, tests/atlocal.in, tests/calc.at
(NO_EXCEPTIONS_CXXFLAGS): Rename as...
(CXXFLAGS_NO_EXCEPTIONS): this.
Clang++ issues warnings when it's used to compile C. This make target
is precisely checking whether we can do that.
* configure.ac (NO_DEPRECATED_CXXFLAGS): New.
* tests/atlocal.in: Use it.
This completes 2d7e743802.
Some shells don't grok "local var=`cmd`" very well: they need the rhs
to be quoted.
./examples/test: 72: local: you.,: bad variable name
FAIL examples/variant.test (exit status: 2)
Reported by Étienne Renault.
* examples/test (run): Quote the values in 'local' assignments.
With GCC7 we have warnings (false positive):
x8.c: In function 'x8_parse':
x8.c:1233:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
yylval = *yypushed_val;
~~~~~~~^~~~~~~~~~~~~~~
x8.c: In function 'x8_pull_parse':
x8.c:1233:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
yylval = *yypushed_val;
~~~~~~~^~~~~~~~~~~~~~~
See also 9645a2b20e.
* tests/local.at (AT_PUSH_IF): New.
(AT_BISON_OPTION_POPDEFS): Pop it, and pop AT_PURE_IF.
* tests/headers.at (Several parsers, Several parsers): Disable these
warnings when in push parser.
Currently, in glr.cc, we emit the definitions of basic_symbol and
symbol_type, although there are not used.
* data/c++.m4 (b4_public_types_declare): Extract these definitions from
here, and move them...
(b4_symbol_type_declare): here.
(b4_public_types_declare): Also remove the definition of the symbol
constructors.
* data/lalr1.cc (b4_shared_declarations): Adjust: call
b4_symbol_type_declare and b4_symbol_constructor_declare.
Some shells don't grok `local var=$val` very well: they need the rhs
to be quoted.
./examples/test: 66: local: you.,: bad variable name
FAIL examples/variant.test (exit status: 2)
Reported by Étienne Renault.
* examples/test (run): Quote the values in 'local' assignments.
# $SSH_PRIVATE_KEY is multiline. Use $'...' to register its value: $'-----BEGIN OPENSSH PRIVATE KEY-----\nXXXXX...\n...==\n-----END OPENSSH PRIVATE KEY-----'.
# We always use _deps and _libs instead of .deps and .libs, because
# the latter is an invalid name on 8+3 MS-DOS file system. This makes
# the generated Makefiles good for every DJGPP installation, not only
# the one where the package was configured (which could happen to be
# a Windows box, where leading dots in file names are allowed).
/^rmdir[ ]*\.tst/i\
am__leading_dot=_
# Replace (command) > /dev/null with `command > /dev/null`, since
# parenthesized commands always return zero status in the ported Bash,
# even if the named command doesn't exist
/if ([^|;`]*null/{
s,(,`,
s,),,
/null[ ]*2>&1/s,2>&1,&`,
/null.*null/s,null.*null,&`,
/null.*null/!{
/null[ ]*2>&1/!s,null,&`,
}
}
# DOS-style absolute file names should be supported as well
/\*) top_srcdir=/s,/\*,[\\\\/]* | ?:[\\\\/]*,
# The following two items are changes needed for configuring
# and compiling across partitions.
# 1) The given srcdir value is always translated from the
# "x:" syntax into "/dev/x" syntax while we run configure.
/^[ ]*-srcdir=\*.*$/a\
ac_optarg=`echo "$ac_optarg" | sed "s,^\\([A-Za-z]\\):,/dev/\\1,"`
/set X `ls -Lt \$srcdir/i\
if `echo $srcdir | grep "^/dev/" - > /dev/null`; then\
srcdir=`echo "$srcdir" | sed -e "s%^/dev/%%" -e "s%/%:/%"`\
fi
# Autoconf 2.52e generated configure scripts
# write absolute paths into Makefiles and bison.in
# making them useless for DJGPP installations for
# which the package has not been configured for.
/MISSING=/,/^$/{
/^fi$/a\
am_missing_run=`echo "$am_missing_run" | sed 's%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%'`
}
/^install_sh=/a\
install_sh=`echo "$install_sh" | sed 's%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%'`
# This will only work if the assumption that the
# testsuite is ran from the following path:
# ${top_srcdir}/tests/testsuite.dir
# holds. The explicit names are of no importance.
# ../../. == ${top_srcdir}
#/^esac$/,/^ac_abs_srcdir=/ {
/^ac_abs_srcdir=/i\
ac_abs_builddir=`echo "$ac_abs_builddir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%"`\
ac_abs_top_builddir=`echo "$ac_abs_top_builddir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%"`\
ac_abs_top_srcdir=`echo "$ac_abs_top_srcdir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%"`
#ac_abs_srcdir=`echo "$ac_abs_srcdir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%"`
/^ac_abs_srcdir=/a\
for dj_dir in $ac_abs_builddir $ac_abs_top_builddir $ac_abs_top_srcdir $ac_abs_srcdir\
do\
ac_abs_builddir=`echo "$ac_abs_builddir" | sed "s%//%/%g"`\
ac_abs_top_builddir=`echo "$ac_abs_top_builddir" | sed "s%//%/%g"`\
ac_abs_top_srcdir=`echo "$ac_abs_top_srcdir" | sed "s%//%/%g"`\
ac_abs_srcdir=`echo "$ac_abs_srcdir" | sed "s%//%/%g"`\
done
# Add DJGPP version information.
/^#define VERSION/s/\$VERSION/& (DJGPP port (r1))/
# We need makeinfo to make the html formated docs.
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Paul Eggert
Bruno HaibleandPaul Eggert
FutureandPaul Eggert
Akim Demaille
Adela VaisandAkim Demaille
Anthony HeadingandAkim Demaille
Alyssa RossandAkim Demaille
Joshua WattandAkim Demaille
Kaz KylhekuandAkim Demaille
Vincent ImbimboandAkim Demaille
Martin RehakandAkim Demaille
Valentin TolmerandAkim Demaille
Adela VaisandAkim Demaille
Todd C. MillerandAkim Demaille
Nick GassonandAkim Demaille
Maarten De BraekeleerandAkim Demaille
Akim DemailleandAkim Demaille
Thomas PetazzoniandAkim Demaille
Adrian VogelsgesangandAkim Demaille
Victor Morales CayuelaandAkim Demaille
Jim MeyeringandAkim Demaille
kaneko yandAkim Demaille
László VáradyandAkim Demaille
Theophile RanquetandAkim Demaille
H. S. TeohandAkim Demaille
Eric S. RaymondandAkim Demaille
Kiyoshi KanazawaandAkim Demaille
Eduard StaniloiuandAkim Demaille
Paul HilfingerandAkim Demaille
JannickandAkim Demaille
Adam SampsonandAkim Demaille
Josh SorefandAkim Demaille