Readline may emit escape sequences before the prompt.
Reported by Bruno Haible.
https://lists.gnu.org/r/platform-testers/2020-05/msg00001.html.
* examples/c/bistromathic/bistromathic.test: Trust readline _only_ if
we get what we expect on some reference computation.
Appearing on IRIX with gcc -mabi=n32.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00039.html
* examples/c++/variant-11.yy, examples/c/bistromathic/parse.y: Don't
give chars to isdigit, cast them to unsigned char before.
* src/complain.c: Use c_isdigit.
* src/fixits.c (fixits_run): Avoid casts.
* src/lalr.c (goto_print): Use %zu for a size_t.
Don't try to build bistromathic if we don't have readline.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00028.html
* configure.ac (ENABLE_BISTROMATHIC): New.
* examples/c/bistromathic/local.mk: Use it.
* examples/c/bistromathic/bistromathic.test: Exit 77 for skip.
"foo || bar" does not invoke bar on AIX 7.2 when foo does not exist.
It just dies.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00029.html
* examples/c/reccalc/reccalc.test: Check for seq in a subshell.
* data/skeletons/lalr1.java (Location): Make it a static class.
(Lexer.yylex, Lexer.getLVal, Lexer.getStartPos, Lexer.getEndPos):
These are not needed in push parsers.
* examples/java/calc/Calc.y: Demonstrate push parsers in the Java.
* doc/bison.texi: Push parsers have been supported for a long time,
remove incorrect statements stating the opposite.
Reported by Horst von Brand.
https://lists.gnu.org/r/bug-bison/2020-04/msg00033.html
* examples/c/bistromathic/Makefile: libintl might not be needed, but
libm probably is.
* examples/c/bistromathic/parse.y: Include locale.h.
On OpenBSD 6.5, the prompt is repeated, but not the actual command
line... Don't try to cope with that.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00015.html
* examples/c/bistromathic/bistromathic.test: Skip when readline behave
this way.
The option --header was introduced in version 2.5.6.
The option --header-file was introduced in version 2.6.4.
Reported by Bruno Haible.
https://lists.gnu.org/r/bug-bison/2020-05/msg00013.html
So use --header, and do bother with versions that don't support it.
* m4/flex.m4: Check whether flex supports --header.
* configure.ac (FLEX_WORKS, FLEX_CXX_WORKS): Set to false if it doesn't.
* * examples/c/reccalc/local.mk, examples/c/reccalc/Makefile:
Use --header rather than --header-file.
To write unit tests for their scanners, some users depended on
symbol_type::token():
Lexer lex("12345");
symbol_type t = lex.nextToken();
assert(t.token() == token::INTLIT);
assert(t.value.as<int>() == 12345);
But symbol_type::token() was removed in Bison 3.5 because it relied on
a conversion table. So users had to find other patterns, such as
assert(t.type_get() == by_type(token::INTLIT).type_get());
which relies on several private implementation details.
As part of transitioning from "token type" to "token kind", and making
this a public and documented interface, "by_type" was renamed
"by_kind" and "type_get()" was renamed as "kind()". The latter had
backward compatibility mechanisms, not the former.
In Bison 3.6 none of this should be used, but rather
assert(t.kind() == symbol_kind::S_INTLIT);
Reported by Pramod Kumbhar.
https://lists.gnu.org/r/bug-bison/2020-05/msg00012.html
* data/skeletons/c++.m4 (by_type): Make it an alias to by_kind.
I don't plan to fix everything in one go. But this was in the way of
the next commit.
* data/skeletons/lalr1.java: Avoid space before parens.
* tests/java.at: Adjust.
Bison emits strings to translate in the generated code, for builtin
tokens. So they appear only in generated parsers, which are not
shipped, so they are not in the src tree, so we cannot use them in our
POTFILE.
Except src/parse-gram.c, which is in the source tree. And even in the
git repo. But to avoid useless diffs in the repo, we do not keep the
src/parse-gram.c _with_ the #lines. This is done in a dist-hook which
regenerates src/parse-gram.c when we run "make dist".
Unfortunately, then, update-po traverses the whole tree and sees that
the location of the strings to translate in src/parse-gram.c have
changed, so the bison.pot is to be updated. And that is not possible
in the "make dist" which is run within "make distcheck"
(not the one preparing the dist for distcheck, the one run by
distcheck to check that a distributed tarball can build a tarball)
because then the src tree is read-only.
So let's not put src/parse-gram.c in the POTFILE, and expose these
strings to gettextize by hand.
* src/i18n-strings.c: New.
* po/POTFILES.in: Add it, and remove src/parse-gram.c.
Some people have been using that symbol. Some even have #defined it
themselves.
https://lists.gnu.org/r/bison-patches/2020-04/msg00138.html
Let's provide backward compatibility, having it point to YYUNDEF, so
that an error message is generated.
* data/skeletons/yacc.c (YYERRCODE): New, at the exact same location
it was defined before.
These are internal details. `type_get ()` is still there to ensure
backward compatibility, `kind ()` being the modern way.
* data/skeletons/c++.m4 (by_type, by_type::type): Rename as...
(by_kind, by_kind::kind_): this.
Adjust dependencies.
From
public interface Lexer {
/* Token kinds. */
/** Token number, to be returned by the scanner. */
static final int YYEOF = 0;
/** Token number, to be returned by the scanner. */
static final int YYERRCODE = 256;
/** Token number, to be returned by the scanner. */
static final int YYUNDEF = 257;
/** Token number, to be returned by the scanner. */
static final int BANG = 258;
...
/** Deprecated, use b4_symbol(0, id) instead. */
public static final int EOF = YYEOF;
to
public interface Lexer {
/* Token kinds. */
/** Token "end of file", to be returned by the scanner. */
static final int YYEOF = 0;
/** Token error, to be returned by the scanner. */
static final int YYerror = 256;
/** Token "invalid token", to be returned by the scanner. */
static final int YYUNDEF = 257;
/** Token "!", to be returned by the scanner. */
static final int BANG = 258;
...
/** Deprecated, use YYEOF instead. */
public static final int EOF = YYEOF;
* data/skeletons/java.m4 (b4_token_enum): Display the symbol's tag in
comment.
* data/skeletons/lalr1.java: Address overquotation issue.
* examples/java/calc/Calc.y, examples/java/simple/Calc.y: Use YYEOF,
not EOF.
On an invalid character literal such as "'\777'" we used to produce
two errors:
input.y:2.9-12: error: invalid number after \-escape: 777
input.y:2.8-13: error: empty character literal
Get rid of the second one.
* src/scan-gram.l (STRING_GROW_ESCAPE): New.
* tests/input.at: Adjust.
I'm quite pleased to see that the tricky case of glr.c was already
prepared by the changes to support syntax_error exceptions. Better
yet, it is actually syntax_error that becomes a special case of the
general pattern: make yytoken be YYERRCODE.
* data/skeletons/glr.c (YYFAULTYTOK): Remove the now useless (Basil)
Faulty token.
Instead, use the error token.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java: When computing
the action, first check the case of the error token.
* tests/calc.at: Check cases for the error token symbols before and
after it.
* data/skeletons/yacc.c (yyparse): When the scanner returns YYERRCODE,
go directly to error recovery (yyerrlab1).
However, don't keep the error token as lookahead, that token is too
special.
* data/skeletons/lalr1.cc: Likewise.
* examples/c/bistromathic/parse.y (yylex): Use that feature to report
nicely invalid characters.
* examples/c/bistromathic/bistromathic.test: Check that.
* examples/test: Neutralize gratuitous differences such as rule
position.
* tests/calc.at: Check that case in C only.
The other case seem to be working, but that's an illusion that the
next commit will address (in fact, they can enter endless loops, and
report the error several times anyway).
* examples/c/bistromathic/parse.y: here.
* examples/c/bistromathic/bistromathic.test: Check it.
Included a stupid case where the error is actually ignored.
When the user ctrl-d the line, we left the cursor not at col 0.
Let's fix that.
This revealed a few short-comings in the testing framework.
* examples/test (run): Also display the diffs.
And support -n.
* examples/c/bistromathic/bistromathic.test
* examples/c/bistromathic/parse.y
We will not keep YYERRCODE anyway, it causes backward compatibility
issues. So as a first step, let all the skeletons use that name,
until we have a better one.
* data/skeletons/bison.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/lalr1.cc,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/yacc.c, doc/bison.texi, tests/headers.at,
* tests/input.at:
here.
* etc/bench.pl.in: Don't force parse.error=detailed
Use a simpler way to display the pseudo %bison directive.
(&bench_with_gbenchmark): Give details about the compiler.
We could try to avoid the weird "#if 1", but then the indentation of
the inner #if would be wrong. Let' keep it this way.
* data/skeletons/yacc.c: here.
Also, avoid sticking the comment to the directive.
Currently it was only using stubs. Let's actually translate the
strings using gettext.
* examples/c/bistromathic/local.mk: Define LOCALEDIR, BISON_LOCALEDIR
and link with libintl.
* examples/c/bistromathic/parse.y: Use them.
Remove useless includes.
Take ENABLE_NLS into account.
(error_format_string): New.
(yyreport_syntax_error): Rewrite to rely on a format string, which is
more appropriate for internationalization.
* examples/c/bistromathic/Makefile: We no longer use Flex.
We need readline and intl.
* doc/bison.texi: Point to bistromathic for a better option for
internationalization.
* po/POTFILES.in: Add bistromathic.
In C/C++, N_ is a no-op. Define it if the user didn't.
Suggested by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2020-04/msg00010.html
* src/output.c (prepare_symbol_names): Rename has_translations as
has_translations_flag.
* data/skeletons/bison.m4 (b4_has_translations_if): New.
* data/skeletons/java.m4 (b4_trans): Use it.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/yacc.c
(N_): Provide a default definition.
* src/output.c (prepare_symbol_names): Don't play tricks with the
symbols, it's quite too late.
(has_translations): Move to...
* src/symtab.c: here.
(symbols_pack): Use it to enable translation for special symbols.
On macOS, wc -l always prepends the result with a tab, even when fed
by stdin. But anyway, we should have used `grep -c -v`, which appears
to be portable according to Autoconf's "Limitations of Usual Tools"
section.
Reported by Denis Excoffier.
https://lists.gnu.org/r/bug-bison/2020-04/msg00009.html
* tests/calc.at (_AT_CHECK_CALC): Use grep's -c instead.
macOS' version of readline does not repeat stdin on stdout in
non-interactive mode, contrary to the current version of GNU readline.
* examples/test: Add support for strip_prompt.
* examples/c/bistromathic/bistromathic.test (strip_prompt): Set it
when needed.
Early exit when needed.
symbol_type::token () was removed: it returned the token kind of a
symbol. To do that, one needs to convert from the symbol kind to the
token kind, which requires a table.
This broke some users' unit tests for scanners, see
https://lists.gnu.org/r/bug-bison/2020-01/msg00001.htmlhttps://lists.gnu.org/r/bug-bison/2020-03/msg00020.htmlhttps://lists.gnu.org/r/help-bison/2020-04/msg00005.html
Instead of making this possible again, let's check the symbol's kind
instead. So give proper access to a symbol's kind.
That feature existed, undocumented, as 'type_get()'. Let's rename
this as 'kind()'.
* data/skeletons/c++.m4, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc (type_get): Rename as...
(kind): This.
(type_get): Install a backward compatibility alias.
* doc/bison.texi (Complete Symbols): Document symbol_type and
symbol_type::kind.
* data/skeletons/c++.m4: Define the old names in terms on the new
ones, instead of the converse.
* doc/bison.texi (C++ Parser Interface): Be more extensive about
token_kind_type.
Not all the symbols have a fixed symbol code. UNDEF's one is fixed:
-2.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/yacc.c: here.
Why didn't I think about this before??? symbolName should be a method
of SymbolKind.
* data/skeletons/lalr1.java (YYParser::yysymbolName): Move as...
* data/skeletons/java.m4 (SymbolKind::getName): this.
Make the table a static final table, not a local variable.
Adjust dependencies.
* doc/bison.texi (Java Parser Interface): Document i18n.
(Java Parser Context Interface): Document SymbolKind.
* examples/java/calc/Calc.y, tests/local.at: Adjust.
* doc/bison.texi (C++ Parser Context): New.
* data/skeletons/lalr1.cc (parser::yysymbol_name): Rename as...
(parser::symbol_name): this.
(A Complete C++ Example): Promote LAC, now that we have it.
Promote parse.error detailed over verbose.
* examples/c++/calc++/calc++.test, tests/local.at: Adjust.
* NEWS (Deep overhaul of the symbol and token kinds): New.
* doc/bison.texi: Promote YYEOF over "0" in scanners.
(Token Decl): No longer show YYEOF here, it now works by default.
(Token I18n): More details about YYEOF here.
(Calc++): Just use YYEOF.
I have been hesitating a lot before doing it ---after all the user
must not use this kind, so what's the point of showing it in
yytoken_kind_t. And eventually I chose to play it safe with the
typing system and make it possible to use yytoken_kind_t for all the
tokens, even the "empty token".
* data/skeletons/c.m4: Give an id and a tag to YYEMPTY.
(b4_token_enums): Define YYEMPTY.
* data/skeletons/c++.m4 (b4_token_enums): Define YYEMPTY.
* data/skeletons/glr.c, data/skeletons/glr.cc, data/skeletons/yacc.c:
(YYEMPTY): Remove.
Use b4_symbol(-2, id) instead.
The user should think of yypcontext fields as accessible only via
yypcontext_* functions. So let's rename yyexpected_tokens to reflect
that.
Let's _not_ rename yyreport_syntax_error, as the user may define this
function, and is not allowed to access directly the fields of
yypcontext_t: she *must* use the "accessors". This is comparable to
the case of C++/Java where the user defines
parser::report_syntax_error, not parser::context::report_syntax_error.
* data/skeletons/glr.c, data/skeletons/yacc.c (yyexpected_tokens):
Rename as...
(yypcontext_expected_tokens): this.
Adjust dependencies.
The name "$end" is nice in the report, in particular it avoids that
pointed-rules (aka items) be too long. It also helps keeping them
"standard".
But it is bad in error messages, we should report "end of file" (or
maybe "end of input", this is debatable). So, unless the user already
defined the alias for the error token herself, make it "end of file".
It should even be translated if the user already translated some
tokens, so that there is now no strong reason to redefine the $end
token.
* src/output.c (prepare_symbol_names): Issue "end of file" instead of
"$end".
* data/skeletons/lalr1.java (yytnamerr_): Remove the renaming hack.
* build-aux/update-test: Accept files with names containing a "+",
such as c++.at.
* tests/actions.at, tests/c++.at, tests/conflicts.at,
* tests/glr-regression.at, tests/regression.at, tests/skeletons.at:
Adjust.
Yet, don't change the structure identifier to avoid introducing
conflicts in Vincent Imbimbo's PR (which, amusingly enough, is about
conflicts).
* src/symtab.c: here.
* tests/diagnostics.at, tests/input.at: Adjust.
yy::parser features a parse() function, not a yyparse() one.
* data/skeletons/lalr1.cc (yyreport_syntax_error)
(context::yyexpected_tokens): Rename as...
(report_syntax_error, context::expected_tokens): these.
Currently EOF is handled in an adhoc way, with a #define YYEOF 0 in
the implementation file. As a result, the user has to define her own
EOF token if she wants to use it, which is a pity.
Give the $end token a visible kind name, YYEOF. Except that in C,
where enums are not scoped, we would have collisions between all the
definitions of YYEOFs in the header files, so in C, make it
<api.PREFIX>EOF.
* data/skeletons/c.m4 (YYEOF): Override its name to avoid collisions.
Unless the user already gave it a different name.
* data/skeletons/glr.c (YYEOF): Remove.
Use ]b4_symbol(0, [id])[ instead.
Add support for "pre_epilogue", for glr.cc.
* data/skeletons/glr.cc: Remove dead code (never emitted #undefs).
* data/skeletons/yacc.c
* src/parse-gram.c
* src/reader.c
* src/symtab.c
* tests/actions.at
* tests/input.at
* data/skeletons/bison.m4 (b4_symbol_token_kind): Give a definition to
$undefined.
(b4_token_visible_if): $undefined has an id.
* src/output.c (prepare_symbol_definitions): Stop lying: $undefined
_is_ a token.
* tests/input.at: Adjust.
There are people out there that do use YYERRCODE (the token kind of
the error token). See for instance
https://github.com/borbolla-automation/SPC_Machines/blob/3812012bb782bfdfe7b325950a35cd337925fcad/unixODBC-2.3.2/Drivers/nn/yylex.c.
Currently, YYERRCODE is defined by yacc.c in an adhoc way as a #define
in the *.c file only. It belongs with the other token kinds.
YYERRCODE is not a nice name, it does not fit in our naming scheme.
YYERROR would be more logical, but it collides with the YYERROR macro.
Shall we keep the same name in all the skeletons? Besides, to avoid
collisions in C, we need to apply the api prefix: YYERRCODE is
actually <PREFIX>ERRCODE. This is not needed in the other languages.
* data/skeletons/bison.m4 (b4_symbol_token_kind): New.
Map the error token to "YYERRCODE".
* data/skeletons/yacc.c (YYERRCODE): Don't define it, it's handled by...
* src/output.c (prepare_symbol_definitions): this.
* tests/input.at (Redefining the error token): Check it.
* data/skeletons/bison.m4 (b4_symbol_kind): Dispatch on the UNDEF
token number rather than its name.
* data/skeletons/c++.m4, data/skeletons/c.m4, data/skeletons/java.m4:
Comment changes.
A forthcoming commit (tokens: properly define the "error" token kind)
revealed a problem in the C++ generated headers: they are not
self-contained. With this file:
%language "c++"
%define api.value.type variant
%code {
static int yylex (yy::parser::semantic_type *lvalp);
}
%token <int> X
%%
exp:
X { printf ("x\n"); }
;
%%
void
yy::parser::error (const std::string& m)
{
std::cerr << m << '\n';
}
static
int yylex (yy::parser::semantic_type *lvalp)
{
static int const input[] = {yy::parser::token::X, 0};
static int toknum = 0;
return input[toknum++];
}
int
main (int argc, char const* argv[])
{
yy::parser p;
return p.parse ();
}
the generated header fails to compile cleanly (foo.cc just #includes
the generated header):
$ clang++-mp-9.0 -c -Wundefined-func-template foo.cc
In file included from foo.cc:1:
bar.tab.hh:550:12: warning: instantiation of function 'yy::parser::basic_symbol<yy::parser::by_type>::basic_symbol' required here, but no definition is available
[-Wundefined-func-template]
struct symbol_type : basic_symbol<by_type>
^
bar.tab.hh:436:7: note: forward declaration of template entity is here
basic_symbol (basic_symbol&& that);
^
bar.tab.hh:550:12: note: add an explicit instantiation declaration to suppress this warning if 'yy::parser::basic_symbol<yy::parser::by_type>::basic_symbol' is explicitly instantiated
in another translation unit
struct symbol_type : basic_symbol<by_type>
^
1 warning generated.
* data/skeletons/c++.m4 (b4_public_types_define): Move the
implementation of the basic_symbol move-ctor to...
(b4_public_types_define): here, its declaration.
* tests/headers.at (Sane headers): Use a declared token so that the
corresponding token constructor is declared. Which triggers the
aforementioned issue.
That's one nice benefit from using enums.
* data/skeletons/lalr1.java (YYSYMBOL_YYEMPTY): No longer define it.
Use 'null' instead.
* examples/java/calc/Calc.y, tests/local.at: Adjust.
* data/skeletons/bison.m4, data/skeletons/c++.m4, data/skeletons/c.m4,
* data/skeletons/glr.cc, data/skeletons/lalr1.cc,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java:
Refer to the "kind" of a symbol, not its "type", where appropriate.
* doc/bison.texi: Replace occurrences of "token type" with "token
kind".
Stop referring to the "macro definitions" of the token kinds, just
name them "definitions".
* maint:
maint: post-release administrivia
version 3.5.4
examples: reccalc: really compile cleanly in C99
news: announce that Bison 3.6 drops YYERROR_VERBOSE
news: update for 3.5.4
style: fix spellos
typo: succesful -> successful
package: improve the readme
java: check and fix support for api.token.raw
java: style: prefer 'int[] foo' to 'int foo[]'
build: fix syntax-check issues
tests: recheck: work properly when the test suite was interrupted
doc: c++: promote api.token.raw
build: fix compatibility with old compilers
examples: reccalc: compile cleanly in C99
The previous fix does not suffice, and actually managed to make things
worse by defining yyscan_t twice in parse.y...
Reported by kencu.
https://trac.macports.org/ticket/59927#comment:29
* examples/c/reccalc/parse.y (yyscan_t): Define it with the same
guards as used by Flex.
The first name is too long. We already have `yypstate`, so
`yypcontext` is ok. We are also migrating to using `*_t` for our
types.
* NEWS, data/skeletons/glr.c, data/skeletons/yacc.c, doc/bison.texi,
* examples/c/bistromathic/parse.y, src/parse-gram.y, tests/local.at:
(yyparse_context_t, yyparse_context_location, yyparse_context_token):
Rename as...
(yypcontext_t, yypcontext_location, yypcontext_token): these.
The Java enums are very different from the C model. As a consequence,
one cannot "build" an enum directly from an integer, we must retrieve
it. That's the purpose of the SymbolType.get class method.
* data/skeletons/java.m4 (b4_symbol_enum, b4_case_code_symbol)
(b4_declare_symbol_enum): New.
* data/skeletons/lalr1.java: Use SymbolType,
SymbolType.YYSYMBOL_YYEMPTY, etc.
* examples/java/calc/Calc.y, tests/local.at: Adjust.
* tests/local.at (AT_LANG_MATCH, AT_YYERROR_DECLARE(java))
(AT_YYERROR_DECLARE_EXTERN(java), AT_PARSER_CLASS): New.
(AT_MAIN_DEFINE(java)): Use AT_PARSER_CLASS.
* tests/scanner.at: Add a test for Java.
* data/skeletons/lalr1.java (yytranslate_): Cast the result.
* data/skeletons/d.m4 (b4_symbol_enum, b4_declare_symbol_enum): New.
* data/skeletons/lalr1.d: Use them.
Use SymbolType, SymbolType.YYSYMBOL_YYEMPTY etc. where appropriate.
(undef_token_, token_number_type, yy_error_token_): Remove.
* tests/local.mk (recheck): Look at the per-test logs, not the overall
log, which, when interrupted, contains only information about... the
tests that passed.
GCC 4.2 dies with
src/InadequacyList.c: In function 'InadequacyList__new_conflict':
src/InadequacyList.c:37: error: #pragma GCC diagnostic not allowed inside functions
src/InadequacyList.c:37: error: #pragma GCC diagnostic not allowed inside functions
src/InadequacyList.c:40: error: #pragma GCC diagnostic not allowed inside functions
Reported by Evan Lavelle.
See https://lists.gnu.org/r/bug-bison/2020-03/msg00021.html
and https://trac.macports.org/ticket/59927.
* src/system.h (GCC_VERSION): New.
Use it to control IGNORE_TYPE_LIMITS_BEGIN and
IGNORE_TYPE_LIMITS_END.
Because of the insane current implementation of glr.cc, things are a
bit nasty. We will rename symbol_number_type as symbol_type_type
later, to keep this commit small.
* data/skeletons/c++.m4 (b4_declare_symbol_enum): New.
Also define YYNTOKENS to avoid type clashes when yyntokens_ was
actually defined in another enum.
Use it.
(symbol_number_type): Be an alias of symbol_type_type.
Use YYSYMBOL_YYEMPTY and the like.
Use symbol_number_type where appropriate.
(empty_symbol): Remove.
(yytranslate_): Use symbol_number_type, not token_number_type.
* data/skeletons/lalr1.cc: Use symbol_number_type where appropriate.
Adjust to the replacement of empty_symbol by YYSYMBOL_YYEMPTY.
(yy_error_token_, yy_undef_token_, yyeof_, yyntokens_): Remove.
Adjust dependencies.
* data/skeletons/glr.cc: Use symbol_number_type where appropriate.
Forward definitions of YYSYMBOL_YYEMPTY, etc. to glr.c.
* tests/headers.at: Accept YYNTOKENS and other YYSYMBOL_*.
* tests/local.at (AT_YYERROR_DEFINE(c++)): Use symbol_number_type.
Now that yacc.c and glr.c both know yysymbol_type_t, convert the
common routines.
* data/skeletons/c.m4 (yydestruct, yy_symbol_value_print)
(yy_symbol_print): Use yysymbol_type_t instead of int.
* data/skeletons/glr.c: Use yySymbol where appropriate.
* data/skeletons/yacc.c (YY_ACCESSING_SYMBOL): New wrapper around
yystos.
Use it.
* tests/local.at (yyreport_syntax_error): Use yysymbol_type_t where
appropriate.
Apply the same changes as in yacc.c. Now yySymbol and yysymbol_type_t
are aliases. We will remove the former later, to avoid cluttering
this commit.
* data/skeletons/glr.c: Use b4_declare_symbol_enum.
Use YYSYMBOL_YYEOF etc. where appropriate.
(YYUNDEFTOK, YYTERROR): Remove.
(YYTRANSLATE, yySymbol, yyexpected_tokens, yysyntax_error_arguments):
Adjust.
(yy_accessing_symbol): New.
Use it where appropriate.
This triggers warnings with several compilers. For instance ICC fills
the logs with pages and pages of
input.c(477): error: a value of type "int" cannot be used to initialize an entity of type "const yysymbol_type_t={yysymbol_type_t}"
0, 2, 2, 2, 2, 2, 2, 2, 2, 2,
^
input.c(477): error: a value of type "int" cannot be used to initialize an entity of type "const yysymbol_type_t={yysymbol_type_t}"
0, 2, 2, 2, 2, 2, 2, 2, 2, 2,
^
And so does G++9 when compiling yacc.c's (C) output
input.c:545:8: error: invalid conversion from 'int' to 'yysymbol_type_t' [-fpermissive]
545 | 0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
| ^
| |
| int
input.c:545:15: error: invalid conversion from 'int' to 'yysymbol_type_t' [-fpermissive]
545 | 0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
| ^
| |
| int
Clang++ is no exception
input.c:545:8: error: cannot initialize an array element of type 'const yysymbol_type_t' with an rvalue of type 'int'
0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
^
input.c:545:15: error: cannot initialize an array element of type 'const yysymbol_type_t' with an rvalue of type 'int'
0, 5, 9, 2, 2, 2, 2, 2, 2, 2,
^
At some point we could use yysymbol_type_t's enumerators to define
yytranslate. Meanwhile...
* data/skeletons/yacc.c (yytranslate): Use the original integral type
to define it.
(YYTRANSLATE): Cast the result into yysymbol_type_t.
Currently we define enumerators only for symbols that have an
identifier. That rules out tokens such as '+', and nonterminals such
as foo-bar and foo.bar. As a consequence we are taking chances: the
compiler might compile yysymbol_type_t as too small an integral type
for some symbol codes.
* data/skeletons/bison.m4 (b4_symbol_sid): Forge a unique symbol
identifier for symbols that don't have an ID.
This is not only cleaner, it also protects us from mixing signed
values (YYEMPTY is #defined as -2) with unsigned types (the
yysymbol_type_t enum is typically compiled as a small unsigned).
For instance GCC 9:
input.c: In function 'yyparse':
input.c:1107:7: error: conversion to 'unsigned int' from 'int'
may change the sign of the result
[-Werror=sign-conversion]
1107 | yyn += yytoken;
| ^~
input.c:1107:10: error: conversion to 'int' from 'unsigned int'
may change the sign of the result
[-Werror=sign-conversion]
1107 | yyn += yytoken;
| ^~~~~~~
input.c:1108:47: error: comparison of integer expressions of
different signedness:
'yytype_int8' {aka 'const signed char'} and
'yysymbol_type_t' {aka 'enum yysymbol_type_t'}
[-Werror=sign-compare]
1108 | if (yyn < 0 || YYLAST < yyn || yycheck[yyn] != yytoken)
| ^~
input.c:702:25: error: operand of ?: changes signedness from 'int'
to 'unsigned int' due to unsignedness of
other operand [-Werror=sign-compare]
702 | #define YYEMPTY (-2)
| ^~~~
input.c:1220:33: note: in expansion of macro 'YYEMPTY'
1220 | yytoken = yychar == YYEMPTY ? YYEMPTY : YYTRANSLATE (yychar);
| ^~~~~~~
input.c:1220:41: error: unsigned conversion from 'int' to
'unsigned int' changes value
from '-2' to '4294967294'
[-Werror=sign-conversion]
1220 | yytoken = yychar == YYEMPTY ? YYEMPTY : YYTRANSLATE (yychar);
| ^
Eventually, it might be interesting to move away from -2 (which is the
only possible negative symbol number) and use the next available
number, to save bits. We could actually even simply use "0" and shift
the rest, which would allow to write "!yytoken" to mean really
"yytoken != YYEMPTY".
* data/skeletons/c.m4 (b4_declare_symbol_enum): Define YYSYMBOL_YYEMPTY.
* data/skeletons/yacc.c: Use it.
* src/parse-gram.y (yyreport_syntax_error): Use YYSYMBOL_YYEMPTY, not
YYEMPTY, when dealing with a symbol.
* tests/regression.at: Adjust.
Now that we have a proper type for internal symbol numbers, let's use
it. More code needs conversion, e.g., printers and destructors, but
they are shared with glr.c, which is not ready yet for this change.
It will also help us deal with warnings such as (GCC9 on GNU/Linux):
input.c: In function 'int yyparse()':
input.c:475:37: error: enumeral and non-enumeral type in conditional expression [-Werror=extra]
475 | (0 <= (YYX) && (YYX) <= YYMAXUTOK ? yytranslate[YYX] : YYSYMBOL_YYUNDEF)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input.c:1024:17: note: in expansion of macro 'YYTRANSLATE'
1024 | yytoken = YYTRANSLATE (yychar);
| ^~~~~~~~~~~
* data/skeletons/yacc.c (yytranslate, yysymbol_name)
(yyparse_context_t, yyexpected_tokens, yypstate_expected_tokens)
(yysyntax_error_arguments):
Use yysymbol_type_t instead of int.
There's a number of advantage in exposing the symbol (internal)
numbers:
- custom error messages can use them to decide how to represent a
given symbol, or a set of symbols.
- we need something similar in uses of yyexpected_tokens. For
instance, currently, bistromathic's completion() reads:
int ntokens = expected_tokens (line, tokens, YYNTOKENS);
[...]
for (int i = 0; i < ntokens; ++i)
if (tokens[i] == YYTRANSLATE (TOK_VAR))
[...]
else if (tokens[i] == YYTRANSLATE (TOK_FUN))
[...]
else
[...]
- now that it's a compile-time expression, we can easily build static
tables, switch, etc.
- some users depended on the ability to get the token number from a
symbol to write test cases for their scanners. But Bison 3.5
removed the table this feature depended upon (a reverse
yytranslate). Now they can check against the actual symbol number,
without having pay (space and time) a conversion.
See https://lists.gnu.org/r/bug-bison/2020-01/msg00001.html, and
https://lists.gnu.org/archive/html/bug-bison/2020-03/msg00015.html.
- it helps us clearly separate the internal symbol numbers from the
external token numbers, whose difference is sometimes blurred in the
code when values coincide (e.g. "yychar = yytoken = YYEOF").
- it allows us to get rid of ugly macros with inconsistent names such
as YYUNDEFTOK and YYTERROR, and to group related definitions
together.
- similarly it provides a clean access to the $accept symbol (which
proves convenient in a current experimentation of mine with several
%start symbols).
Let's declare this type as a private type (in the *.c file, not
the *.h one). So it does not need to be influenced by the api prefix.
* data/skeletons/bison.m4 (b4_symbol_sid): New.
(b4_symbol): Use it.
* data/skeletons/c.m4 (b4_symbol_enum, b4_declare_symbol_enum): New.
* data/skeletons/yacc.c: Use b4_declare_symbol_enum.
(YYUNDEFTOK, YYTERROR): Remove.
Use the corresponding symbol enum instead.
* tests/local.mk (recheck): Look at the per-test logs, not the overall
log, which, when interrupted, contains only information about... the
tests that passed.
* data/skeletons/lalr1.java (yysyntaxErrorArguments): Move it from the
context, to the parser object.
Generate only for detailed and verbose error messages.
* tests/local.at (AT_YYERROR_DEFINE(java)): Use yyexpectedTokens
instead.
We could just "inline yysyntax_error_arguments back" in the routines
it was originally extracted from, but I think the code is nicer to
read this way.
* data/skeletons/glr.c (yysyntax_error_arguments): Generate only for
detailed and verbose error messages.
* data/skeletons/yacc.c: Likewise.
* data/skeletons/lalr1.cc (parser::context::yysyntax_error_arguments):
Move as...
(parser::yysyntax_error_arguments_): this.
And only for detailed and verbose error messages.
Because glr.c shares the same testing routines, we also need to
convert it.
* data/skeletons/glr.c (yyparse_context_token): New.
* tests/local.at (yyreport_syntax_error): here.
Suggested by Adrian Vogelsgesang.
https://lists.gnu.org/archive/html/bison-patches/2020-02/msg00069.html
* data/skeletons/lalr1.java (Context.EMPTY, Context.getToken): New.
(Context.yyntokens): Rename as...
(Context.NTOKENS): this.
Because (i) all the Java coding styles recommend upper case for
constants, and (ii) the Java Skeleton exposes Lexer.EOF, not
Lexer.YYEOF.
* data/skeletons/yacc.c (yyparse_context_token): New.
* examples/c/bistromathic/parse.y (yyreport_syntax_error): Don't use
yysyntax_error_arguments.
* examples/java/calc/Calc.y (yyreportSyntaxError): Likewise.
yyparse returns 0, 1, 2 since ages (accept, reject, memory exhausted).
Some of our auxiliary functions such as yy_lac and
yyreport_syntax_error also need to return error codes and also use 0,
1, 2. Because it uses yy_lac, yyexpected_tokens also needs to return
"problem", "memory exhausted", but in case of success, it needs to
return the number of tokens, so it cannot use 1 and 2 as error code.
Currently it uses -1 and -2, which is later converted into 1 and 2 as
yacc.c expects it.
Let's simplify this and use consistently -1 and -2 for auxiliary
functions that are not exposed (or not yet exposed) to the user. In
particular this will save the user from having to convert
yyexpected_tokens's -2 into yyreport_syntax_error's 2: both return -1
or -2.
* data/skeletons/yacc.c (yy_lac, yyreport_syntax_error)
(yy_lac_stack_realloc): Return -1, -2 for errors instead of 1, 2.
Adjust callers.
* examples/c/bistromathic/parse.y (yyreport_syntax_error): Do take
error codes into account.
Issue a syntax error message even if we ran out of memory.
* src/parse-gram.y, tests/local.at (yyreport_syntax_error): Adjust.
The cost of the file layer is large and makes benchmarks too coarse,
as seen for in following example, first with a file, then with a
literal string:
0. %skeleton "yacc.c" %define parse.lac full
1. %skeleton "yacc-v1.c" %define nofinal %define parse.lac full
2. %skeleton "yacc-v2.c" %define nofinal %define parse.lac full
3. %skeleton "yacc-v3.c" %define nofinal %define parse.lac full
4. %skeleton "yacc.c"
5. %skeleton "yacc-v1.c" %define nofinal
6. %skeleton "yacc-v2.c" %define nofinal
7. %skeleton "yacc-v3.c" %define nofinal
--------------------------------------------------
Benchmark Time CPU Iterations
--------------------------------------------------
BM_y0 32558 ns 32537 ns 21228
BM_y1 32400 ns 32369 ns 21233
BM_y2 33485 ns 33464 ns 20625
BM_y3 32139 ns 32125 ns 21446
BM_y4 31343 ns 31329 ns 21747
BM_y5 31344 ns 31317 ns 22035
BM_y6 31287 ns 31255 ns 22039
BM_y7 31387 ns 31373 ns 22178
--------------------------------------------------
Benchmark Time CPU Iterations
--------------------------------------------------
BM_y0 10642 ns 10634 ns 63601
BM_y1 10657 ns 10654 ns 63625
BM_y2 10441 ns 10432 ns 65957
BM_y3 10558 ns 10554 ns 64546
BM_y4 9521 ns 9516 ns 72011
BM_y5 9179 ns 9157 ns 75028
BM_y6 9360 ns 9356 ns 73770
BM_y7 9365 ns 9359 ns 72609
Of course, at the same time it is less realistic: most users read
files rather that strings, so it might lead to us to pay attention to
costs most people don't see.
* etc/bench.pl.in (&calc_input): Output into a file given as argument.
Output in C syntax.
(&generate_grammar_calc): Use it.
Simplify the grammar: remove operators we don't care about.
Rewrite the scanner to work on a char* instead of a FILE*.
* etc/bench.pl.in (&bench_with_timethese): Also use y$i, as in
&bench_with_gbenchmark.
(&generate_grammar_calc): Don't add a prefix, let the callers do it.
* etc/bench.pl.in (&compiler): New, extracted from...
(&compile): here.
Don't link when using gbm.
(&calc_input): Don't make massive input for micro
benchmarks.
(&generate_grammar_calc): When using gbm, use api.prefix to avoid name
collisions.
Be ready to issue BENCHMARKS instead of a main.
(&bench): Rename as...
(&bench_with_timethese): this.
(&bench_with_gbenchmark): New.
(&bench): New.
Dispatch on these two.
In push parsers, when asking for the list of expected tokens at some
point, it makes no sense to build a yyparse_context_t: the yypstate
alone suffices (the only difference being the lookahead). Instead of
forcing the user to build a useless shell around yypstate, let's offer
yypstate_expected_tokens.
See https://lists.gnu.org/r/bison-patches/2020-03/msg00025.html.
* data/skeletons/yacc.c (yypstate): Declare earlier, so that we can
use it for...
(yypstate_expected_tokens): this new function, when in push parsers.
Adjust dependencies.
* examples/c/bistromathic/parse.y: Simplify: use
yypstate_expected_tokens.
Style fixes.
Reduce scopes (reported by Joel E. Denny).
* upstream/maint:
maint: post-release administrivia
version 3.5.3
news: update for 3.5.3
yacc.c: make sure we properly propagated the user's number for error
diagnostics: don't crash because of repeated definitions of error
style: initialize some struct members
diagnostics: beware of zero-width characters
diagnostics: be sure to close the styling when lines are too short
muscles: fix incorrect decoding of $
code: be robust to reference with invalid tags
build: fix typo
doc: update recommandation for libtextstyle
style: comment changes
examples: use consistently the GFDL header for readmes
style: remove useless declarations
typo: succesful -> successful
README: point to tests/bison, and document --trace
gnulib: update
maint: post-release administrivia
According to https://www.unix.com/man-page/POSIX/1posix/yacc/, the
user is allowed to specify her user number for the error token:
The token error shall be reserved for error handling. The name
error can be used in grammar rules. It indicates places where the
parser can recover from a syntax error. The default value of error
shall be 256. Its value can be changed using a %token
declaration. The lexical analyzer should not return the value of
error.
I think this feature is useless, the user should not have to deal with
that. The intend is probably to give the user a means to use 256 if
she wants to, but provided "error" cleared the path first by being
assigned another number. In the case of Bison, 256 is assigned to
"error" at the end if the user did not use it for a token of hers. So
this feature is useless.
Yet it is valid, and if the user assigns twice a token number to
"error", then the second time we want to complain about it and want to
show the original definition. At this point, we try to display the
built-in definition of "error", whose location is NULL, and we crash.
Rather, the location of the first user definition of "error" should
become its defining location.
Reported byg Ahcheong Lee.
https://lists.gnu.org/r/bug-bison/2020-03/msg00007.html
* src/symtab.c (symbol_class_set): If this is a declaration and the
symbol was not declared yet, keep this as defining location.
* tests/input.at (Redefining the error token): New.
Currenly we rely on (visual) width of the characters to decide where
to open and close the styling of the quoted lines. This breaks when
we deal with zero-width characters: we cannot just rely on (visual)
columns, we need to know whether we are before, inside, or after the
highlighted portion.
* src/location.c (location_caret): col_end: no longer add 1, "regular"
characters have a width of 1, only 0-width characters have 0-width.
opened: replace with 'state', a three-valued enum.
Don't reopen the style if we already did.
* tests/diagnostics.at (Zero-width characters): New.
bar.y:4.12-17: <error>error:</error> redefining user token number of foo
- 4 | %token foo <error>123
+ 4 | %token foo <error>123</error>
| <error>^~~~~~</error>
* src/location.c (location_caret): Be sure to close.
* tests/diagnostics.at (Line is too short, and then you die): New.
Because we want to support $<a->b>$, we must accept -> in type tags,
and reject $<->$, as it is unfinished.
Reported by Ahcheong Lee.
* src/scan-code.l (yylex): Make sure "tag" does not end with -, since
-> does not close the tag.
* tests/input.at (Stray $ or @): Check this.
Currently pstate_new does not set up its variables, this task is left
to yypush_parse. This was probably to share more code with usual pull
parsers, where these (local) variables are indeed initialized by
yyparse.
But as a consequence yyexpected_tokens crashes at the very beginning
of the parse, since, for instance, the stacks are not even set up.
See https://lists.gnu.org/r/bison-patches/2020-03/msg00001.html.
The fix could have very simple, but the documentation actually makes
it very clear that we can reuse a pstate for several parses:
After yypush_parse returns a status other than YYPUSH_MORE, the
parser instance yyps may be reused for a new parse.
so we need to restore the parser to its pristine state so that (i) it
is ready to run the next parse, (ii) it properly supports
yyexpected_tokens for the next run.
* data/skeletons/yacc.c (b4_initialize_parser_state_variables): New,
extracted from the top of yyparse/yypush_parse.
(yypstate_clear): New.
(yypstate_new): Use it when push parsers are enabled.
Define after the yyps macros so that we can use the same code as the
regular pull parsers.
(yyparse): Use it when push parsers are _not_ enabled.
* examples/c/bistromathic/bistromathic.test: Check the completion on
the beginning of the line.
Currently completion on "at" proposes only "atan", but does not
actually complete "at" into "atan".
* examples/c/bistromathic/parse.y (completion): Install the lcp in
matches[0].
* examples/c/bistromathic/bistromathic.test: Check that case.
Currently "(1+<TAB>" does not work as expected, because "+" is not a
word breaking character.
* examples/c/bistromathic/parse.y (init_readline): Specify our word
breaking characters.
* examples/c/bistromathic/bistromathic.test: Avoid trailing spaces.
These macros have been extremely useful when we had to support K&R C,
which we dropped long ago. Now, they merely make the code uselessly
hard to read.
* data/skeletons/c.m4, data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/yacc.c:
Stop using b4_function_define.
Let's use GNU readline and its TAB autocompletion to demonstrate the
use of yyexpected_tokens.
This shows a number of weaknesses in our current approach:
- some macros (yyssp, etc.) from push parsers "leak" in user code, we
need to undefine them
- the context needed by yyexpected_tokens does not need the token,
yypstate actually suffices
- yypstate is not properly setup when first allocated, which results
in a crash of yyexpected_tokens if fired before a first token was
read. We should move initialization from yypush_parse into
yypstate_new.
* examples/c/bistromathic/parse.y (yylex): Take input as a string, not
a file.
(EXIT): New token.
(input): Adjust to work only on a line.
(line): Remove.
(symbol_count, process_line, expected_tokens, completion)
(init_readline): New.
* examples/c/bistromathic/bistromathic.test: Adjust expectations.
This example will soon use GNU readline, so its scanner should be easy
to use (concurrently) on strings, not streams. This is not a place
where Flex shines, and anyway, these are examples of Bison, not Flex.
There's already lexcalc and reccalc that demonstrate the use of Flex.
* examples/c/bistromathic/scan.l: Remove.
* examples/c/bistromathic/parse.y (yylex): New.
Adjust dependencies.
The bistromathic example should not use Flex, it makes it too complex.
But it was the only example to show location tracking with Flex.
* examples/c/lexcalc/lexcalc.test, examples/c/lexcalc/parse.y,
* examples/c/lexcalc/scan.l: Demonstrate location tracking as is done
in bistromathic.
The current implementation of parser::context keeps a copy of the
lookahead. This is troublesome since we support move-only types.
Besides, while GCC is happy with the current implementation, Clang
complains that the ctor it needs to build the copy of the lookahead is
not yet available.
461. calc.at:1120: testing Calculator C++ %defines %locations parse.error=verbose %name-prefix "calc" %verbose ...
calc.at:1120: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -Wno-deprecated -o calc.cc calc.y
calc.at:1120: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o calc calc.cc calc-lex.cc calc-main.cc $LIBS
stderr:
In file included from calc-lex.cc:7:
calc.hh:351:12: error: instantiation of function 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' required here, but no definition is available [-Werror,-Wundefined-func-template]
struct symbol_type : basic_symbol<by_type>
^
calc.hh:273:7: note: forward declaration of template entity is here
basic_symbol (const basic_symbol& that);
^
calc.hh:351:12: note: add an explicit instantiation declaration to suppress this warning if 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' is explicitly instantiated in another translation unit
struct symbol_type : basic_symbol<by_type>
^
1 error generated.
In file included from calc-main.cc:7:
calc.hh:351:12: error: instantiation of function 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' required here, but no definition is available [-Werror,-Wundefined-func-template]
struct symbol_type : basic_symbol<by_type>
^
calc.hh:273:7: note: forward declaration of template entity is here
basic_symbol (const basic_symbol& that);
^
calc.hh:351:12: note: add an explicit instantiation declaration to suppress this warning if 'calc::parser::basic_symbol<calc::parser::by_type>::basic_symbol' is explicitly instantiated in another translation unit
struct symbol_type : basic_symbol<by_type>
^
1 error generated.
stdout:
calc.at:1120: exit code was 1, expected 0
461. calc.at:1120: 461. Calculator C++ %defines %locations parse.error=verbose %name-prefix "calc" %verbose (calc.at:1120): FAILED (calc.at:1120)
* data/skeletons/lalr1.cc (context::yyla_): Make it a const-ref.
Move the implementation out of the declaration.
Address compiler warnings such as
warning: declaration of 'yyla' shadows a member of 'yy::parser::context' [-Wshadow]
* data/skeletons/lalr1.cc (context): Don't use the same names for
variables and members.
Use foo_ for private members, as in parser.
Also, use the + trick in array accesses to please ICC and provide it
with an int.
* data/skeletons/lalr1.cc: added support here
* tests/calc.at: added test cases
* tests/local.at: added yyreport_syntax_error implementation
for C++ test cases
Prefer b4_parse_error_case over the adhoc solution
`m4_case + b4_percent_define_get`. Same for b4_parse_error_bmatch.
* data/skeletons/glr.c: here
* data/skeletons/yacc.c: here
* etc/bench.pl.in: Adjust to the current use of %define's values.
Don't use %error-verbose.
Prefer Bison to CPP (e.g., api.value.type).
Avoid returning characters directly, so that %define api.token.raw works.
* doc/bison.texi (Parser Internationalization): Move most of its
content into...
(Enabling I18n): this new node.
(Token I18n): New.
(Token Decl): Refer to token internationalization.
(Error Reporting Function): Promote parse.error detailed.
Since Bison 2.7, output was indented four spaces for explanatory
statements. For example:
input.y:2.7-13: error: %type redeclaration for exp
input.y:1.7-11: previous declaration
Since the introduction of caret-diagnostics, it became less clear.
Remove the indentation and display submessages as in GCC:
input.y:2.7-13: error: %type redeclaration for exp
2 | %type <float> exp
| ^~~~~~~
input.y:1.7-11: note: previous declaration
1 | %type <int> exp
| ^~~~~
* src/complain.h (SUB_INDENT): Remove.
(warnings): Add "note" to the enum.
* src/complain.h, src/complain.c (complain_indent): Replace by...
(subcomplain): this.
Adjust all dependencies.
* tests/actions.at, tests/diagnostics.at, tests/glr-regression.at,
* tests/input.at, tests/named-refs.at, tests/regression.at:
Adjust expectations.
bison 3.5.2
* tag 'v3.5.2':
version 3.5.2
news: 3.5.2
gnulib: update
doc: update Doxygen template
java: avoid trailing white spaces
m4: fix b4_token_format
doc: clearly state that %yacc only makes sense with yacc.c
doc: spell check
examples: be more robust to spaces in paths
larlr1.cc: Reject unsupported values for parse.lac
We used to emit:
/** Token number,to be returned by the scanner. */
static final int NUM = 258;
/** Token number,to be returned by the scanner. */
static final int NEG = 259;
with no space after the comma. Fix that.
* data/skeletons/bison.m4 (b4_token_format): Quote where appropriate.
This revealed a number of things I had not realized:
- the Java location tracking was aliasing the same pair of positions
for all the symbols (see previous commit).
- in impure parsers, it's quite easy to use incorrect locations for
diagnostics, since yyerror uses yylloc, which is the location of the
lookahead, not that of the current lhs. So we need something like
{
YYLTYPE old_yylloc = yylloc;
yylloc = @$;
yyerror (]AT_PARAM_IF([result, count, nerrs, ])[buf);
yylloc = old_yylloc;
}
Maybe we should do that little yylloc dance in the skeleton instead
of leaving it to the user? It might be costly... But that's only
for users of the impure parsers, which are asking for trouble
anyway.
- in glr.cc invoking yyerror is somewhat cumbersome: the C++ interface
is not available as we are in yyparse (which in C), and yyerror is
used by glr.cc itself to bind it to the user's parser::error. If we
call yyerror, we need:
yyerror (]AT_LOCATION_IF([[&@$, ]])[yyparser, ]AT_PARAM_IF([result, count, nerrs, ])[msg);
However calling yy::parser::error is easier, once we know that the
current parser object is available as 'yyparser'. Which also saves
us from having to pass the parse-params ourselves:
yyparser.error (]AT_LOCATION_IF([[@$, ]])[msg);
* tests/calc.at: Invoke yyerror by hand, instead of using fprintf etc.
Adjust expectations.
* data/skeletons/lalr1.java (Context): Make data members private.
(Context.getLocation): New.
* examples/java/calc/Calc.y, tests/java.at, tests/local.at: Adjust.
* doc/bison.texi (Tokens from Literals): Move to code using
%token-table to...
(Decl Summary: %token-table): here.
* data/skeletons/bison.m4: Implement mutual exclusion.
* tests/input.at: Check it.
* doc/local.mk: Be robust to the removal of doc/.
With texinfo.tex 2019-09-24.13, node names with + are not properly
handled.
https://lists.gnu.org/r/bug-texinfo/2020-02/msg00004.html
* doc/bison.texi: Always use the three-argument form for references to
node with a + in the name.
This reverts commit ebab1ffca8.
This commit removed "useless" initializers, going from
/* YYPACT[STATE-NUM] -- Index in YYTABLE of the portion describing
STATE-NUM. */
private static final byte yypact_[] = yypact_init ();
private static final byte[] yypact_init ()
{
return new byte[]
{
25, -7, -8, 37, -8, 40, -8, 20, -8, 61,
-8, -8, 3, 9, 51, -8, -8, -2, -2, -2,
-2, -2, -2, -8, -8, -8, 1, 66, 66, 3,
3, 3
};
}
to
/* YYPACT[STATE-NUM] -- Index in YYTABLE of the portion describing
STATE-NUM. */
private static final byte[] yypact_ =
{
25, -7, -8, 37, -8, 40, -8, 20, -8, 61,
-8, -8, 3, 9, 51, -8, -8, -2, -2, -2,
-2, -2, -2, -8, -8, -8, 1, 66, 66, 3,
3, 3
};
But it turns out that this was on purpose, to work around the 64KB
limitation in JVM methods. It was introduced on the 2008-11-10 by
Di-an Jan in 09ccae9b18: "Work around
Java's ``code too large'' problem for parser tables". See
https://lists.gnu.org/r/help-bison/2008-11/msg00004.html. A real
test, where we would hit the JVM limitation, would be nice.
To avoid further regressions, add comments.
parse.error has more than two possible values.
* data/skeletons/bison.m4 (b4_error_verbose_if, b4_error_verbose_flag):
Remove.
(b4_parse_error_case, b4_parse_error_bmatch): New.
Adjust dependencies.
* data/skeletons/lalr1.java (yyexpectedTokens)
(yysyntaxErrorArguments): Make them methods of Context.
(Context.yysymbolName): New.
* tests/local.at: Adjust.
The error format should be translated, but contrary to the case of
C/C++, we cannot just depend on macros to adapt on the
presence/absence of '_'. Let's consider that the message format is to
be translated iff there are some internationalized tokens.
* src/output.c (prepare_symbol_names): Define b4_has_translations.
* data/skeletons/java.m4 (b4_trans): New.
* data/skeletons/lalr1.java: Use it to emit translatable or not the
format string.
And allow syntax error messages for 'detailed' and 'verbose' to be
translated.
* data/skeletons/lalr1.java (Context, yyexpectedTokens)
(yysyntaxErrorArguments): New.
(yysyntax_error): Use it.
In Java there is no need for N_ and yytranslate_. So instead of
hard-coding the use of N_ in the table of the symbol names, rely on
b4_symbol_translate.
* src/output.c (prepare_symbol_names): Use b4_symbol_translate instead
of N_.
* data/skeletons/c.m4 (b4_symbol_translate): New.
* data/skeletons/lalr1.java (yysymbolName): New.
Use it.
* examples/java/calc/Calc.y: Use parse.error=detailed.
* tests/calc.at: Check parse.error=detailed.
We used to emit:
/** Token number,to be returned by the scanner. */
static final int NUM = 258;
/** Token number,to be returned by the scanner. */
static final int NEG = 259;
with no space after the comma. Fix that.
* data/skeletons/bison.m4 (b4_token_format): Quote where appropriate.
Unfortunately in the Java skeleton the user cannot override the way
locations are displayed, and locations don't know the structure of the
positions. So they cannot implement the tricks used in the C/C++
skeletons to display "1.1" instead of "1.1-1.2".
* tests/local.at (Java): Add support for column tracking in the
locations, as we did in examples/java/calc.
* tests/calc.at: Use AT_CALC_YYLEX.
Soon calculator tests for Java will move from java.at to calc.at.
Which implies improving the Java testing infrastructure in
local.at (for instance really tracking columns in positions, not just
token number). Detach java.at from local.at.
* tests/java.at (AT_JAVA_POSITION_DEFINE_OLD): New.
Use it.
* examples/java/calc/Calc.y: The StreamTokenizer cannot "peek" for the
next character, it reads it, and keeps it for the next call. So the
current location is one passed the end of the current token. To avoid
this, keep the previous position, and use it to end the current token.
* examples/java/calc/Calc.test: Adjust.
The C, C++ and D skeletons used to show the stack right after popping
the stack during the reduction. Now that the stack is printed after
reaching a new state, that has become useless:
Entering state 1
Stack now 0 1
Reducing stack by rule 5 (line 83):
$1 = token "number" (1)
-> $$ = nterm exp (1)
Stack now 0
Entering state 8
Stack now 0 8
Remove the "Stack now 0" line.
* data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java, data/skeletons/yacc.c:
Here.
Currently, if we have long rules and series of shift, we stack states
without showing stack. Let's be more incremental, and do how the Java
skeleton does.
* data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/yacc.c:
Here.
Adjust test cases.
* tests/torture.at (AT_DATA_STACK_TORTURE): Disable stack traces: this
test produces a very large stack, and showing the stack each time we
shift a token goes quadatric.
The Java skeleton displays
Reading a token:
Next token is token "number" (1)
while the other display
Reading a token: Next token is token "number" (1)
When generating logs in the scanner, the first part is separated from
the second, and the end of the scanner logs have the second part
pasted in. So let's propagate the Java way, but with the colon.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java, data/skeletons/yacc.c: Do it.
Adjust test cases and doc.
* tests/local.at (AT_LANG_MATCH): New.
(AT_YYERROR_DECLARE(java), AT_YYERROR_DECLARE_EXTERN(java)): New.
* tests/calc.at: The grammar file for Java is quite different for the
others, and continuing to assemble it from pieces makes the grammar
file hard to understand. Let's also dispatch on the language to
assemble it, and isolate Java from the others.
Most of this comes from java.at.
This example, so far, was tracking the current token number, not the
current column number. This is not nice for an example...
* examples/java/Calc.y (PositionReader): New.
Use it.
* examples/java/Calc.test: Check the output.
AFAICT, autoboxing/unboxing was added in Java 5 (September 30, 2004).
I think we can afford to use it. It should help us merge some Java
tests with the main ones.
However, beware that != does not unbox: it compares the object
addresses.
* examples/java/Calc.y, tests/java.at: Simplify.
* examples/java/Calc.test, tests/java.at: Improve tests.
When building the test cases, emitting code in the epilogue is very
constraining. Let's make it simpler thanks to %code epilogue.
However, I don't want to document this: it is bad style to use it (we
should avoid having too many ways to write the same thing,
TI!MTOWTDI), just put your code in the true epilogue section.
* data/skeletons/glr.c, data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/yacc.c: Implement support for %code epilogue.
Remove useless comments.
* tests/calc.at, tests/java.at: Simplify.
Modeled after what was done in yacc.c, yet somewhat different since
yyGLRStack play the role of the full yyparse_context_t. It will
probably be defined as a alias, to make interfaces compatible.
LAC is not supported here. And is somewhat tricky.
* data/skeletons/glr.c (yyexpected_tokens, yysyntax_error_arguments): New.
* data/skeletons/glr.c: Move the handling of error messages after the
definition of types. Especially after yyGLRStack, which we will need
in forthcoming patches.
Simplify: yyGLRStack is defined at this point.
The PDF output is more consistent: some nodes were not pointed to with
their title. The HTML output becomes "see section Foo" instead of
"see Foo", but this should be addressed in the next Texinfo release.
Info output is simplified, as it uses only the node name and not its
title. But it's considered easier to read this way.
See https://lists.gnu.org/r/help-texinfo/2020-01/msg00031.html.
* doc/bison.texi: Set @xrefautomaticsectiontitle on.
Simplify all uses of ref.
Add an example to demonstrate the use of push parser. I'm pleasantly
surprised: parse.error=detailed works like a charm with push parsers.
* examples/c/local.mk, examples/c/pushcalc/Makefile
* examples/c/pushcalc/README.md, examples/c/pushcalc/calc.test,
* examples/c/pushcalc/calc.y, examples/c/pushcalc/local.mk:
New.
* examples/c/calc/calc.y: Restore to its original state, with
parse.error=detailed instead of parse.error=custom (this example
should be simple).
* examples/c/calc/calc.test: Check syntax errors.
* examples/c/lexcalc/parse.y: Add comments.
We used to display the unexpected token first:
$ bison foo.y
foo.y:1.8-13: error: syntax error, unexpected %token, expecting character literal or identifier or <tag>
1 | %token %token
| ^~~~~~
GCC uses a different format:
$ gcc-mp-9 foo.c
foo.c:1:5: error: expected identifier or '(' before ')' token
1 | int()()()
| ^
and so does Clang:
$ clang-mp-9.0 foo.c
foo.c:1:5: error: expected identifier or '('
int()()()
^
1 error generated.
They display the unexpected token last (or not at all). Also, they
don't waste width with "syntax error". Let's try that. It gives, for
the same example as above:
$ bison foo.y
foo.y:1.8-13: error: expected character literal or identifier or <tag> before %token
1 | %token %token
| ^~~~~~
* src/complain.h, src/complain.c (syntax_error): New.
* src/parse-gram.y (yyreport_syntax_error): Use it.
As a test case, support translations in Bison itself.
* src/parse-gram.y: Mark the translatable tokens.
While at it, use clearer names.
* tests/input.at: Adjust expectations.
Don't force callers of location_caret to have to deal with flags that
disable it.
* src/location.h, src/location.c (location_caret)
(location_caret_suggestion): Early return if disabled.
* src/complain.c: Simplify.
Just as the yacc.c skeleton, the lalr1.cc skeleton should reject
invalid values for parse.lac.
* data/skeletons/lalr1.cc: check validity of parse.lac
* tests/input.at: new test cases
Just as the yacc.c skeleton, the lalr1.cc skeleton should reject
invalid values for parse.lac.
* data/skeletons/lalr1.cc: check validity of parse.lac
* tests/input.at: new test cases
Some users would like to avoid having to "parse" the *.y file to find
the strings to translate. Let's issue the translatable tokens with N_
to allow "parsing" the generated parsers instead.
See
https://lists.gnu.org/archive/html/bison-patches/2019-01/msg00015.html
* src/output.c (prepare_symbol_names): Issue symbol_names with N_()
markup.
In addition to
%token NUM "number"
accept
%token NUM _("number")
in which case the token will be translated in error messages.
Do not use _() in the output if there are no translatable tokens.
* src/symtab.h, src/symtab.c (symbol): Add a 'translatable' member.
* src/parse-gram.y (TSTRING): New token.
(string_as_id.opt): Replace with...
(alias): this.
Use it.
* src/scan-gram.l (SC_ESCAPED_TSTRING): New start conditions, to match
TSTRINGs.
* src/output.c (prepare_symbols): Define b4_translatable if there are
translatable strings.
* data/skeletons/glr.c, data/skeletons/lalr1.cc,
* data/skeletons/yacc.c (yytnamerr): Receive b4_translatable, and use it.
* src/output.c (escape_trigraphs, xescape_trigraphs): New.
(prepare_symbol_names): Use it.
* tests/regression.at: Check the handling of trigraphs with
parse.error = detailed.
"detailed" error messages are almost like "verbose", except that we
don't double escape them, they don't get inner quotes, we don't use
yytnamerr, and we hide the table.
"custom" is exposed with the "detailed" tokens, not the "verbose"
ones: they are not double-quoted.
Because there's a risk that some people use yytname even without
"verbose", let's keep yytname (instead of yys_name) in "simple"
parse.error.
* src/output.c (prepare_symbol_names): Be ready to output symbol names
unquoted.
(prepare_symbol_names): Output both the old tname table, and the new
symbol_names one.
* data/skeletons/bison.m4: Accept 'detailed'.
* data/skeletons/yacc.c: When parse.error is 'detailed', don't emit
yytname and yytnamerr, just yysymbol_name with the table inside.
* tests/calc.at: Adjust.
Only parse.error verbose and simple will get the original yytname: the
other options will rely on a different table. So let's move on top of
the yysymbol_name function.
* data/skeletons/c.m4 (yy_symbol_print): Use yysymbol_name.
* data/skeletons/glr.c (yytokenName): Rename as...
(yysymbol_name): this.
The change of naming scheme is unfortunate, but it's definitely glr.c
which is "wrong".
Currently yy_symbol_print is defined before yytokenName, although it
should use it instead of read yytname directly. Move blocks around to
avoid this.
* data/skeletons/glr.c (yy_symbol_print): Move its definition after
that of yytokenName.
See https://github.com/nemequ/icc-travis/issues/15.
Thanks to Jeff Hammond and Evan Nemerson for their help.
* configure.ac (warn_common): Disable dubious warnings.
* .travis.yml: Use ICC again.
Be robust to newer versions of Autoconf where the package URL defaults
to https instead of http.
* configure.ac (AC_INIT): Use https.
* tests/report.at: Adjust expected output s/http/https/
to match updated URL.
Currently we get warnings with GCC 4.8 when running the
maintainer-check-g++ tests:
143. skeletons.at:85: testing Installed skeleton file names ...
../../tests/skeletons.at:120: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret --skeleton=yacc.c -o input-cmd-line.c input-cmd-line.y
../../tests/skeletons.at:121: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o input-cmd-line input-cmd-line.c $LIBS
stderr:
input-cmd-line.c: In function 'int yysyntax_error(long int*, char**, const yyparse_context_t*)':
input-cmd-line.c:977:52: error: conversion to 'int' from 'long int' may alter its value [-Werror=conversion]
YYSIZEOF (yyarg) / YYSIZEOF (*yyarg));
^
cc1plus: all warnings being treated as errors
stdout:
../../tests/skeletons.at:121: exit code was 1, expected 0
and
429. calc.at:823: testing Calculator parse.error=custom %locations api.prefix={calc} ...
../../tests/calc.at:823: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -Wno-deprecated -o calc.c calc.y
../../tests/calc.at:823: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o calc calc.c $LIBS
stderr:
calc.y: In function 'int yyreport_syntax_error(const yyparse_context_t*)':
calc.y:157:58: error: conversion to 'int' from 'long unsigned int' may alter its value [-Werror=conversion]
int n = yysyntax_error_arguments (ctx, arg, sizeof arg / sizeof *arg);
^
cc1plus: all warnings being treated as errors
stdout:
../../tests/calc.at:823: exit code was 1, expected 0
We could use a cast to avoid the warning, but it becomes too
cluttered. We can also use YYPTRDIFF_T, but that forces the user to
use YYPTRDIFF_T too, although this is an array of tokens, which is
limited by YYNTOKENS, an int. So let's completely avoid this warning.
* data/skeletons/yacc.c, tests/local.at (yyreport_syntax_error): Avoid
relying on sizeof to compute the array capacity.
Enhance the calculator tests: show that passing arguments to yyerror
works.
* tests/calc.at: Add a new parse-param, nerrs, which counts the number
of syntax errors in a run.
* tests/local.at: Adjust to handle the new 'nerrs' argument, when
present.
The custom error reporting function show sees the user's additional
arguments. Let's experiment with passing them as arguments to
yyreport_syntax_error, but maybe storing them in the context would be
a bettter alternative.
* data/skeletons/yacc.c (yyreport_syntax_error): Handle the
parse-params.
* tests/calc.at, tests/local.at: Adjust.
Currently the parse-params are tested in calc.at by checking that the
global variable and the parse-params have the same value. But it does
not check that value, that could remain being 0 just as well.
* tests/calc.at: Don't define the params when they are not used.
Check the final value of result and count.
Also, do count the number of line of logs.
Provide users with a means to query for the currently allowed tokens.
Could be used for autocompletion for instance.
* data/skeletons/yacc.c (yyexpected_tokens): New, extracted from
yysyntax_error_arguments.
* examples/c/calc/calc.y (PRINT_EXPECTED_TOKENS): New.
Use it.
We use a different format to check parse.error custom. Compute the
"verbose" one from it instead of forcing the test author to provide
the various formats of expected error messages.
* tests/calc.at (_AT_CHECK_CALC_ERROR): Handle this transformation
when needed.
Simplify callers.
* tests/local.at (AT_ERROR_CUSTOM_IF, AT_ERROR_VERBOSE_IF)
(AT_ERROR_SIMPLE_IF): New.
(AT_YYERROR_DEFINE(c)): Generate yyreport_syntax_error.
* tests/calc.at (_AT_CHECK_CALC_ERROR): Accept custom error messages
as additional test case.
Use it.
Add a new test case for %define parse.error custom.
When parse.error is custom, let users define a yyreport_syntax_error
function, and use it.
* data/skeletons/bison.m4 (b4_error_verbose_if): Accept 'custom'.
* data/skeletons/yacc.c: Implement it.
* examples/c/calc/calc.y: Experiment with it.
That allows users to cover more cases, such as easily filtering some
arguments they don't want to expose. But they now have to call
yysymbol_name explicitly.
* data/skeletons/yacc.c (yysyntax_error_arguments, yysyntax_error):
Deal with symbol numbers instead of symbol names.
Isolate a function that returns the list of expected and unexpected
tokens. It will be exposed to users willing to customize their error
messages.
* data/skeletons/yacc.c (yyparse_context_t): New.
(yyerror_message_arguments): New, extracted from yysyntax_error.
When not empty, AT_PARSE_PARAMS was guaranteed to end with a comma.
Remove the trailing comma, so that we can use AT_PARSE_PARAMS at the
end of the arguments, not only at the beginning.
* tests/local.at: here.
Unfortunately, m4_append relies on the macro not being defined whereas
we would have preferred it to check for emptiness. So use
m4_define/m4_undefine instead of m4_pushdef/m4_popdef.
Let's have C be the reference, and match it elsewhere. Maybe C is too
verbose and some adjustments are needed, but then that would be done
in another batch of patches.
* data/skeletons/lalr1.cc: Print the stack once we popped after
YYERROR, and before emptying the stack at the end of parsing.
Currently the C and C++ parse traces differ in the order in which the
stack is displayed: bottom up in C, top down in C++. Let's stick to
the C order.
* data/skeletons/stack.hh (stack::iterator, stack::const_iterator)
(begin, end): Be forward, not backward.
Provide the users with a public API to get the name of the tokens. A
thin wrapper around yytname.
* data/skeletons/yacc.c (yysymbol_name): New.
Use it.
I was hoping it would help us catch warnings when char is
unsigned (see 78bb152a63), but it does
not seem to help. It's a pity that the compiler is the same all over
the place, I would have preferred testing others.
* .travis.yml: here.
Supporting YYERROR_VERBOSE via cpp is a nuisance: m4 is in charge of
handling alternatives. When adding more options for %define
parse.error, supporting both CPP and M4 is too complex. Anyway,
YYERROR_VERBOSE was deprecated long ago.
* data/skeletons/yacc.c: Use m4 only to handle verbose/simple error
messages.
I would like to offer new ways to build the error message. As a first
step, let's simplify yysyntax_error whose first loop does two things
at the same time: (i) collect the tokens to be reported in the error
message, and (ii) accumulate their sizes and possibly return
"overflow". Let's pull (ii) in a second step.
Then test 525 (regression.at:1193: parse.error=verbose overflow)
failed. This test checks that we correctly report "memory overflow"
when the error message is too large. However the test is mistaken: it
is triggered in a place where there are five (large) expected tokens,
so anyway we would not display them, so there is no (memory) overflow
here! Transform this test to (i) check that indeed there is no
overflow, and (ii) create syntax_error3 which does check the intended
behavior, but with four expected tokens.
* data/skeletons/yacc.c (yysyntax_error): First compute the list of
arguments, then compute yysize.
* tests/regression.at (parse.error=verbose overflow): Enhance and fix.
GCC's -Wchar-subscripts may report issues on platforms where char is
unsigned. Unfortunately the current CI does not reproduce the
problem. But that would allow contributors to report issues if the
warning appears somewhere.
See 139d065594.
Problem reported by Andy Fiddaman in:
https://lists.gnu.org/r/bug-bison/2019-12/msg00021.html
* configure.ac (warn_common): Add -Wchar-subscripts.
Problem reported by Andy Fiddaman in:
https://lists.gnu.org/r/bug-bison/2019-12/msg00021.html
* data/skeletons/yacc.c (yy_reduce_print, yy_lac, yysyntax_error)
(yyreturn): If I might be a char, write a[+I] instead of a[I],
so that ‘gcc -Wchar-subscripts’ does not complain.
Another breakage revealed by vcsn.
* data/skeletons/c++.m4 (yytranslate_): Do not hard code "yy" and
"parser", both can be changed by the user.
Actually, since we are in the parser itself, there's really no need to
qualify the type.
* data/skeletons/glr.c (YYASSERT): Rename as...
(YY_ASSERT): this, for consistency with yacc.c, and also to emphasize
the fact that this is not for the end user (YY_ prefix).
* tests/glr-regression.at: Define parse.assert.
Now that we use small integral types, possibly unsigned (e.g.,
unsigned char), to store state numbers, using -1 to denote an empty
state (i.e., a state that stores no semantical value) is very
dangerous: it will be confused with state 255, which might be
non-empty.
Rather than allocating a larger range of state numbers to keep the
empty-state apart, let's use the number of a state known to store no
value. The initial state, numbered 0, seems to fit perfectly the job.
Reported by Frank Heckenbach.
https://lists.gnu.org/archive/html/bug-bison/2019-11/msg00016.html
* data/skeletons/lalr1.cc (empty_state): Be 0.
It is not used. And its implementation was wrong when api.token.raw
was defined, as it was still mapping to the external token numbers,
instead of the internal ones. Besides it was provided only when
api.token.constructor is defined, yet always declared.
* data/skeletons/c++.m4 (by_type::token): Remove, useless.
Reported by Frank Heckenbach.
https://lists.gnu.org/archive/html/bug-bison/2019-11/msg00016.html
The cast is needed when yytranslate_'s argument type is token_type,
i.e., when api.token.constructor is defined.
373. types.at:138: testing lalr1.cc api.value.type=variant api.token.constructor ...
======== Testing with C++ standard flags: ''
../../tests/types.at:138: bison --color=no -fno-caret -o test.cc test.y
../../tests/types.at:138: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o test test.cc $LIBS
stderr:
test.cc:966:16: error: result of comparison of constant 257 with
expression of type 'yy::parser::token_type'
(aka 'yy::parser::token::yytokentype') is always true
[-Werror,-Wtautological-constant-out-of-range-compare]
else if (t <= user_token_number_max_)
~ ^ ~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
It is because it is expected that when api.token.constructor is
defined, only symbol constructors will be used, that yytranslate_ then
takes a token_type. But it is wrong: we still allow literal
characters in this case, as demonstrated by test 373 for instance.
%define api.value.type variant
%define api.token.constructor
%token <std::pair<int, int>> '1' '2';
[...]
static yy::parser::symbol_type yylex ()
{
static char const input[] = "12";
int res = input[toknum++];
typedef yy::parser::symbol_type symbol;
if (res)
return symbol (res, std::make_pair (res - '0', res - '0' + 1));
else
return symbol (res);
}
So let yytranslate_ always take an int, which makes the cast truly
useless.
* data/skeletons/c++.m4, data/skeletons/lalr1.cc (yytranslate_): here.
The C++ implementation of LAC did not skip the $undefined token,
probably because it was not exposed. Expose it, and use clearer
names.
* data/skeletons/c++.m4: Don't define undef_token_ in yytranslate_,
but...
* data/skeletons/lalr1.cc (yy_undef_token_): here.
Use a more precise type to define yy_undef_token_ and yy_error_token_.
Unfortunately we move from a compile-time value defined via an enum to
a static const member. Eventually we should make it constexpr.
Make LAC implementation more alike yacc.c's one.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java: Don't expose
yyuser_token_number_max_ and yyundef_token_. Do as in C++: scope them
into yytranslate_, and only when api.token.raw is not defined.
(yyterror_): Rename as...
(yy_error_token_): this.
* data/skeletons/lalr1.d (token_number_type): New.
Use it.
Can't be done in the Java backend, as Java does not have type aliases.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java (yytoken_number_):
Remove, useless.
Was used in ancient C skeletons to support YYPRINT, long obsoleted by
%printer.
It is not used at all. We will remove it also from yacc.c, but
later (see TODO).
* data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java (yyerrcode_):
Remove.
Reported by Frank Heckenbach.
https://lists.gnu.org/archive/html/bug-bison/2019-11/msg00016.html
* data/skeletons/c++.m4 (b4_yytranslate_define): Don't use yyeof_ as
if it had two different types.
It is used once against the input argument, which is the value
returned by yylex, which is an "external token number", typically an
int. It is also used as output type, an "internal symbol number".
It turns out that in both cases we mean "0", but let's keep yyeof_
only for the case "internal symbol number", i.e., _after_ conversion
by yytranslate.
This frees us from one cast.
The current code for yysyntax_error for %define parse.error verbose is
fishy (given that YYEMPTY is -2, invalid argument for yytname[]):
static int
yysyntax_error ([...])
{
YYPTRDIFF_T yysize0 = yytnamerr (YY_NULLPTR, yytname[yytoken]);
[...]
if (yytoken != YYEMPTY)
A nearby comment reports
The only way there can be no lookahead present (in yychar) is if
this state is a consistent state with a default action. Thus,
detecting the absence of a lookahead is sufficient to determine
that there is no unexpected or expected token to report. In that
case, just report a simple "syntax error".
So it _is_ possible to call yysyntax_error with yytoken == YYEMPTY,
albeit quite difficult when meaning to, so virtually impossible by
accident (after all, there was never a bug report about this).
I failed to produce a test case, but Joel E. Denny provided me with
one (added to the test suite below). The yacc.c skeleton fails on
this, and once fixed dies on a second problem. The glr.c skeleton was
also dying, but immediately of this second problem.
Indeed we were not allocating space for the error message's final \0.
This was hidden by the fact that we only had error messages with at
least an unexpected token displayed, so with at least one "%s" in the
format string, whose size (2) was included (incorrectly) in the final
size of the message (where the %s have been replaced by the actual
content).
* data/skeletons/glr.c, data/skeletons/yacc.c (yysyntax_error):
Do not invoke yytnamerr on YYEMPTY.
Clarify the computation of the length of the _final_ error message,
with the NUL terminator but without the '%s's.
* tests/conflicts.at (Syntax error in consistent error state):
New, contributed by Joel E. Denny.
Having a file named "exception" is risky: the compiler might use that
file in #include.
Reported by 马俊 <[email protected]>.
* tests/local.at (AT_SKIP_IF_EXCEPTION_SUPPORT_IS_POOR): Generate
'exceptions', not 'exception'.
String literals as tokens serve two distinct purposes: freeing from
having to implement the keyword matching in the scanner, and improving
error messages. Most of the time both can be achieved at the same
time, but on occasions, it does not work so well.
We promote their use for error messages. We will also still support
the former case, but it is _not_ the recommended approach.
* doc/bison.texi (Tokens from Literals): Clearly state that we don't
recommend looking up the token types in the list of token names.
String literals, which allow for better error messages, are (too)
liberally accepted by Bison, which might result in silent errors. For
instance
%type <exVal> cond "condition"
does not define “condition” as a string alias to 'cond' (nonterminal
symbols do not have string aliases). It is rather equivalent to
%nterm <exVal> cond
%token <exVal> "condition"
i.e., it gives the type 'exVal' to the "condition" token, which was
clearly not the intention.
Introduce -Wdangling-alias to catch this.
* src/complain.h, src/complain.c: Add support for -Wdangling-alias.
(argmatch_warning_args): Sort.
* src/symtab.c (symbol_check_defined): Complain about dangling
aliases.
* doc/bison.texi: Document it.
* tests/input.at (Dangling aliases): New test.
On
%token TOKEN1
%type <ival> TOKEN1 TOKEN2 't'
%token TOKEN2
%%
expr:
bison -Wyacc gives
input.y:2.15-20: warning: POSIX yacc reserves %type to nonterminals [-Wyacc]
2 | %type <ival> TOKEN1 TOKEN2 't'
| ^~~~~~
input.y:2.29-31: warning: POSIX yacc reserves %type to nonterminals [-Wyacc]
2 | %type <ival> TOKEN1 TOKEN2 't'
| ^~~
input.y:2.22-27: warning: POSIX yacc reserves %type to nonterminals [-Wyacc]
2 | %type <ival> TOKEN1 TOKEN2 't'
| ^~~~~~
The messages appear to be out of order, but they are emitted when the
error is found.
* src/symtab.h (symbol_class): Add pct_type_sym, used to denote
symbols appearing in %type.
* src/symtab.c (complain_pct_type_on_token): New.
(symbol_class_set): Check that %type is not applied to tokens.
(symbol_check_defined): pct_type_sym also means undefined.
* src/parse-gram.y (symbol_decl.1): Set the class to pct_type_sym.
* src/reader.c (grammar_current_rule_begin): pct_type_sym also means
undefined.
* tests/input.at (Yacc's %type): New.
As an extension to POSIX Yacc, Bison's %type accepts tokens.
Unfortunately with string literals as implicit tokens, this is
misleading, and led some users to write
%type <exVal> cond "condition"
believing that "condition" would be associated to the 'cond'
nonterminal (see https://github.com/apache/httpd/pull/72).
* doc/bison.texi: Promote %nterm rather than %type to declare the type
of nonterminals.
* README: A few fixes.
Explain how to install color support.
* README-hacking: Rename as...
* README-hacking.md: this, and convert to Markdown.
Improve typography.
Improve explanations about update-test.
* src/gram.c (grammar_dump): Print terminals likewise non terminals.
* tests/sets.at (Reduced Grammar): Update test case to catch up the
change and add a test case where prec and assoc are used.
* tests/diagnostics.at (Locations from M4, Tabulations and multibyte
characters from M4): These tests are actually checking a message
coming from C, not from M4. Replace with...
(Complaints from M4): This.
We still have a few old C casts in lalr1.cc, let's get rid of them.
Reported by Frank Heckenbach.
Actually, let's monitor all our casts using easy to grep macros.
Let's use these macros to use the C++ standard casts when we are in
C++.
* data/skeletons/c.m4 (b4_cast_define): New.
* data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, data/skeletons/stack.hh,
* data/skeletons/yacc.c:
Use it and/or its casts.
* tests/actions.at, tests/cxx-type.at,
* tests/glr-regression.at, tests/headers.at, tests/torture.at,
* tests/types.at:
Use YY_CAST instead of C casts.
* configure.ac (warn_cxx): Add -Wold-style-cast.
* doc/bison.texi: Disable it.
* data/skeletons/location.cc: Remove the u (for unsigned) suffix from
the initial line and column.
* NEWS: AFAICT, only C++ backends have their location types changed.
We have too many global variables, adding structure would help. For a
start, let's hide some of the variables closer to their usage.
* src/getargs.c, src/files.h (current_file): Move to...
* src/scan-gram.c: here.
* src/scan-gram.h (gram_in, gram__flex_debug): Remove, make them
private to the scanner.
* src/reader.h, src/reader.c (reader): Take a grammar file as argument.
Move the handling of scanner variables to...
* src/scan-gram.l (gram_scanner_open, gram_scanner_close): here.
(gram_scanner_initialize): Remove, replaced by gram_scanner_open.
* src/main.c: Adjust.
* src/parse-gram (%initial-action): here.
(handle_skeleton): Don't depend on the current file name to look for
"local" skeletons (subject to changes coming from "#lines"): depend
only on the initial file name, the one given on the command line.
Currently there are two globals denoting the input file: grammar_file
is the one from the command line, and current_file which might change
because of #line. Use only the former.
* src/complain.c (error_message): here.
* tests/diagnostics.at: Adjust.
This skeleton uses a single stack of state structures, so it is less
likely to benefit from a stack size reduction than yacc.c (which uses
several stacks: state number, value and location). But it will reduce
the size of the LAC stack.
This skeleton was already using int for state numbers, so, contrary to
yacc.c, this brings nothing for large automata.
Overall, it is still nicer to make the skeletons alike.
* data/skeletons/lalr1.cc (state_type): Here.
Reported by Thomas Petazzoni.
https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00000.html
* examples/c/reccalc/local.mk: Complete dependencies, including for
earlier versions of Automake (for sake of our CI, on top of Ubuntu
Xenial/Bionic, which feature only Automake 1.15).
(%D%/scan.c %D%/scan.h): Upgrade to the full version provided in
Automake's documentation.
See https://lists.gnu.org/archive/html/bug-bison/2019-10/msg00061.html
to https://lists.gnu.org/archive/html/bug-bison/2019-10/msg00073.html.
Paul Eggert's changes in gnulib do fix the issue for modern GCCs (7,
8, 9) on macOS. Unfortunately these warnings are back on the
CI (GNU/Linux) with GCC 4.6, 4.7, (not 4.8) and 4.9.
Disable the warning locally.
* configure.ac (warn_common, warn_tests): Remove -Wtype-limits.
* src/system.h (IGNORE_TYPE_LIMITS_BEGIN, IGNORE_TYPE_LIMITS_END): New.
* src/InadequacyList.c, src/parse-gram.c, src/parse-gram.y,
* src/symtab.c: Use it.
This is really weird: GCC points to the LHS of the assignment...
260. headers.at:184: testing Sane headers: api.pure api.push-pull=both ...
tests/headers.at:184: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -d -o input.c input.y
tests/headers.at:184: $CC $CFLAGS $CPPFLAGS -c -o input.o input.c
stderr:
input.c: In function 'yyparse':
input.c:1276:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1276 | yylval = *yypushed_val;
| ~~~~~~~^~~~~~~~~~~~~~~
input.c: In function 'yypull_parse':
input.c:1276:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
1276 | yylval = *yypushed_val;
| ~~~~~~~^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
stdout:
tests/headers.at:184: exit code was 1, expected 0
See also d87c8ac79a
and 9645a2b20e.
* tests/headers.at (Several parsers, Several parsers): Disable these
warnings when in push parser.
The documentation for Oracle Solaris Studio 12.3 (Sun C++ 5.12
2011/11/16) says it supports C++03. This compiler rejects the
location.cc use of std::max for some reason; I don’t know why
since I don’t use C++ as a rule. The simplest workaround is to
open-code ‘max’.
* data/skeletons/location.cc (add_):
Do max by hand rather than relying on std::max.
Don’t include <algorithm.h>; no longer needed.
As documented in the Autoconf manual, Solaris 10 sed rejects
script labels contianing more than 7 characters. POSIX requires
support for at least 8 characters, but we might as well be portable
to Solaris 10 which is still supported.
* tests/local.at (AT_SETS_CHECK): Use only the first 7 characters
in sed labels.
* src/symtab.c: Include intprops.h
(symbol_user_token_number_set): Don’t allow user_token_number ==
INT_MAX because too much other code adds 1 to the user token number.
(symbols_token_translations_init): Complain on integer overflow
instead of indulging in undefined behavior.
* src/scan-gram.l: Include errno.h, for errno.
(scan_integer, handle_syncline): Check for integer overflow.
* tests/input.at (too-large.y): Adjust to match new diagnostics.
* src/parse-gram.y: Include intprops.h.
(handle_require): Don’t indulge in undefined behavior if the major
or minor number is out of range. Instead, check that the
resulting value is nonnegative, fits in int, and that the minor
number is less than 100. Also, check that a number was parsed.
Sun C 5.12 defines __SUNPRO_C to 0x5120 but diagnoses
‘__attribute__ ((__unused__))’. Change the ifdefs to use
the same method as Gnulib in this area.
* data/skeletons/c.m4 (YY_ATTRIBUTE): Remove, since
not all attributes were added in the same compiler version.
(YY_ATTRIBUTE_PURE, YY_ATTRIBUTE_UNUSED):
Use specific GCC version for each attribute.
Pay no attention to __SUNPRO_C.
* tests/headers.at (Several parsers): Tighten tests accordingly.
Oracle Solaris Studio 12.3 (Sun C 5.12 2011/11/16) by default does
not conform to C99; it defines __STDC_VERSION__ to be 199409L, so
the Bison code does not include <stdint.h> (not required by C89
amendment 1) even though this compiler does have <stdint.h>. On
this platform <limits.h> defines INT_LEAST8_MAX (POSIX allows
this) so the skeleton got confused and thought that <stdint.h> had
been included even though it wasn’t.
* data/skeletons/c.m4 (b4_c99_int_type_define) [!__PTRDIFF_MAX__]:
Always include <limits.h>.
(YY_STDINT_H): Define when <stdint.h> was included.
All uses of expressions like ‘defined INT_LEAST8_MAX’ changed to
‘defined YY_STDINT_H’, since Sun C 5.12 <limits.h> defines macros
like INT_LEAST8_MAX but does not declare types like int_least8_t.
Let's make a difference between places where Perl is required for the
test (AT_PERL_REQUIRE), and the places where it's used to run the
test, but it's not not to run the test (AT_PERL_CHECK).
* tests/local.at (AT_REQUIRE): New.
(AT_PERL_CHECK, AT_PERL_REQUIRE): New.
Use them where appropriate.
* tests/local.mk ($(TESTSUITE)): Beware not to start the line with
'-pi' if Perl is empty, as Make understands this as "it's ok to fail".
Which it is not.
My previous tests (with ./configure PERL=false) have been fooled by
configure, that managed to find perl anyway. This time, I ran this on
a Fedora in Docker, without Perl.
* tests/calc.at, tests/diagnostics.at, tests/headers.at,
* tests/input.at, tests/local.at, tests/named-refs.at,
* tests/output.at, tests/regression.at, tests/skeletons.at,
* tests/synclines.at, tests/torture.at: Don't require Perl.
* data/skeletons/c.m4 (b4_c99_int_type_define): Reorder to put the
signed types first, since they’re simpler and this keeps similar
code closer. For signed types, don’t bother checking whether the
type promotes to int since the type must be signed anyway. For
unsigned types, protect a test like ‘UCHAR_MAX <= INT_MAX’ with
‘!defined __UINT_LEAST8_MAX__’, as otherwise the logic is wrong
for oddball platforms; and once we do that, there should no need
for ‘defined INT_MAX’ so remove that.
Currently we face test suite failures in different environments,
because of a conflict between the definitions of isnan by gnulib, and
by the C++ library:
262. headers.at:186: testing Sane headers: %locations %debug c++ ...
./headers.at:186: COLUMNS=1000; export COLUMNS; bison --color=no -fno-caret -d -o input.cc input.y
./headers.at:186: $CXX $CXXFLAGS $CPPFLAGS -c -o input.o input.cc
stderr:
In file included from /usr/include/c++/4.8.2/cmath:44:0,
from /usr/include/c++/4.8.2/random:38,
from /usr/include/c++/4.8.2/bits/stl_algo.h:65,
from /usr/include/c++/4.8.2/algorithm:62,
from location.hh:41,
from input.hh:90,
from input.cc:50:
/u/cs/fac/eggert/src/gnu/bison/lib/math.h: In function 'bool isnan(double)':
/u/cs/fac/eggert/src/gnu/bison/lib/math.h:2849:1: error: new declaration 'bool isnan(double)'
_GL_MATH_CXX_REAL_FLOATING_DECL_2 (isnan, isnan, bool)
^
In file included from /usr/include/features.h:375:0,
from /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/os_defines.h:39,
from /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/c++config.h:2097,
from /usr/include/c++/4.8.2/cstdlib:41,
from input.hh:48,
from input.cc:50:
/usr/include/bits/mathcalls.h:235:1: error: ambiguates old declaration 'int isnan(double)'
__MATHDECL_1 (int,isnan,, (_Mdouble_ __value)) __attribute__ ((__const__));
^
There might be something to do in gnulib about this, but I believe
that gnulib should not be used in the test suite in the first place.
The test suite should work with other compilers than the one used to
compile the package. For a start, Bison sources are more
demanding (C99) than the generated parsers. Last time I tried, tcc
for example, was not able to compile Bison, yet our generated parsers
should compile cleanly with it.
Besides the problem at hand is with the C++ compiler, with is not the
one used to set up gnulib at configuration-time (config.h is mainly
built from probing the C compiler).
We should really not depend on gnulib in tests.
This was introduced in 2001 to check whether including
stdlib.h/string.h is safe thanks to STDC_HEADERS
(2ce1014469). Today, we assume at least
a C90 compiler, it should be safe enough.
* tests/local.at, tests/testsuite.h: Do not include config.h.
* tests/atlocal.in (conftest.cc): Likewise.
(CPPFLAGS): Do not expose lib/, as because of this we might picked up
gnulib replacement headers for system headers.
* tests/input.at: Use int instead of ptrdiff_t, for easier portability
(some machine on the CI did not find ptrdiff_t).
* tests/c++.at: Add missing include for getchar.
A number of portability issues with GCC 4.6 .. 4.9 (inclusive):
input.c:184:7: error: "UCHAR_MAX" is not defined [-Werror=undef]
#elif UCHAR_MAX <= INT_MAX
^
input.c:184:20: error: "INT_MAX" is not defined [-Werror=undef]
#elif UCHAR_MAX <= INT_MAX
^
input.c:202:7: error: "USHRT_MAX" is not defined [-Werror=undef]
#elif USHRT_MAX <= INT_MAX
^
input.c:202:20: error: "INT_MAX" is not defined [-Werror=undef]
#elif USHRT_MAX <= INT_MAX
^
* data/skeletons/c.m4 (b4_c99_int_type_define): Don't assume they are
defined.
Flex should not be required to build Bison or run the test suite (of
course it is needed for maintaining Bison). Yet the Automake
conditional FLEX_WORKS does not work.
* m4/flex.m4 (_AC_PROG_LEX_YYTEXT_DECL): Since this is called
conditionally, don't define LEX_IS_FLEX here, but rather...
(AC_PROG_LEX): here.
* configure.ac: Be more cautious about possibly undefined variables.
That way, glr.c can use it too.
* data/skeletons/c.m4 (b4_int_type):
Do not special-case ‘char’; it’s not worth the trouble,
as clang complains about char subscripts.
(b4_c99_int_type, b4_c99_int_type_define): New macros,
taken from yacc.c.
* data/skeletons/glr.c: Use b4_int_type_define.
* data/skeletons/yacc.c (b4_int_type): Remove, since there’s
no longer any need to redefine it.
Use b4_c99_int_type_define rather than its body.
This changes the Yacc skeleton to use “least” integer types to
keep tables smaller on some platforms, which should lessen cache
pressure. Since Bison uses the Yacc skeleton, it follows suit.
* data/skeletons/yacc.c: Include limits.h and stdint.h if this
seems to be needed.
(yytype_uint8, yytype_int8, yytype_uint16, yytype_int16):
If available, use GCC predefined macros __INT_MAX__ etc. to select
a “least” type, as this avoids namespace hassles. Otherwise, if
available fall back on selecting a “least” type via the C99 macros
INT_MAX, INT_LEAST8_MAX, etc. Otherwise, fall further back on one of
the builtin C99 types signed char, short, and int. Make sure that
any selected type promotes to int. Ignore any macros YYTYPE_INT16,
YYTYPE_INT8, YYTYPE_UINT16, YYTYPE_UINT8 defined by the user.
(ptrdiff_t, PTRDIFF_MAX): Simplify in the light of the above.
(yytype_uint8, yytype_uint16): Do not assume that unsigned char
and unsigned short promote to int, as this isn’t true on some
platforms (e.g., TI TMS320C55x).
* src/parse-gram.y (YYTYPE_INT16, YYTYPE_INT8, YYTYPE_UINT16)
(YYTYPE_UINT8): Remove, as these are no longer effective.
* data/skeletons/yacc.c (YYPTRDIFF_T, YYPTRDIFF_MAXIMUM):
Default to long, not int.
(yy_lac_stack_realloc, yy_lac, yytnamerr, yyparse):
Avoid casts to YYPTRDIFF_T that were masking the problem.
Because the checking of the grammar is made by phases after the whole
grammar was read, we sometimes have diagnostics that look weird. In
some case, within one type of checking, the entities are not checked
in the order in which they appear in the file. For instance, checking
symbols is done on the list of symbols sorted by tag:
foo.y:1.20-22: warning: symbol BAR is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
foo.y:1.16-18: warning: symbol QUX is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
Let's sort them by location instead:
foo.y:1.16-18: warning: symbol 'QUX' is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
foo.y:1.20-22: warning: symbol 'BAR' is used, but is not defined as a token and has no rules [-Wother]
1 | %destructor {} QUX BAR
| ^~~
* src/location.h (location_cmp): Be robust to empty file names.
* src/symtab.c (symbol_cmp): Sort by location.
* tests/input.at: Adjust expectations.
From
input.y:1.17-19: warning: symbol baz is used, but is not defined as a token and has no rules [-Wother]
1 | %printer {} foo baz
| ^~~
to
input.y:1.17-19: warning: symbol 'baz' is used, but is not defined as a token and has no rules; did you mean 'bar'? [-Wother]
1 | %printer {} foo baz
| ^~~
| bar
* bootstrap.conf: We need fstrcmp.
* src/symtab.c (symbol_from_uniqstr_fuzzy): New.
(complain_symbol_undeclared): Use it.
* tests/diagnostics.at (Suggestions): New.
* data/bison-default.css (insertion): Rename as...
(fixit-insert): this, as this is what GCC uses.
* src/symtab.c (complain_symbol_undeclared): New.
Use it.
Use quote on the guilty symbol (like GCC does, and we also do
elsewhere).
* tests/input.at: Adjust.
Both are stored in a hash, and back in the days, we used to iterate
over these tables using hash_do_for_each. However, the order of
traversal was not deterministic, which was a nuisance for
deterministic output (and therefore also a problem for tests). So at
some point (83b60c97ee) we generated a
sorted list of these symbols, and symbols_do actually iterated on that
list. But we kept the constraints of using hash_do_for_each, which
requires a lot of ceremonial code, and makes it hard/unnatural to
preserve data between iterations (see the next commit).
Alas, this is C, not C++.
Let's remove this abstraction, and directly iterate on the sorted
tables.
* src/symtab.c (symbols_do): Remove.
Adjust callers to use a simple for-loop instead.
(table_sort): New.
(symbols_check_defined): Use it.
(symbol_check_defined_processor, symbol_pack_processor)
(semantic_type_check_defined_processor, symbol_translation_processor):
Remove.
Simplify the corresponding functions (that no longer need to return a
bool).
This commit adds the suggestion in green, on the line below the
caret-and-tildes.
foo.y:1.1-14: warning: deprecated directive: '%error-verbose', use '%define parse.error verbose' [-Wdeprecated]
1 | %error-verbose
| ^~~~~~~~~~~~~~
| %define parse.error verbose
The current approach, with location_caret_suggestion, is fragile:
there's a protocol of calls to the complain functions which is strict.
We should rather have a richer structure describing the diagnostics,
including with submessages such as the suggestions, passed in the end
to the routines in charge of formatting and printing them.
* src/location.h, src/location.c (location_caret_suggestion): New.
* src/complain.c (deprecated_directive): Use it.
* tests/diagnostics.at, tests/input.at: Adjust expectations.
* src/location.c (caret_set_file): New.
Store the current line's length in caret_info.line_len.
Pay attention to fseek's return value.
Extracted from...
(location_caret): here.
I no longer agree with that item, there are indeed two things to
report: lack of definition, and being useless. We could have either
one without the other, they are not directly related.
input.c: In function 'int yyparse()':
input.c: error: conversion to 'long int' from 'long unsigned int'
may change the sign of the result [-Werror=sign-conversion]
yyes_capacity = sizeof yyesa / sizeof *yyes;
^
cc1plus: all warnings being treated as errors
* data/skeletons/yacc.c: here.
GCC 4.8 reports:
input.y:57:33: error: conversion to 'int' from 'long unsigned int'
may alter its value [-Werror=conversion]
int input_elts = sizeof input / sizeof input[0];
^
* tests/local.at (AT_YYLEX_DEFINE(c)): Add a cast (sorry, Paul!).
* src/location.c (caret_set_file):
* src/scan-code.l (contains_dot_or_dash):
Do not quietly convert pointer to bool, as Oracle Developer Studio
12.6 complains and it is arguably confusing style anyway.
For instance with GCC 4.9 and --enable-gcc-warnings:
25. input.at:1201: testing Torturing the Scanner ...
../../tests/input.at:1344: $CC $CFLAGS $CPPFLAGS -c -o input.o input.c
stderr:
input.c:239:18: error: "__STDC_VERSION__" is not defined [-Werror=undef]
# elif 199901 <= __STDC_VERSION__
^
input.c:256:18: error: "__STDC_VERSION__" is not defined [-Werror=undef]
# elif 199901 <= __STDC_VERSION__
^
* data/skeletons/yacc.c: Check that __STDC_VERSION__ is defined before
using it.
* doc/bison.texi (Mfcalc Symbol Table, Mfcalc Lexer):
Don’t abort on memory allocation failure or integer overflow.
Instead, comment that these things aren’t checked for.
On the CI with GCC 6:
examples/c++/calc++/parser.cc:845:5: error: 'ptrdiff_t' was not declared in this scope
ptrdiff_t yycount = 0;
^~~~~~~~~
examples/c++/calc++/parser.cc:845:5: note: suggested alternatives:
/usr/include/x86_64-linux-gnu/c++/6/bits/c++config.h:202:28: note: 'std::ptrdiff_t'
typedef __PTRDIFF_TYPE__ ptrdiff_t;
^~~~~~~~~
* data/skeletons/lalr1.cc: Qualify ptrdiff_t and size_t with std::.
* tests/torture.at (Big horizontal): Adjust to recent changes with
integers. If there are states 0..256, Bison now uses a signed
rather than an unsigned 16-bit integer.
This patch contains more fixes to prefer signed to unsigned
integer types, as modern tools like 'gcc -fsanitize=undefined'
can check for signed integer overflow but not unsigned overflow.
* NEWS: Document the API change.
* boostrap.conf (gnulib_modules): Add intprops.
* data/skeletons/glr.c: Include stddef.h and stdint.h,
since this skeleton can assume C99 or later.
(YYSIZEMAX): Now signed, and the minimum of SIZE_MAX and PTRDIFF_MAX.
(yybool) [!__cplusplus]: Now signed (which is how bool behaves).
(YYTRANSLATE): Avoid use of unsigned, and make the macro
safe even for values greater than UINT_MAX.
(yytnamerr, struct yyGLRState, struct yyGLRStateSet, struct yyGLRStack)
(yyaddDeferredAction, yyinitStateSet, yyinitGLRStack)
(yyexpandGLRStack, yymarkStackDeleted, yyremoveDeletes)
(yyglrShift, yyglrShiftDefer, yy_reduce_print, yydoAction)
(yyglrReduce, yysplitStack, yyreportTree, yycompressStack)
(yyprocessOneStack, yyreportSyntaxError, yyrecoverSyntaxError)
(yyparse, yy_yypstack, yypstack, yypdumpstack):
* tests/input.at (Torturing the Scanner):
Prefer ptrdiff_t to size_t.
* data/skeletons/c++.m4 (b4_yytranslate_define):
* src/AnnotationList.c (AnnotationList__computePredecessorAnnotations):
* src/AnnotationList.h (AnnotationIndex):
* src/InadequacyList.h (InadequacyListNodeCount):
* src/closure.c (closure_new):
* src/complain.c (error_message, complains, complain_indent)
(complain_args, duplicate_directive, duplicate_rule_directive):
* src/gram.c (nritems, ritem_print, grammar_dump):
* src/ielr.c (ielr_compute_ritem_sees_lookahead_set)
(ielr_item_has_lookahead, ielr_compute_annotation_lists)
(ielr_compute_lookaheads):
* src/location.c (columns, boundary_print, location_print):
* src/muscle-tab.c (muscle_percent_define_insert)
(muscle_percent_define_check_values):
* src/output.c (prepare_rules, prepare_actions):
* src/parse-gram.y (id, handle_require):
* src/reader.c (record_merge_function_type, packgram):
* src/reduce.c (nuseless_productions, nuseless_nonterminals)
(inaccessable_symbols):
* src/relation.c (relation_print):
* src/scan-code.l (variant, variant_table_size, variant_count)
(variant_add, get_at_spec, show_sub_message, show_sub_messages)
(parse_ref):
* src/scan-gram.l (<SC_ESCAPED_STRING,SC_ESCAPED_CHARACTER>)
(scan_integer, convert_ucn_to_byte, handle_syncline):
* src/scan-skel.l (at_complain):
* src/symtab.c (complain_symbol_redeclared)
(complain_semantic_type_redeclared, complain_class_redeclared)
(symbol_class_set, complain_user_token_number_redeclared):
* src/tables.c (conflict_tos, conflrow, conflict_table)
(conflict_list, save_row, pack_vector):
* tests/local.at (AT_YYLEX_DEFINE(c)):
Prefer signed to unsigned integer.
* data/skeletons/lalr1.cc (yy_lac_check_):
* tests/actions.at (_AT_CHECK_PRINTER_AND_DESTRUCTOR):
* tests/local.at (AT_YYLEX_DEFINE(c)):
Omit now-unnecessary casts.
* data/skeletons/location.cc (b4_location_define):
* doc/bison.texi (Mfcalc Lexer, C++ position, C++ location):
Prefer int to unsigned for line and column numbers.
Change example to abort explicitly on memory exhaustion,
and fix an off-by-one bug that led to undefined behavior.
* data/skeletons/stack.hh (stack::operator[]):
Also allow ptrdiff_t indexes.
(stack::pop, slice::slice, slice::operator[]):
Index arg is now ptrdiff_t, not int.
(stack::ssize): New method.
(slice::range_): Now ptrdiff_t, not int.
* data/skeletons/yacc.c (b4_state_num_type): Remove.
All uses replaced by b4_int_type.
(YY_CONVERT_INT_BEGIN, YY_CONVERT_INT_END): New macros.
(yylac, yyparse): Use them around conversions that -Wconversion
would give false alarms about. Omit unnecessary casts.
(yy_stack_print): Use int rather than unsigned, and omit
a cast that doesn’t seem to be needed here any more.
* examples/c++/variant.yy (yylex):
* examples/c++/variant-11.yy (yylex):
Omit no-longer-needed conversions to unsigned.
* src/InadequacyList.c (InadequacyList__new_conflict):
Don’t assume *node_count is unsigned.
* src/output.c (muscle_insert_unsigned_table):
Remove; no longer used.
* NEWS: Mention this.
* data/skeletons/c.m4 (b4_int_type):
Prefer char if it will do, and prefer signed types to unsigned if
either will do.
* data/skeletons/glr.c (yy_reduce_print): No need to
convert rule line to unsigned long.
(yyrecoverSyntaxError): Put action into an int to
avoid GCC warning of using a char subscript.
* data/skeletons/lalr1.cc (yy_lac_check_, yysyntax_error_):
Prefer ptrdiff_t to size_t.
* data/skeletons/yacc.c (b4_int_type):
Prefer signed types to unsigned if either will do.
* data/skeletons/yacc.c (b4_declare_parser_state_variables):
(YYSTACK_RELOCATE, YYCOPY, yy_lac_stack_realloc, yy_lac)
(yytnamerr, yysyntax_error, yyparse): Prefer ptrdiff_t to size_t.
(YYPTRDIFF_T, YYPTRDIFF_MAXIMUM): New macros.
(YYSIZE_T): Fix "! defined YYSIZE_T" typo.
(YYSIZE_MAXIMUM): Take the minimum of PTRDIFF_MAX and SIZE_MAX.
(YYSIZEOF): New macro.
(YYSTACK_GAP_MAXIMUM, YYSTACK_BYTES, YYSTACK_RELOCATE)
(yy_lac_stack_realloc, yyparse): Use it.
(YYCOPY, yy_lac_stack_realloc): Cast to YYSIZE_T to pacify GCC.
(yy_reduce_print): Use int instead of unsigned long when int
will do.
(yy_lac_stack_realloc): Prefer long to unsigned long when
either will do.
* tests/regression.at: Adjust to these changes.
Currently we properly use the "best" integral type for tables,
including those storing state numbers. However the variables for
state numbers used in yyparse (and its dependencies such as
yy_stack_print) still use int16_t invariably. As a consequence, very
large models overflow these variables.
Let's use the "best" type for these variables too. It turns out that
we can still use 16 bits for twice larger automata: stick to unsigned
types.
However using 'unsigned' when 16 bits are not enough is troublesome
and generates tons of warnings about signedness issues. Instead,
let's use 'int'.
Reported by Tom Kramer.
https://lists.gnu.org/archive/html/bug-bison/2019-09/msg00018.html
* data/skeletons/yacc.c (b4_state_num_type): New.
(yy_state_num): Be computed from YYNSTATES.
* tests/linear: New.
* tests/torture.at (State number type): New.
Use it.
Instead of
#define YYPACT_NINF -130
#define yypact_value_is_default(Yystate) \
(!!((Yystate) == (-130)))
generate
#define YYPACT_NINF (-130)
#define yypact_value_is_default(Yyn) \
((Yyn) == YYPACT_NINF)
* data/skeletons/c.m4 (b4_table_value_equals): Add support for $4.
* data/skeletons/glr.c, data/skeletons/yacc.c: Use it.
Also, use shorter macro argument names, the name of the macro is clear
enough.
* upstream/maint:
c++: add copy ctors for compatibility with the IAR compiler
CI: show git status
CI: disable ICC
tests: pass -jN from Make to the test suite
quotearg: avoid leaks
maint: post-release administrivia
Reported by Andreas Damm.
https://savannah.gnu.org/support/?110032
* data/skeletons/lalr1.cc (stack_symbol_type::operator=): New
overload, const, to please the IAR C++ compiler (version ca 2013).
* bootstrap.conf: We need winsz-ioctl and winsz-termios.
* src/location.c (columns): Use winsize to get the number of
columns.
Code taken from the GNU Coreutils.
* src/location.h, src/location.c (caret_init): New.
* src/complain.c (complain_init): Call it.
* tests/bison.in: Export COLUMNS so that users of tests/bison can
enjoy proper line truncation.
* src/location.c (min_int, columns): New.
(location_caret): Compute the line width. Based on it, compute how
many columns must be skipped before the quoted location and truncated
after, to fit the sceen width.
* tests/local.at (AT_QUELL_VALGRIND): Transform into...
(AT_SET_ENV_IF, AT_SET_ENV): these.
Define COLUMNS to protect the test suite from the user's environment.
So far diagnostics were cheating: in addition to the 'column' field of
locations (based on actual screen width per multibyte characters and
on tabulation expansion), the scanner sets the 'byte' field.
Diagnostics used this byte count to decide where to insert (color)
style.
We want to be able to truncate the quoted lines when there are too
wide to fit the screen. This requires that the diagnostics learn how
to count columns, the byte-in-boundary trick no longer works.
Bytes are still used for fix-its.
* bootstrap.conf: We need mbfile for mbf_getc.
* src/location.c (caret_info): We need an mbfile.
(caret_set_file): Initialize it.
(caret_getc): Convert to mbfile.
(location_caret): Instead of relying on the byte position to decide
where to insert the color style, count the current column using
boundary_compute.
* src/location.c (caret_info): Replace file and line with pos, a
boundary. This will allow us to use features of the boundary type,
such as boundary_compute.
The handling of the contributions of the tabulations in the columns is
burried inside location_compute. We will soon be willing to use the
boundary part of the computation (to compute the current column number
each time we read a multibyte char).
* src/location.c (boundary_compute): New, extracted from...
(location_compute): here.
This changes the traces from
Reading a token:
Now at end of input.
to
Reading a token:
Next token is token $end (7FFEE56E6474)
which is ok. Actually it is even better, as it gives the location
when locations are enabled, and is clearer when rules explicitly use
the EOF token.
* data/skeletons/lalr1.d (yytranslate_): Handle eof here, as is done
in lalr1.cc.
Bison used to feature %raw, documented as follows:
@item %raw
The output file @file{@var{name}.h} normally defines the tokens with
Yacc-compatible token numbers. If this option is specified, the
internal Bison numbers are used instead. (Yacc-compatible numbers start
at 257 except for single character tokens; Bison assigns token numbers
sequentially for all tokens starting at 3.)
Unfortunately, as far as I can tell, it never worked: token numbers
are indeed changed in the generated tables (from external token number
to internal), yet the code was still applying the mapping from
external token numbers to internal token numbers.
This commit reintroduces the feature as it was expected to be.
* data/skeletons/bison.m4 (b4_token_format): When api.token.raw is
enabled, use the internal token number.
* data/skeletons/yacc.c (yytranslate): Don't emit if api.token.raw is
enabled.
(YYTRANSLATE): Adjust.
We used to treat lone CRs (\r, aka ^M) as regular NLs (\n), probably
to please Classic MacOS. As of today, it makes more sense to treat \r
like a plain white space character.
https://lists.gnu.org/archive/html/bison-patches/2019-09/msg00027.html
* src/scan-gram.l (no_cr_read): Remove. Instead, use...
(eol): this new abbreviation denoting end-of-line.
* src/location.c (caret_getc): New.
(location_caret): Use it.
* tests/diagnostics.at (Carriage return): Adjust expectations.
(CR NL): New.
When the input file contains lone CRs (aka, ^M, \r), the locations see
a new line. Diagnostics look only at \n as end-of-line, so sometimes
there is an offset in diagnostics. Worse yet: sometimes we loop
endlessly waiting for \n to come from a continuous stream of EOF.
Fix that:
- check for EOF
- beware not to call end_use_class if begin_use_class was not
called (which would abort). This could happen if the actual
line is shorter that the expected one.
Prompted by a (private) report from Marc Schönefeld.
* src/location.c (location_caret): here.
* tests/diagnostics.at (Carriage return): New.
When the input file contains lone CRs (aka, ^M, \r), the locations see
a new line. Diagnostics look only at \n as end-of-line, so sometimes
there is an offset in diagnostics. Worse yet: sometimes we loop
endlessly waiting for \n to come from a continuous stream of EOF.
Fix that:
- check for EOF
- beware not to call end_use_class if begin_use_class was not
called (which would abort). This could happen if the actual
line is shorter that the expected one.
Prompted by a (private) report from Marc Schönefeld.
* src/location.c (location_caret): here.
* tests/diagnostics.at (Carriage return): New.
With
%token EOF 0 EOF 0
we get
input.y:3.14-16: warning: symbol EOF redeclared [-Wother]
3 | %token EOF 0 EOF 0
| ^~~
input.y:3.8-10: previous declaration
3 | %token EOF 0 EOF 0
| ^~~
Assertion failed: (nsyms == ntokens + nvars), function check_and_convert_grammar,
file /Users/akim/src/gnu/bison/src/reader.c, line 839.
Reported by Marc Schönefeld.
* src/symtab.c (symbol_user_token_number_set): Register only the
first definition of the end of input token.
* tests/input.at (Symbol redeclared): Check that case.
The CI, with CC='gcc-7 -fsanitize=undefined,address
-fno-omit-frame-pointer', reports:
calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
../../tests/calc.at:867: cat stderr
--- expout 2019-09-05 20:30:37.887257545 +0000
+++ /home/travis/build/bison-3.4.1.72-79a1-dirty/_build/tests/testsuite.dir/at-groups/438/stdout 2019-09-05 20:30:37.887257545 +0000
@@ -1 +1,2 @@
syntax error
+calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
438. calc.at:867: 438. Calculator glr.cc (calc.at:867): FAILED (calc.at:867)
The problem is that yylookaheadNeeds is not initialized in
yyinitStateSet, and when it is copied, the value is not 0 or 1.
* data/skeletons/glr.c (yylookaheadNeeds): Initialize yylookaheadNeeds.
hash_initialize returns NULL when out of memory. Check for it, and
die cleanly instead of crashing.
Reported by 江 祖铭 (Zu-Ming Jiang).
https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00015.html
* src/muscle-tab.c, src/state.c, src/symtab.c, src/uniqstr.c:
Check the value returned by hash_initialize.
https://lists.gnu.org/archive/html/bison-patches/2019-08/msg00007.html
When Bison is started with a flag that suppresses warning messages, the
error_message() function can produce a few gigabytes of indentation
because of a dangling pointer.
* src/complain.c (error_message): Don't reset indent_ptr here, but...
(complain_indent): here.
* tests/diagnostics.at (Indentation with message suppression): Check
this case.
* .travis.yml: Bionic is now available, with GCC8.
GCC7 sanitizers work, but they are too longer: cover only part 1.
Redefine part 1 and part 2 so that part 1 is really the core of the
tests: not playing with POSIX and C++ compiler for C code.
Travis answered favorably to my suggestion to provide a means to
disable git clone on some jobs (issue 7542). See
https://docs.travis-ci.com/user/customizing-the-build/#disabling-git-clone.
* .travis.yml: Disable git globally, enable it for i. the compile job,
and ii. the test job on ICC which needs the install-icc.sh script.
With
%token EOF 0 EOF 0
we get
input.y:3.14-16: warning: symbol EOF redeclared [-Wother]
3 | %token EOF 0 EOF 0
| ^~~
input.y:3.8-10: previous declaration
3 | %token EOF 0 EOF 0
| ^~~
Assertion failed: (nsyms == ntokens + nvars), function check_and_convert_grammar,
file /Users/akim/src/gnu/bison/src/reader.c, line 839.
Reported by Marc Schönefeld.
* src/symtab.c (symbol_user_token_number_set): Register only the
first definition of the end of input token.
* tests/input.at (Symbol redeclared): Check that case.
The CI, with CC='gcc-7 -fsanitize=undefined,address
-fno-omit-frame-pointer', reports:
calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
../../tests/calc.at:867: cat stderr
--- expout 2019-09-05 20:30:37.887257545 +0000
+++ /home/travis/build/bison-3.4.1.72-79a1-dirty/_build/tests/testsuite.dir/at-groups/438/stdout 2019-09-05 20:30:37.887257545 +0000
@@ -1 +1,2 @@
syntax error
+calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
438. calc.at:867: 438. Calculator glr.cc (calc.at:867): FAILED (calc.at:867)
The problem is that yylookaheadNeeds is not initialized in
yyinitStateSet, and when it is copied, the value is not 0 or 1.
* data/skeletons/glr.c (yylookaheadNeeds): Initialize yylookaheadNeeds.
* .travis.yml: Bionic is now available, with GCC8.
GCC7 sanitizers work, but they are too longer: cover only part 1.
Redefine part 1 and part 2 so that part 1 is really the core of the
tests: not playing with POSIX and C++ compiler for C code.
hash_initialize returns NULL when out of memory. Check for it, and
die cleanly instead of crashing.
Reported by 江 祖铭 (Zu-Ming Jiang).
https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00015.html
* src/muscle-tab.c, src/state.c, src/symtab.c, src/uniqstr.c:
Check the value returned by hash_initialize.
https://lists.gnu.org/archive/html/bison-patches/2019-08/msg00007.html
When Bison is started with a flag that suppresses warning messages, the
error_message() function can produce a few gigabytes of indentation
because of a dangling pointer.
* src/complain.c (error_message): Don't reset indent_ptr here, but...
(complain_indent): here.
* tests/diagnostics.at (Indentation with message suppression): Check
this case.
Implement lookahead correction (LAC) for the C++ skeleton. LAC is a
mechanism to make sure that we report the correct list of expected
tokens if a syntax error occurs. So far, LAC was only supported for
the C skeleton "yacc.c".
* data/skeletons/lalr1.cc: Add LAC support.
* doc/bison.texi: Update.
* data/skeletons/yacc.c (yysyntax_error): Change the nesting of `m4`
conditions slightly to make it more readable.
The generated C code stays unchanged.
* tests/regression.at: Use %expect to avoid warnings.
Set the keywords to facilitate running specific tests.
Use macros such as AT_YYLEX_DECLARE to facilitate tests for other
languages.
Likewise for AT_FULL_COMPILE.
In Texinfo. empty lines in multitable rows generate empty lines in the
output. Avoid them altogether.
With help from Gavin Smith.
https://lists.gnu.org/archive/html/bug-texinfo/2019-07/msg00000.html
* build-aux/cross-options.pl: Separate rows with empty lines.
So, to be more readable, generate a single line for each row.
Use Perl format to this end.
See the previous commit. This option should be removed, -o suffices.
* src/getargs.c (FIXED_OUTPUT_FILES): New.
Add support for it.
(getargs): Define loc, and use it.
This is safer when we need to pass a pointer to a location.
The name fixed-output-files is pretty clear: generate y.tab.c, as Yacc
does. So let's detach this from %yacc which does more: it requires
POSIX Yacc behavior.
This directive is obsolete since December 29th 2001
8c9a50bee1. It does not show in the
doc. I don't want to spend more time on improving its diagnostics, it
could be removed just as well as far as I'm concerned.
* src/scan-gram.l, src/parse-gram.y (%fixed-output-files): Detach from
%yacc.
Years ago we moved from 'look-ahead' to 'lookahead', and that alias
was kept for backward compatibility. But now that we use argmatch to
generate the documentation, that value clutters the doc.
* src/getargs.c (argmatch_report_args): Remove the
--report=look-aheads alias.
The doc says that -Dfoo=bar is the same as %define foo "bar". It is
not: the quotes are not added (and it makes a difference).
* doc/bison.texi (Tuning the Parser): Fix the definition of -D/-F
* src/getargs.c (usage): Likewise.
Let's clarify --help: use clearer "section" names, as in the doc.
Move --yacc to where it belongs.
* src/getargs.c (usage): Rename "Parser" as "Tuning the Parser", as in
the doc.
Rename "Output" as "Output Files"
Move --yacc to "Tuning the Parser".
* doc/bison.texi: Likewise.
It can now generate the usage message.
* src/complain.h (feature_fixit_parsable): Rename as...
(feature_fixit): this, for column economy.
Adjust dependencies.
(warning_usage): New.
Use it.
* src/complain.h, src/complain.c, src/getargs.h, src/getargs.c:
Use ARGMATCH_DEFINE_GROUP instead of the older interface.
The code is inconsistent: sometimes we pass by value, sometimes by
reference. Let's stick to the last, more conventional for large
values in C.
* src/scan-code.l: Pass locations by reference.
* configure.ac (DCFLAGS): Pass -g.
* data/skeletons/d.m4 (b4_locations_if): Remove, let bison.m4's one do
its job.
* data/skeletons/lalr1.d (position): Leave filename empty by default.
(position::toString): Don't print empty file names.
(location::this): New ctor.
(location::toString): Match the implementations of C/C++.
(yy_semantic_null): Leave undefined, the previous implementation does
not compile.
* tests/calc.at: Improve the implementation for D.
Enable more checks, in particular using locations.
* tests/local.at (AT_YYERROR_DEFINE(d)): Fix its implementation.
Sadly enough, AFAIK, there were never answers to the "More user
feedback will help to stabilize it" sentences. Remove them.
* src/getargs.c: IELR, canonical LR and XML output are here to stay,
and they are no more experimental than some other features.
* doc/bison.texi: Likewise.
Also remove "experimental" warning for Java, LAC, LR tuning options,
and named references.
* data/skeletons/lalr1.java: Use more conventional function names for
Java.
Prefer < and <= to => and >.
Use the same approach for m4 quotation as in the other skeletons.
Fix indentation issues.
* tests/calc.at, tests/java.at, tests/javapush.at: Fix quotation style.
(main): Use 'args', not 'argv', the former seems more conventional and
is used elsewhere in Bison.
Prefer character literals to integers to denote characters.
* examples/java/Calc.y: Likewise.
Travis answered favorably to my suggestion to provide a means to
disable git clone on some jobs (issue 7542). See
https://docs.travis-ci.com/user/customizing-the-build/#disabling-git-clone.
* .travis.yml: Disable git globally, enable it for i. the compile job,
and ii. the test job on ICC which needs the install-icc.sh script.
This is an experiment. Maybe more styles will be used (in which case
a short-hand function will be useful), maybe it will be just reverted.
* data/bison-default.css (.traces0): New.
* src/lalr.c (lalr): Use it.
See e3fdc37049: in C++ we generate
explicitly the code for the default action instead of simply copying
blindly the semantic value buffer. This is important when copying
raw memory is not enough, as exemplified by move-only types.
This is currently tested by examples/c++/variant.yy and variant-11.yy.
But it is safer to also have a test in the main test suite.
* tests/local.at (AT_REQUIRE_CXX_STD): Fix.
(AT_BISON_OPTION_PUSHDEFS, AT_BISON_OPTION_POPDEFS): Define/undefine
AT_BISON_OPTIONS.
* tests/c++.at (Default action): New.
It will be convenient to check sets elsewhere.
* tests/sets.at (AT_EXTRACT_SETS): Transform into...
* tests/local.at (AT_SETS_CHECK): this.
* tests/sets.at: Adjust.
The final gain is small: 2h2min instead 2h9min. But that is still an
improvement.
* .travis.yml (git.depth): Make the clone very shallow.
(git.submodules): Don't clone gnulib in test jobs.
(jobs.include.compile.script): Do it here.
* data/diagnostics.css: Rename as...
* data/bison-default.css: this.
Add the GPL header.
This is the convention followed by Bruno Haible in gettext.
Adjust dependencies.
* src/complain.c (complain_init_color): Use BISON_STYLE instead of
BISON_DIAGNOSTICS_STYLE.
Build the tarball in one job, check it in many.
Unfortunately no real gain in overall duration.
With help from Clément Démoulins.
* .travis.yml: here.
Remove all the tricks that were used to be able to boostrap on old
distros.
(before_install): Merge into 'script', because before_install applies
to all the jobs, and we don't want to run it for the 'compile' job.
In preparation for Bison 3.4, revert to the version before this
commit:
commit 03752516b21091cf3c4beea7e8b9bcad462d50ed
Author: John Darrington <[email protected]>
Date: Sun May 12 00:42:36 2019 +0200
version-etc: Ease translation.
* lib/version-etc.c (version_etc_arn, emit_bug_reporting_address):
Move URLs and formatting newlines out of translatable string.
because it changes the messages for --version, translated in
gnulib.po. These changes are not yet available on the translation
project, so we would have a regression in the set of translated
strings.
It works only in implicit rules (or with GNU Make, but not with
Solaris Make).
Reported by Bruno Haible.
http://lists.gnu.org/archive/html/bug-bison/2019-05/msg00009.html
Diagnosed thanks to Kiyoshi Kanazawa.
* examples/c/reccalc/local.mk, examples/d/local.mk,
* examples/java/local.mk: Don't use $< in non implicit rules.
It is more consistent with --color=html, --color=test, etc.
* src/getargs.h, src/getargs.c (style_debug): Rename as...
(color_debug): this.
(getargs_colors): Rename --style=debug as --color=debug.
Adjust dependencies.
An experimental commit introduced a fix-it hint that changes comments
such as "/* empty */" into %empty. But in some case, because
diagnostics are not necessarily emitted in order, the fixits also come
in disorder, which must never happen, as the fixes are installed in
one pass.
* src/fixits.c (fixits_register): Insert them in order.
Currently we remove the rhs to install %empty instead.
* src/reader.c (grammar_rule_check_and_complete): Insert the missing
%empty in front of the rhs, not in replacement thereof.
* tests/actions.at (Add missing %empty): Check that.
Some members are called foo_location, others are foo_loc. Stick to
the latter.
* src/gram.h, src/location.h, src/location.c, src/output.c,
* src/parse-gram.y, src/reader.h, src/reader.c, src/reduce.c,
* src/scan-gram.l, src/symlist.h, src/symlist.c, src/symtab.h,
* src/symtab.c:
Use _loc consistently, not _location.
symbol_list features a 'location' and a 'sym_loc' member. The former
is expected to be set only for symbol_lists that denote a symbol (not
a type name), and the latter should only denote the location of the
symbol/type name. Yet both are set, and the name "location" is too
unprecise.
* src/symlist.h, src/symlist.c (symbol_list::location): Rename as
rhs_loc for clarity. Move it to the "section" of data valid only
for rules.
* src/reader.c, src/scan-code.l: Adjust.
Revert "build: also generate the graph
reports" (4ec413da32). The problem is
Automake's ylwrap which does not rename y.dot with the appropriate
name. We should completely stop using Automake's support for Yacc,
which is not something I will do right now. So step back.
* Makefile.am (AM_YFLAGS_WITH_LINES): Don't pass --graph.
Because some of our examples use
%C%_reccalc_SOURCES = %D%/parse.y
Automake ships parse.y and parse.c, and possibly parse.h when it
"understands" that there is one. This is not what we want: ship only
parser.y. Yet we still want to use Automake to compile the sources
from parser.y. The easiest seems to use
nodist_%C%_reccalc_SOURCES = %D%/parse.y
together with
dist_reccalc_DATA = %D%/parse.y %D%/scan.l %D%/Makefile %D%/README.md
which guarantees that parse.y is indeed shipped.
* examples/c/calc/local.mk, examples/c/lexcalc/local.mk,
* examples/c/reccalc/local.mk: Always use nodist_*SOURCES for parsers,
let the dist_*_DATA rules do their job.
* cfg.mk: Disable checks where needed (e.g., we do want to check the
behavior with tabs).
(sc_at_parser_check): Remove. Unfortunately since
a11c144609 we no longer use the './'
prefix to run programs in the current directory. That was so that we
could run Java programs like the other, although they are no run with
the `./` prefix (see 967a59d2c0).
As a consequence this sc check no longer makes sense.
However, since now AT_PARSER_CHECK passes the `./` prefix itself, this
sc-check was superfluous.
* examples/c/reccalc/scan.l: Use memcpy, not strncpy.
* src/ielr.c, src/reader.c: Obfuscate "lr(0)" so that the sc-check for
"space before paren" does not fire.
* tests/diagnostics.at: Avoid space-tab, use tab-tab.
This makes reading the trace slightly easier. It would be very nice
to highlight the "big steps", especially reductions. But this is a
private experiment: do not use it.
* data/diagnostics.css (value): New.
* src/parse-gram.y: Use no delimiters and no c quotation for strings
to facilitate debugging.
(tron, troff, TRACE): New.
Not very elegant, but until there is support for printf-formats in
libtextstyle, it shall be enough.
Currently we pass only the columns based on the screen-width, which is
important for the carets. But we don't pass the bytes-based columns,
which is important for the colors. Pass both.
* src/muscle-tab.c (muscle_boundary_grow): Also pass the byte-based column.
* src/location.c (location_caret): Clarify.
(boundary_set_from_string): Adjust to the new format.
* tests/diagnostics.at (Tabulations and multibyte characters from M4): New.
Locations issued from M4 need the byte-based column for the
diagnostics to work properly. Currently they were unassigned, which
typically resulted in partially non-colored diagnostics.
* src/location.c (boundary_set_from_string): Fix the parsed location.
* src/muscle-tab.c (muscle_percent_define_default): Set the byte values.
* tests/diagnostics.at (Locations from M4): New.
This is meant for developers, not end users, that's why I attached it
to --trace.
* src/getargs.h, src/getargs.c (trace_locations): New.
* src/location.c (location_print): Use it.
We were using the gnulib's gettext module with tricks in
bootstrap.conf to avoid useless files. Instead, use gnulib's
gettext-h module.
* .travis.yml: Force Gettext 0.18.3 on Trusty.
* bootstrap.conf: Use gettext-h instead of gettext.
(excluded_files): Remove.
* configure.ac (AM_GNU_GETTEXT_VERSION): Bump to 0.19.
Reported by Balázs Scheidler.
* data/skeletons/c.m4 (b4_location_type_define): Use api.location.type
if defined.
* doc/bison.texi: Document it.
* tests/local.at (AT_C_IF, AT_LANG_CASE): New.
Support Span in C.
* tests/calc.at (Span): Convert it to be usable in C and C++.
Check api.location.type with yacc.c and glr.c.
The "identifier and colon" of a rule is implemented as a single token,
but whose location is only that of the identifier (so that messages
about the lhs of a rule are accurate). When reducing empty rules, the
default location is the single point location on the end of the
previous symbol. As a consequence, when Bison parses a grammar, the
location of the right-hand side of an empty rule is based on the
lhs, *independently of the position of the colon*. And the colon can
be way farther, separated by comments, white spaces, including empty
lines.
As a result, some messages look really bad. For instance:
$ cat foo.y
%%
foo : /* empty */
bar
: /* empty */
gives
$ bison -Wall foo.y
foo.y:2.4: warning: empty rule without %empty [-Wempty-rule]
2 | foo : /* empty */
| ^
foo.y:3.4: warning: empty rule without %empty [-Wempty-rule]
3 | bar
| ^
The carets are not at the right column, not even the right line.
This commit passes the colon "again" after the "id colon" token, which
gives more accurate locations for these messages:
$ bison -Wall foo.y
foo.y:2.10: warning: empty rule without %empty [-Wempty-rule]
2 | foo : /* empty */
| ^
foo.y:4.2: warning: empty rule without %empty [-Wempty-rule]
4 | : /* empty */
| ^
* src/scan-gram.l (SC_AFTER_IDENTIFIER): Rollback the colon, so that
we scan it again afterwards.
(INITIAL): Scan colons.
* src/parse-gram.y (COLON): New.
(rules): Parse the colon after the rule's id_colon (and possible
named reference).
* tests/actions.at, tests/conflicts.at, tests/diagnostics.at,
* tests/existing.at: Adjust.
Because the fix-its were ready the character-based columns, but were
applied on byte-based columns, the result with multibyte characters or
tabs could be "interesting". For instance
%fixed-output_files
%fixed_output-files
%fixed-output-files
%define api.prefix {foo}
%no-default-prec
would give
%fixed-%fixed-output-files %fixed_output-files
%fixed-orefix= "foo"
o_default-prec
* src/fixits.c (fixit_print, fixits_run): Work on byte-base columns.
* tests/input.at: Check it.
Currently, when we quote the source file, we indent it with one space,
and preserve tabulations, so there is a discrepancy and the visual
rendering is bad. One way out is to indent with a tab instead of a
space, but then this space can be used for more information. This is
what GCC9 does. Let's play copy cats.
See
https://lists.gnu.org/archive/html/bison-patches/2019-04/msg00025.htmlhttps://developers.redhat.com/blog/2019/03/08/usability-improvements-in-gcc-9/https://gcc.gnu.org/onlinedocs/gccint/Guidelines-for-Diagnostics.html#Guidelines-for-Diagnostics
* src/location.c (location_caret): Prefix quoted lines with the line
number and a pipe, fitting 8 columns.
* tests/actions.at, tests/c++.at, tests/conflicts.at,
* tests/diagnostics.at, tests/input.at, tests/java.at,
* tests/named-refs.at, tests/reduce.at, tests/regression.at,
* tests/sets.at: Adjust expectations.
Partly by "./build-aux/update-test tests/testsuite.dir/*/testsuite.log"
repeatedly, and partly by hand.
This is a pity: efforts were invested in computing correctly the
number of screen columns consumed by multibyte characters, but the
routines that do that were fed by single-byte inputs...
As a consequence Bison never displayed correctly locations when there
are multibyte characters.
* src/scan-gram.l (mbchar): New.
Use it instead of . in the catch-all clause.
* tests/diagnostics.at (Tabulations): Enhance into...
(Tabulations and multibyte characters): this.
Single point locations (equal boundaries) are troublesome, and we were
incorrectly ending the style in their case. Which results in an abort
in libtextstyle.
There is also a confusion between columns as displayed on the
screen (which take into account multibyte characters and tabulations),
and the number of bytes. Counting the screen-column
incrementally (character by character) is uneasy (because of multibyte
characters), and I don't want to maintain a buffer of the current line
when displaying the diagnostic. So I believe the simplest solution is
to track the byte number in addition to the screen column.
* src/location.h, src/location.c (boundary): Add the byte-column.
Adjust dependencies.
* src/getargs.c, src/scan-gram.l: Adjust.
* tests/diagnostics.at: Check zero-width locations.
Enable checking of styles even when libtextstyle is not installed.
* src/getargs.h, src/getargs.c (style_debug): New.
(getargs_colors): Set it when --style=debug.
* src/complain.c (begin_use_class, end_use_class): Use it.
* tests/diagnostics.at: New.
* src/lalr.c: Move logs to a better place to understand the chronology
of events.
* src/symlist.c (symbol_list_syms_print): Don't dump core on type
elements.
The 'regen' commit in Bison's history are a nuisance. They are
especially big because of the #lines. Let's generate our parse
without these lines in the repository, but generate them in the
tarball.
* Makefile.am (AM_YFLAGS_WITH_LINES): New.
(AM_YFLAGS): Use it.
(dist-hook): Regenerate the parser with #lines.
Currently, with --no-lines, instead of "#line file line\n", we emit
"\n". Let's emit nothing.
* data/skeletons/bison.m4 (b4_syncline): Emit at end-of-line when enabled.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, src/output.c: Use dnl after b4_syncline to
avoid spurious empty lines.
* tests/synclines.at (Sync Lines): Make sure that --no-lines is like
grep -v #line.
* tests/calc.at: Make sure that a rich grammar file behaves properly
with %no-lines.
Currently we use the syncline to report errors about a symbol's
destructor/printer. This is not accurate (only file and line), and
this is incorrect: the file name is double quotes (a recent change,
needed to make sure we escape properly double quotes in it). And
worst of all: with --no-line, b4_syncline expands to nothing.
Rather, push the locations into the backend, and use them.
* src/muscle-tab.h, src/muscle-tab.c (muscle_location_grow): Make it
public.
* src/output.c (prepare_symbol_definitions): Use it to pubish the
location of the printer and destructor.
* data/skeletons/lalr1.java: Use complain_at instead of complain.
* tests/java.at (Java invalid directives): Adjust expectations.
* data/skeletons/bison.m4 (b4_symbol_action_location): Remove.
We should not use b4_syncline this way.
Fatal errors are inconvenient, and should be reserved to cases where
we cannot continue. Here, it could even be warnings actually: these
directives will simply be ignored.
* data/skeletons/lalr1.java: Prefer error (b4_complain) to fatal
errors (b4_fatal).
* tests/java.at (Java invalid directives): New.
I never understood why we book ngotos+1 slots for relations between
gotos: there are at most ngotos images, not ngotos+1 (and "includes"
does have cases where a goto is in relation with itself, so it's not
ngotos-1).
Maybe bbf37f2534 explains the +1: a bug
left us register a goto several times on occasion, and the +1 might
have been a means to avoid this problem in most cases. Now that this
bug is addressed, we should no longer overbook memory, if only for the
clarity of the code ("why ngotos+1 instead of ngotos?").
* src/lalr.c: A goto has at most ngotos images, not ngotos+1.
While at it, avoid useless repeated call to map_goto introduced in
bbf37f2534.
Reported by wcventure.
http://lists.gnu.org/archive/html/bug-bison/2019-03/msg00008.html
* src/symtab.c (complain_class_redeclared): Don't print empty
locations.
There can only be empty locations for predefined symbols. And the
only symbol that is lexically available is the error token. So this
appears to be the only possible way to have an error involving an
empty location.
* tests/input.at (Symbol class redefinition): Check it.
The "includes" relation [DeRemer 1982] is between gotos, so of course,
for a given goto, there cannot be more that ngotos (number of gotos)
images. But we manipulate the set of images of a goto as a list,
without checking that an image was not already introduced. So we can
"register" way more images than ngotos, leading to a crash (heap
buffer overflow).
Reported by wcventure.
http://lists.gnu.org/archive/html/bug-bison/2019-03/msg00007.html
For the records, this bug is present in the first committed version of
Bison.
* src/lalr.c (build_relations): Don't insert the same goto several
times.
* tests/sets.at (Build Relations): New.
It seems that not many people read these logs: the error was
introduced in 2001 (3067fbef53),
* src/gram.c (grammar_dump): Fix the headers of the table: remove
duplicate display of "Ritem Range".
While at it, remove duplicate display of the rule number (and remove
an incorrect comment about it: these numbers _are_ equal).
* tests/sets.at (Reduced Grammar): Use useless rule, nterm and token
in the example.
When we update some output format, too many adjustements must be made
by hand. This script updates most tests based on the actual output
made during the tests.
* build-aux/update-test: New.
Suggested by Yuri at
<http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00000.html>.
The improvement is marginal for most grammars, but notable for large
grammars (e.g., PosgreSQL's postgre.y), and very large for the
sample.y grammar submitted by Yuri in
http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00012.html.
Measured with --trace=time -fsyntax-only.
parser action tables postgre.y sample.y
Before 0,129 (44%) 37,095 (99%)
After 0,117 (42%) 5,046 (93%)
* src/tables.c (pos): Replace this set of integer coded as an unsorted
array or integers with...
(pos_set): this bitset.
* src/reader.c (grammar_rule_check_and_complete): When 'p' and 'lhs'
are aliases, prefer the latter, for clarity and consistency.
(grammar_current_rule_begin): Avoid 'p', current_rule suffices.
* src/gram.h, src/gram.c: Comment changes.
ptdr# calc.tab.c
Bruno Haible just added a default implementation of libtextstyle's
interface when the library is not available.
https://lists.gnu.org/archive/html/bison-patches/2019-03/msg00025.html
* gnulib: Update.
* bootstrap.conf: Replace libtextstyle with libtextstyle-optional.
* src/complain.c, src/getargs.c: Remove now useless cpp guards.
The timevar and bitset modules now use the c99 module which causes
$CXX to now include -std=gnu++11 when possible. Unfortunately, G++
4.7 does not implement [[noreturn]] in C++11 mode, so our tests of
glr.cc (which uses _Noreturn) fail with
input.cc:954:1: error: expected unqualified-id before '[' token
right before [[noreturn]]. 4.8 works fine.
* data/skeletons/c.m4 (b4_attribute_define): Do not use [[noreturn]]
with GCC 4.7.
See f8408562f8.
* tests/calc.at: Stop imitating the C API.
Prepare more tests to run in the future.
%verbose works as expected (what a surprise, it's unrelated to the
skeleton...).
The variable spec_defines_file denotes the name of the generated
header. Its name is derived from --defines/%defines, whose name in
turn is derived from the fact that the header, in Yacc, contained the
Not only does the header now contain a lot more than just the token
definitions, but we no longer even generate macros, but an enum...
Let's modernize our vocabulary.
* src/files.h, src/files.c (spec_defines_file): Rename as...
(spec_header_file): this.
Currently when --defines is used, we generate a header, and paste an
exact copy of it into the generated parser implementation file. Let's
provide a means to #include it instead.
We don't do it by default because of the Autotools' ylwrap. This
program wraps invocations of yacc (that uses a fixed output name:
y.tab.c, y.tab.h, y.output) to support a more modern naming
scheme (dir/foo.y -> dir/foo.tab.c, dir/foo.tab.h, etc.). It does
that by renaming the generated files, and then by running sed to
propagate these renamings inside the files themselves.
Unfortunately Automake's Makefiles uses Bison as if it were Yacc (with
--yacc or with -o y.tab.c) and invoke bison via ylwrap. As a
consequence, as far as Bison is concerned, the output files are
y.tab.c and y.tab.h, so it emits '#include "y.tab.h"'. So far, so
good. But now ylwrap processes this '#include "y.tab.h"' into
'#include "dir/foo.tab.h"', which is not guaranteed to always work.
So, let's do the Right Thing when the output file is not y.tab.c, in
which case the user should %define api.header.include. Binding this
behavior to --yacc is tempting, but we recently told people to stop
using --yacc (as it also enables the Yacc warnings), but rather to use
-o y.tab.c.
Yacc.c is the only skeleton concerned: all the others do include their
header.
* data/skeletons/yacc.c (b4_header_include_if): New.
(api.header.include): Provide a default value when the output is not
y.tab.c.
* src/parse-gram.y (api.header.include): Define.
Running with CC='gcc-mp-8 -fsanitize=undefined' revealed Undefined
Behaviors.
https://lists.gnu.org/archive/html/bison-patches/2019-03/msg00008.html
* src/state.c (errs_new): Don't call memcpy with NULL as source.
* src/location.c (add_column_width): Don't assume that the column
argument is nonnegative: the scanner sometimes "backtracks" (e.g., see
ROLLBACK_CURRENT_TOKEN and DEPRECATED) in which case we can have
negative column numbers (temporarily).
Found in test 3 (Invalid inputs).
* data/skeletons/yacc.c: here.
This is more logical for the time stamps, but it's also required by
following patches: the shared declarations are also in charge of
handling api.value.type=union. So far, they are run in the
implementation file in both cases (with or without header). But if we
run them only in the header, then the implementation file is emited
with incorrect support for api.value.type=union.
Arguably we should not have such dependencies. This is because we
have side-effects in our backend (redefining the symbols' type and
type_tag). In the future we should find a better solution for this,
without sacrificing the independence of the backend from bison
itself (i.e., I don't think we should handle api.value.type=union in
bison, leave it to m4).
Currently we generate things like:
#line 683 "src/parse-gram.y" /* yacc.c:316 */
The first part is of course very important: compilers point the users
to their grammar file rather than into the generated parser. The
second part points to the place in the skeletons that generated this
piece of code.
This dependency on the Bison skeletons generates lots of useless 'git
diff'. This location is useless for the regular user (who does not
care about the skeletons) and is actually not useful for Bison
developpers too (I never used this to locate the code in skeletons
that generated output). So disable it completely. If someone thinks
this was actually useful, a %define variable should be provided to
control the level of verbosity of '#line', in replacement of
--no-lines.
So now, generate:
#line 683 "src/parse-gram.y"
* data/skeletons/bison.m4 (b4_sync_end): Emit nothing.
* src/lalr.c (follows_print): Just print the symbol tag.
Take and print a title.
Indent the output.
Use it to print the various steps of the computation.
(lookahead_tokens_print): Fix a lie: the number displayed is not the
number of tokens.
Don't display states that don't even have reductions.
* src/relation.h, src/relation.c (relation_print): Accept and use a
title.
Don't print empty rows.
Indent the output.
Adjust dependencies.
* src/lalr.c (initialize_goto_follows): Print 'reads' in traces.
* configure.ac (DCFLAGS): Define.
* tests/atlocal.in: Receive it.
* data/skeletons/d.m4 (api.parser.class): Remove spurious YY.
* data/skeletons/lalr1.d (yylex): Return an int instead of a
YYTokenType, so that we can use characters as tokens.
* examples/d/calc.y: Adjust.
* tests/local.at: Initial support for D.
(AT_D_IF, AT_DATA_GRAMMAR(D), AT_YYERROR_DECLARE(d))
(AT_YYERROR_DECLARE_EXTERN(d), AT_YYERROR_DEFINE(d))
(AT_MAIN_DEFINE(d), AT_COMPILE_D, AT_LANG_COMPILE(d), AT_LANG_EXT(d)):
New.
* tests/calc.at: Initial support for D.
* tests/headers.at
* examples/d/calc.y: Exit with failure on errors.
Remove useless operators (=, !) meant for the test suite.
Add unary + for symmetry.
* examples/d/calc.test: Adjust expectations.
Commit db34f79889 renames the variable F
as goto_follows, but forgot to rename this function.
* src/lalr.c (initialize_F): Rename as...
(initialize_goto_follows): this.
I need to be able to read includes and goto_follows.
* src/relation.h, src/relation.c (relation_print): Provide a means to
pretty-print the nodes of the relation.
* src/lalr.c (goto_print, follows_print): New.
(set_goto_map): Use goto_print.
(build_relations): Show INCLUDES.
(compute_FOLLOWS): Rename as...
(compute_follows): this.
Show FOLLOWS.
* src/lalr.c (build_relations): Reduce the scopes.
Instead of keeping rp alive in two different loops, clarify the second
one by having an index on the path we traverse (i.e., use that index
to compute the source state _and_ the symbol that labels the
transition).
This allows to turn an obscure 'while'-loop in a clearer (IMHO)
'for'-loop. We also consume more variables (by introducing p instead
of making more side effects on length), but we're in 2019, I don't
think this matters. What does matter is that (IMHO again), this is
now clearer.
Also, use clearer names.
* tests/local.at (AT_MAIN_DEFINE(java)): Exit failure on failure.
(AT_PARSER_CHECK): If in Java, run AT_JAVA_PARSER_CHECK.
* tests/conflicts.at (AT_CONSISTENT_ERRORS_CHECK): Simplify.
Currently the caller must specify the ./ prefix to its command. Let's
avoid that: it will be nicer to read, make it easier to have a version
that works for Java and C/C++.
* tests/local.at (AT_PARSER_CHECK): Prefix the command with ./.
Adjust callers.
* tests/conflicts.at (AT_YYLEX_PROTOTYPE): Don't define it, leave that
task to AT_DATA_GRAMMAR.
But be honest: tell AT_BISON_OPTION_PUSHDEFS all the options we use.
Reported by Hans Åberg.
https://lists.gnu.org/archive/html/help-bison/2019-02/msg00064.html
* src/files.c (spec_graph_file): Use `*.gv` when 3.4 or better,
otherwise `*.dot`.
* src/parse-gram.y (handle_require): Pretend we are already 3.4.
* doc/bison.texi: Adjust.
* tests/local.at, tests/output.at: Exercise this.
Currently, we have errors:
./doc/bison.texi:13005: node `History' lacks menu item for `Yacc' despite being its Up target
* doc/bison.texi (History): Add the local menu.
While at it, add a few index entries.
* examples/c/lexcalc/parse.y: Formatting/comment changes.
(line): Don't return a value.
Print the result here, which avoids printing a value for lines with an
error.
(yyerror): Be sure to increment the pointed, not the pointer...
* examples/c/lexcalc/lexcalc.test: Check errors.
* examples/c/lexcalc/local.mk: Fix a dependency.
While hacking on the computation of the automaton, I had yystate being
equal to -1, and the parser loops. Let's catch this when
parser.assert is enabled.
* data/skeletons/yacc.c (YY_ASSERT): New.
Use it.
Not using the name YYASSERT, to make it clear that this is private.
glr.c should probably move to YY_ASSERT too.
Also, while at it, report 'Entering state...' even before growing the
stacks.
* examples/c/calc/local.mk, examples/c/lexcalc/local.mk,
* examples/c/mfcalc/local.mk, examples/c/rpcalc/local.mk:
Regenerate the files if dependencies have changed.
I don't think this style:
/* If buffer is full, make it bigger. */
if (i == length)
{
length *= 2;
symbuf = (char *) realloc (symbuf, length + 1);
}
/* Add this character to the buffer. */
symbuf[i++] = c;
/* Get another character. */
c = getchar ();
or more readable than
/* If buffer is full, make it bigger. */
if (i == length)
{
length *= 2;
symbuf = (char *) realloc (symbuf, length + 1);
}
/* Add this character to the buffer. */
symbuf[i++] = c;
/* Get another character. */
c = getchar ();
Actually, I think the latter is more readable, and helps with width
issues in the PDF. As a matter of fact, I would happily move to //
only for single line comments.
* doc/bison.texi: A single space before closing comments.
* doc/bison.texi: Prefer 'fun' to 'fnct'.
Reduce local variable scopes.
Prefer strdup to malloc + strcpy.
Avoid gratuitous casts.
Use simpler names (e.g., 'name' instead of 'fname').
Avoid uses of 0 for NULL.
Avoid using NULL when possible (e.g., 'p' instead of 'p != NULL').
Prefer union names to casts (e.g. 'yylval.VAR = s' instead of
'*((symrec**) &yylval) = s').
Give arguments a name in fun declarations.
Use our typedefs instead of duplicating them (func_t).
Stop promoting an explicit $$ = $1;, it should be implicit (Bison
might be able to eliminate useless chain rules).
Help a bit Texinfo by making smaller groups.
Rely on the C compiler to call function pointers (prefer
'$1->value.fun ($3)' to (*($1->value.fnctptr))($3)').
Currently we have no simple example: rpcalc in reverse Polish, mfcalc
has functions, and lexcalc is using lex.
* examples/c/calc/Makefile, examples/c/calc/calc.y,
* examples/c/calc/calc.test, examples/c/calc/local.mk: New.
* examples/extexi: Since we issue #lines only at the beginning of
@example, leave empty line when removing content (such as @comment
lines), otherwise the lines that follow have incorrect source line
location. This leaves ugly empty lines, but they are removed when you
tidy the output for the end user: sequences of \n are mapped to at
most two sucessive \n.
The format is inconsistent. For instance most sections are
indented (including "Terminals unused in grammar" for instance), but
the sections "Terminals, with rules where they appear" and
"Nonterminals, with rules where they appear" are not. Let's indent
them. Also, these two sections try to wrap the output to avoid lines
too long. Yet we don't do that in the rest of the file, for instance
when listing the lookaheads of an item.
For instance in the case of Bison's parse-gram.output we go from:
Terminals, with rules where they appear
"end of file" (0) 0
error (256) 28 88
"string" <char*> (258) 9 13 16 17 20 23 24 109 116
[...]
Nonterminals, with rules where they appear
$accept (58)
on left: 0
input (59)
on left: 1, on right: 0
prologue_declarations (60)
on left: 2 3, on right: 1 3
prologue_declaration (61)
on left: 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24
25 26 27 28 29, on right: 3
[...]
to
Terminals, with rules where they appear
"end of file" (0) 0
error (256) 28 88
"string" <char*> (258) 9 13 16 17 20 23 24 109 116
[...]
Nonterminals, with rules where they appear
$accept (58)
on left: 0
input (59)
on left: 1
on right: 0
prologue_declarations (60)
on left: 2 3
on right: 1 3
prologue_declaration (61)
on left: 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24 25 26 27 28 29
on right: 3
[...]
* src/print.c (END_TEST): Remove.
(print_terminal_symbols): Don't try to wrap the output.
(print_nonterminal_symbols): Likewise.
Make two different lines for occurrences on the left, and occurrence
on the rhs of the rules.
Indent by 4 and 8, not 3.
* src/reduce.c (reduce_output): Indent by 4, not 3.
* tests/conflicts.at, tests/existing.at, tests/reduce.at,
* tests/regression.at, tests/report.at:
Adjust.
This should not be used to generate parsers. My point is actually to
facilitate debugging (when tweaking the generation of the LR(0)
automaton for instance, not carying -yet- about lookaheads).
* src/reader.c (prepare_percent_define_front_end_variables): Add lr(0).
* src/conflicts.c (set_conflicts): Be robust to reds not having
lookaheads at all.
* src/ielr.c (LrType, lr_type_get): Adjust.
(ielr): Implement support for LR(0).
* src/lalr.c (lalr_free): Don't free LA when it's not computed.
* maint:
maint: post-release administrivia
version 3.3.2
style: minor fixes
NEWS: named constructors are preferable to symbol_type ctors
gram: fix handling of nterms in actions when some are unused
style: rename local variable
CI: update the ICC serial number for travis-ci.org
Since Bison 3.3, semantic values in rule actions (i.e., '$...') are
passed to the m4 backend as the symbol number. Unfortunately, when
there are unused symbols, the symbols are renumbered _after_ the
numbers were used in the rule actions. As a result, the evaluation of
the skeleton failed because it used non existing symbol numbers.
Which is the happy scenario: we could use numbers of other existing
symbols...
Reported by Balázs Scheidler.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00044.html
Translating the rule actions after the symbol renumbering moves too
many parts in bison. Relying on the symbol identifiers is more
troublesome than it might first seem: some don't have an
identifier (tokens with only a literal string), some might have a
complex one (tokens with a literal string with characters special for
M4). Well, these are tokens, but nterms also have issues: "dummy"
nterms (for midrule actions) are named $@32 etc. which is risky for
M4.
Instead, let's simply give M4 the mapping between the old numbers and
the new ones. To avoid confusion between old and new numbers, always
emit pre-renumbering numbers as "orig NUM".
* data/README: Give details about "orig NUM".
* data/skeletons/bison.m4 (__b4_symbol, _b4_symbol): Resolve the
"orig NUM".
* src/output.c (prepare_symbol_definitions): Pass nterm_map to m4.
* src/reduce.h, src/reduce.c (nterm_map): Extract it from
nonterminals_reduce, to make it public.
(reduce_free): Free it.
* src/scan-code.l (handle_action_dollar): When referring to a nterm,
use "orig NUM".
* tests/reduce.at (Useless Parts): New, based Balázs Scheidler's
report.
A rule is a useless chain iff it's a chain (aka unit, or injection)
rule (i.e., the RHS has length 1), and it's useless (it has no used
defined semantic action).
* src/gram.h, src/gram.c (rule_useless_chain_p): New.
(grammar_dump): Report useless chain rules.
* tests/sets.at: Check the traces.
There are several places where we need to recover the rule from an
item, let's factor that into item_rule. We also want to print items
in a nice way: we do it when generating the *output file, but it is
also useful in debug messages.
* src/gram.h, src/gram.c (item_rule, item_print): New.
* src/print.c (print_core): Use them.
* src/state.h, src/state.c: Propagate constness.
The memory allocated by 'closure' (and some data such as 'fderives')
is used to computed a state's full itemset from its core. This is
needed during the construction of the LR(0) automaton, and the memory
is reclaimed immediately afterwards.
Unfortunately the reports (graph, text, xml) also need this
information when describing the states with their full itemsets. As a
consequence the memory was allocated again, fderives computed again
too, and more --trace reports are generated which only duplicate what
was already reported.
Stop that. It does mean that we release the memory later (hence the
peak memory usage is higher now), but I don't think that's a problem
today.
* src/lr0.c (generate_states): Don't call closure_free.
* src/state.c (states_free): Do it here.
(for symmetry with closure_new which is called in generate_states).
* src/print-graph.c, src/print-xml.c, src/print.c: You can now expect
the closure module to be functional.
This is more consistent with the other files.
* closure.h, closure.c (new_closure, free_closure): Rename as...
(closure_new, closure_free): this.
Adjust dependencies.
This will make it easier to add new elements (that might already be
part of shift_symbol) without having to worry about the size of
shift_symbol (which is currently a fixed size vector).
I could not measure any significant differences in performances in the
generation of LR(0) automaton (benched on gramamrs of Ruby, C, and C++).
* src/lr0.c (shift_symbol): Make it a bitset.
When debugging Bison itself, this is very handy, especially when
tweaking the frontend badly enough to break the backends. It can also
be used to check a grammar.
* src/getargs.h, src/getargs.c (feature_syntax_only): New.
(feature_args, feature_types): Adjust.
* src/main.c (main): Use it.
* src/symtab.h (status): Rename as...
(declaration_status): this, to avoid colliding with status, the
argument of 'usage'.
'status' seems a tad too general to be used only here.
* maint:
maint: post-release administrivia
version 3.3.1
yacc: issue warnings, not errors, for Bison extensions
style: formatting changes in NEWS and complain.c
tests: don't depend on the user's definition of SHELL
* src/gram.c (grammar_dump): Print the effective number first instead
of last. And fix it (remove the incorrect "+1").
Use t/f for Booleans.
* src/reduce.c: When asked, always print the reduced grammar, even if
there was nothing useless.
* tests/sets.at (Reduced Grammar): Check that.
It is inconvenient that we also generate the output files when we
update the grammar file, and it's somewhat unexpected. Let's not do
that.
* src/main.c (main): Skip generation when --update is passed.
* src/getargs.c (usage): Update the help message.
* doc/bison.texi (Bison Options): Likewise.
* tests/input.at: Check that we don't generate the output.
After all, this is clearly harmless.
* src/muscle-tab.c (muscle_percent_define_insert): Let equal
definitions of a %define variable be only a warning.
Adjust test cases.
* tests/input.at ("%define" backward compatibility): Don't define
twice "api.namespace", so that we don't get an error, which stops the
process too soon to see an error about the value given to
'lr.keep-unreachable-state'.
Don't repeat the name of the warning in the sub messages. E.g.,
remove the second "[-Wother]" in the following message
foo.y:2.1-27: warning: %define variable 'parse.error' redefined [-Wother]
%define parse.error verbose
^~~~~~~~~~~~~~~~~~~~~~~~~~~
foo.y:1.1-27: previous definition [-Wother]
%define parse.error verbose
^~~~~~~~~~~~~~~~~~~~~~~~~~~
* src/complain.c (error_message): Don't print the warning type when
it's indented.
Adjust test cases.
Reported by Derek Clegg
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00021.html
aux/parser-internal.h:429:12: error: 'syntax_error' has no out-of-line virtual
method definitions; its vtable will be emitted in every translation unit
[-Werror,-Wweak-vtables]
struct syntax_error : std::runtime_error
To avoid this warning, we need syntax_error to have a virtual function
defined in a compilation unit. Let it be the destructor. To comply
with C++98, this dtor should be 'throw()'. Merely making YY_NOEXCEPT
be 'throw()' in C++98 triggers
errors (http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00022.html),
so let's introduce YY_NOTHROW and flag only ~syntax_error with it.
Also, since we now have an explicit dtor, we need to provide an copy
ctor.
* configure.ac (warn_cxx): Add -Wweak-vtables.
* data/skeletons/c++.m4 (YY_NOTHROW): New.
(syntax_error): Declare the dtor, and define the copy ctor.
* data/skeletons/glr.cc, data/skeletons/lalr1.cc (~syntax_error):
Define.
Some reported by syntax-check.
* po/POTFILES.in: Add fixits.cc.
* src/muscle-tab.c: Don't cast for free.
* src/files.c: Reduce scopes.
* cfg.mk: We need the cast for free in muscle_percent_define_insert.
* doc/bison.texi: Use @kbd where appropriate.
Update ^~~~ marks for caret-errors.
* build-aux/cross-options.pl: Do not add quotes to %define's argument.
"handle_" is the prefix used in scan-code.l for instance.
* src/parse-gram.y (do_error_verbose, do_name_prefix, do_require)
(do_skeleton, do_yacc):
Rename as...
(handle_error_verbose, handle_name_prefix, handle_require)
(handle_skeleton, handle_yacc):
these.
We should use -ffixit and --update to clean files with duplicate
directives. And we should complain only once about duplicate obsolete
directives: keep only the "duplicate" warning. Let's start with %yacc.
For instance on:
%fixed-output_files
%fixed-output-files
%yacc
%%
exp:
This run of bison:
$ bison /tmp/foo.y -u
foo.y:1.1-19: warning: deprecated directive, use '%fixed-output-files' [-Wdeprecated]
%fixed-output_files
^~~~~~~~~~~~~~~~~~~
foo.y:2.1-19: warning: duplicate directive [-Wother]
%fixed-output-files
^~~~~~~~~~~~~~~~~~~
foo.y:1.1-19: previous declaration
%fixed-output_files
^~~~~~~~~~~~~~~~~~~
foo.y:3.1-5: warning: duplicate directive [-Wother]
%yacc
^~~~~
foo.y:1.1-19: previous declaration
%fixed-output_files
^~~~~~~~~~~~~~~~~~~
bison: file 'foo.y' was updated (backup: 'foo.y~')
gives:
%fixed-output-files
%%
exp:
* src/location.h, src/location.c (location_empty): New.
* src/complain.h, src/complain.c (duplicate_directive): New.
* src/getargs.h, src/getargs.c (yacc_flag): Instead of a Boolean, be
the location of the definition.
Update dependencies.
* src/scan-gram.l (%yacc, %fixed-output-files): Move the handling of
its warnings to...
* src/parse-gram.y (do_yacc): This new function.
* tests/input.at (Deprecated Directives): Adjust expectations.
* src/fixits.h, src/fixits.c (fixits_empty): New.
* src/complain.c (deprecated_directive): Register the Wdeprecated
fixits only if -Wdeprecated was enabled, so that we don't apply
updates if the user didn't ask for them.
* src/main.c (main): If there were fixits, issue a warning suggesting
running with --update.
Free uniqstrs after the fixits, since the latter use the former.
* tests/headers.at, tests/input.at: Update expectations.
* src/fixits.c (fixits_run): If erase the content of a line, also
erase the following \n.
* tests/input.at (Deprecated directives): Update expectations.
Reported by Derek Clegg.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00004.html
* configure.ac (warn_common): Add -Wimplicit-fallthrough.
This does trigger failures in the test suite.
* data/skeletons/glr.c, data/skeletons/lalr1.cc,
* data/skeletons/yacc.c, tests/c++.at:
Make fall-throws explicit.
Reported by Derek Clegg.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00006.html
Clang does not like this:
template <typename D>
struct basic_symbol : D
{
basic_symbol();
};
struct by_type {};
struct symbol_type : basic_symbol<by_type>
{
symbol_type(){}
};
It gives:
$ clang++-mp-7.0 -Wundefined-func-template foo.cc -c
foo.cc:11:3: warning: instantiation of function 'basic_symbol<by_type>::basic_symbol'
required here, but no definition is available [-Wundefined-func-template]
symbol_type(){}
^
foo.cc:4:3: note: forward declaration of template entity is here
basic_symbol();
^
foo.cc:11:3: note: add an explicit instantiation declaration to suppress this warning
if 'basic_symbol<by_type>::basic_symbol' is explicitly instantiated in
another translation unit
symbol_type(){}
^
1 warning generated.
The same applies for the basic_symbol's destructor and `clear()`.
* configure.ac (warn_cxx): Add -Wundefined-func-template.
This triggered one failure in the test suite:
* tests/headers.at (Sane headers): here, where we check that we can
compile the generated headers in other compilation units than the
parser's.
Add a variant type to make sure that basic_symbol and symbol_type are
properly generated in this case.
* data/skeletons/c++.m4 (basic_symbol): Inline the definitions of the
destructor and of `clear` in the class definition.
Avoid duplicate warnings about %error-verbose, once for deprecation,
another for duplicate. Keep only the duplicate warning for the second
occurrence of %error-verbose.
This will help removal fixits.
* src/scan-gram.l (%error-verbose): Return as a PERCENT_ERROR_VERBOSE
token.
* src/parse-gram.y (do_error_verbose): New.
Use it.
* src/muscle-tab.c (muscle_percent_variable_update): Handle pseudo
variables such as %error-verbose.
Currently, on
%define parser_class_name "Parser"
%define parser_class_name "Parser"
%%
exp:;
we issue:
foo.y:1.9-25: warning: deprecated directive, use '%define api.parser.class {Parser}' [-Wdeprecated]
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
foo.y:2.9-25: warning: deprecated directive, use '%define api.parser.class {Parser}' [-Wdeprecated]
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
foo.y:2.9-25: error: %define variable 'api.parser.class' redefined
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
foo.y:1.9-25: previous definition
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~
Let's get rid of the second warning about the deprecated variable
parser_class_name. This is noise, but it will also be a problem with
fixits for removing duplicates, as we will first generate the update,
and then it's too late to remove it: fixits do not edit the result of
previous fixits.
So generate this instead:
foo.y:1.1-34: warning: deprecated directive, use '%define api.parser.class {Parser}' [-Wdeprecated]
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
foo.y:2.1-34: error: %define variable 'api.parser.class' redefined
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
foo.y:1.1-34: previous definition
%define parser_class_name "Parser"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* src/muscle-tab.c (muscle_percent_variable_update): Pass the warning
to the caller, instead of issuing it.
(muscle_percent_define_insert): Issue this warning only if we don't
have to complain about a duplicate definition.
* tests/input.at: Adjust expectations.
* src/parse-gram.y: Use the location of the whole definition to record
the location of a %define variable, instead of just the name of the
variable.
Adjust tests.
Introduce proper support for fixits, instead of just printing them on
demand.
* bootstrap.conf: We need gnulib's xlists.
* src/fixits.h, src/fixits.c: New.
* src/complain.c (deprecated_directive): Use fixits_register.
* src/main.c (main): Use fixits_free.
Currently the diagnostics for %name-prefix are not precise enough. In
particular, they does not show that braces must be used instead of
quotes.
Before:
foo.y:3.1-14: warning: deprecated directive, use '%define api.prefix' [-Wdeprecated]
%name-prefix = "foo"
^^^^^^^^^^^^^^
After:
foo.y:3.1-20: warning: deprecated directive, use '%define api.prefix {foo}' [-Wdeprecated]
%name-prefix = "foo"
^^^^^^^^^^^^^^^^^^^^
To do this we need the value passed to %name-prefix, so move the
warning from the scanner to the parser.
Accuracy will be very important for the forthcoming changes.
* src/parse-gram.y (do_name_prefix): New.
(PERCENT_NAME_PREFIX): Have a semantic value: the raw source, with
possibly underscores, equal sign, and spaces. This is used to provide
a more accurate message. It does not take comments into account,
but...
* src/scan-gram.l (%name-prefix): Delegate the warnings to the parser.
* tests/headers.at, tests/input.at: Adjust expectations.
* src/muscle-tab.c (muscle_percent_variable_update): Avoid allocating
memory when it is not needed, which should be most of the time (when
there's no update to perform).
Adjust callers.
This line:
slice<stack_symbol_type, stack_type> slice (yystack_, yylen);
triggers warnings:
parse.h:1790:11: note: shadowed declaration is here
Reported by Frank Heckenbach.
http://lists.gnu.org/archive/html/bug-bison/2019-01/msg00002.html
* configure.ac (warn_c): Move -Wshadow to...
(warn_common): here.
* data/skeletons/stack.hh (slice): Define as an inner class of stack.
* data/skeletons/lalr1.cc: Adjust.
Rename the variable as 'range' instead of 'slice'.
There are many macros that are defined and used just
once (b4_public_if, b4_abstract_if, etc.). That's overkill. Rather,
let's define a macro to build the "public class YYParser" line.
It appears that the same syntax with "extends", "abstract", etc. is
implemented in the D parser, which looks very fishy...
* data/skeletons/d.m4, data/skeletons/java.m4 (b4_public_if)
(b4_abstract_if, b4_final_if, b4_strictfp_if): Replace with
(b4_parser_class_declaration): this.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java: Adjust.
Commit 90a8537e62 was right, but issued
two error messages. Commit 80ef7e7639
tried to address that by mapping yychar and yytoken to empty, but that
completely breaks the invariants of glr.c. In particular, yygetToken
can be called repeatedly and is expected to return the latest result,
unless yytoken is YYEMPTY. Since the previous attempt was "recording"
that the token was coming from an exception by setting it to YYEMPTY,
instead of getting again the faulty token, we fetched another one.
Rather, revert to the first approach: map yytoken to "invalid token",
but record in yychar the fact that we come from an exception thrown in
the scanner.
* data/skeletons/glr.c (YYFAULTYTOK): New.
(yygetToken): Use it to record syntax errors from the scanner.
* tests/c++.at (Syntax error as exception): In addition to checking
syntax_error with error recovery, make sure it also behaves as
expected without.
The previous name was historical and inconsistent.
* src/muscle-tab.c (define_directive): Use the proper value passing
syntax, based on the muscle kind.
(muscle_percent_variable_update): Use the right value passing syntax.
Migrate from parser_class_name to api.parser.class.
* data/skeletons: Migrate from parser_class_name to api.parser.class.
* doc/bison.texi (%define Summary): Document both parser_class_name
and api.parser.class.
Promote the latter over the former.
This is very debatable. This function is not pure at all, so it could
stick to returning void: that's a common coding style to tell the
difference between "real" (pure) functions and side-effecting
subroutines. However, we already have this style elsewhere (e.g.,
yylex), and I feel the callers are somewhat nice to read this way.
* data/skeletons/glr.c (yygetLRActions): Return the action rather than
passing by pointer.
While at it, fix type of yytoken.
Adjust callers.
Reported by Askar Safin.
https://lists.gnu.org/archive/html/bison-patches/2019-01/msg00000.html
* data/skeletons/glr.c (yygetToken): Return YYEMPTY when an exception
is thrown.
* data/skeletons/lalr1.cc: Log when an exception is caught.
* tests/c++.at (Syntax error as exception): Be sure to recover from
error before triggering another error.
This way, it is easier to make sure its implementation is available in
glr.cc too, which is not the case currently.
* data/skeletons/c++.m4 (b4_public_types_define): Move the
implementation of syntax_error...
(b4_public_types_declare): here.
* tests/local.at (AT_LOCATION_TYPE_IF): Turn into...
(AT_LOCATION_TYPE_SPAN_IF): this.
Adjust dependencies.
* tests/headers.at (Several parsers): Add another C++ parser,
which uses the first C++ parser's locations.
We used to create a short definition of yy::parser with all the
implementations of its member functions outside. But yy::parser is no
longer short and simple to read. Maintaining each function twice is
painful: a lot of redundancy but different indentation levels, output
which depends on whether we are in a header or not (see
d132c2d545), etc.
Let's simplify this and put the implementations into the class
definition itself.
Discussed in this monologue:
https://lists.gnu.org/archive/html/bison-patches/2018-12/msg00058.html.
* data/skeletons/c++.m4, data/skeletons/lalr1.cc,
* data/skeletons/variant.hh (b4_basic_symbol_constructor_define)
(_b4_token_constructor_declare, b4_token_constructor_declare)
Merge into...
(b4_basic_symbol_constructor_define, _b4_token_constructor_define)
(b4_token_constructor_define): these.
Commit 112ccb5ed7 moved the skeletons
from dist_pkgdata_DATA to dist_skeletons_DATA, hence broke the dependencies.
* Makefile.am (dependencies): New.
Use it where appropriate.
We used to define such auxiliary structures outside the class, mainly
as a matter of style to keep the definition of yy::parser short and
simple. However, now there's a lot more code generated inside the
class definition (e.g., all the token constructors), so the
readability no longer applies.
However, if we move stack (and slice) inside yy::parser, then it
should no longer be needed to change the namespace to have multiple
parsers: changing the class name should suffice.
One common argument against inner classes is that they code bloat. It
hardly applies here, since typically different parsers will have
different semantic value types, hence different actual stack types.
* data/skeletons/lalr1.cc: Invoke b4_stack_define inside yy::parser.
Suggested by David Barto
https://lists.gnu.org/archive/html/help-bison/2015-02/msg00004.html
and Victor Zverovich.
https://lists.gnu.org/archive/html/bison-patches/2018-10/msg00121.html
This is very easy to do, thanks to work by Bruno Haible in gnulib.
See "Supporting Relocation" in gnulib's documentation.
* bootstrap.conf: We need relocatable-prog and relocatable-script (for yacc).
* src/yacc.in: New.
* configure.ac, src/local.mk: Instantiate it.
* src/main.c, src/output.c (main, pkgdatadir): Use relocatable2.
* doc/bison.texi (FAQ): Document it.
Suggested by Wolfgang Thaller.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00081.html
* data/c++.m4 (basic_symbol, by_type): Instead of provide either move
or copy constructor, always provide the copy one.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Check it.
Currently the following piece of code crashes (with parse.assert),
because we don't record that s was moved-from, and we invoke its dtor.
{
auto s = parser::make_INT (42);
auto s2 = std::move (s);
}
Reported by Wolfgang Thaller.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00077.html
* data/c++.m4 (by_type): Provide a move-ctor.
(basic_symbol): Be sure not to read a moved-from value.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Check this case.
* tests/c++.at: Use AT_YYLEX_PROTOTYPE etc.
Which requires that we use the same argument names (lvalp, etc.).
* tests/local.at (AT_NAME_PREFIX): Fix regex.
Instead of introducing make_symbol (whose name, btw, somewhat
infringes on the user's "name space", if she defines a token named
"symbol"), let's make the construction of symbol_type safer, using
assertions.
For instance with:
%token ':' <std::string> ID <int> INT;
generate:
symbol_type (int token, const std::string&);
symbol_type (int token, const int&);
symbol_type (int token);
It does mean that now named token constructors (make_ID, make_INT,
etc.) go through a useless assert, but I think we can ignore this: I
assume any decent compiler will inline the symbol_type ctor inside the
make_TOKEN functions, which will show that the assert is trivially
verified, hence I expect no code will be emitted for it. And anyway,
that's an assert, NDEBUG controls it.
* data/c++.m4 (symbol_type): Turn into a subclass of
basic_symbol<by_type>.
Declare symbol constructors when variants are enabled.
* data/variant.hh (_b4_type_constructor_declare)
(_b4_type_constructor_define): Replace with...
(_b4_symbol_constructor_declare, _b4_symbol_constructor_def): these.
Generate symbol_type constructors.
* doc/bison.texi (Complete Symbols): Document.
* tests/types.at: Check.
On
%token <int> FOO BAR
we currently generate make_FOO(int) and make_BAR(int). However, in
order to factor their scanners, some users would also like to have
make_symbol(tok, int), where tok is FOO or BAR. To ensure type
safety, add assertions that do check that value type and token type
match. Bind this assertion to the parse.assert %define variable.
Suggested by Frank Heckenbach.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00034.html
Should also match expectations from Аскар Сафин.
http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00023.html
* data/variant.hh: Use b4_token_visible_if where applicable.
(_b4_type_constructor_declare, _b4_type_constructor_define): New.
Use them.
Bitten by macros, again.
See 680b715518.
* data/variant.hh (_b4_symbol_constructor_declare)
(_b4_symbol_constructor_define): Do not use user types, which can
include commas as in `std::pair<int, int>`, to macros.
* tests/local.at: Adjust the lex related macros to support the
case of token constructors.
* tests/types.at: Also check token constructors on types with commas.
Prompted by Rici Lake.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00000.html
We have four classes of directives that declare symbols: %nterm,
%type, %token, and the family of %left etc. Currently not all of them
support the possibility to have several type tags (`<type>`), and not
all of them support the fact of not having any type tag at all
(%type). Let's unify this.
- %type
POSIX Yacc specifies that %type is for nonterminals only. However,
some Bison users want to use it for both tokens and nterms
(actually, Bison's own grammar does this in several places, e.g.,
CHAR). So it should accept char/string literals.
As a consequence cannot be used to declare tokens with their alias:
`%type foo "foo"` would be ambiguous (are we defining foo = "foo",
or are these two different symbols?)
POSIX specifies that it is OK to use %type without a type tag. I'm
not sure what it means, but we support it.
- %token
Accept token declarations with number and string literal:
(ID|CHAR) NUM? STRING?.
- %left, etc.
They cannot be the same as %token, because we accept to declare the
symbol with %token, and to then qualify its precedence with %left.
Then `%left foo "foo"` would also be ambiguous: foo="foo", or two
symbols.
They cannot be simply a list of identifiers, but POSIX Yacc says we
can declare token numbers here. I personally think this is a bad
idea, precedence management is tricky in itself and should not be
cluttered with token declaration issues.
We used to accept declaring a token number on a string literal here
(e.g., `%left "token" 1`). This is abnormal. Either the feature is
useful, and then it should be supported in %token, or it's useless
and we should not support it in corner cases.
- %nterm
Obviously cannot accept tokens, nor char/string literals. Does not
exist in POSIX Yacc, but since %type also works for terminals, it is
a nice option to have.
* src/parse-gram.y: Avoid relying on side effects. For instance, get
rid of current_type, rather, build the list of symbols and iterate
over it to assign the type.
It's not always possible/convenient. For instance, we still use
current_class.
Prefer "decl" to "def", since in the rest of the implementation we
actually "declare" symbols, we don't "define" them.
(token_decls, token_decls_for_prec, symbol_decls, nterm_decls): New.
Use them for %token, %left, %type and %nterm.
* src/symlist.h, src/symlist.c (symbol_list_type_set): New.
* tests/regression.at b/tests/regression.at
(Token number in precedence declaration): We no longer accept
to give a number to string literals.
After having spent quite some time on cleaning the handling of symbol
declarations in the grammar files, I believe we should keep it.
It looks like it's a duplicate of %type, but it is not. While POSIX
Yacc requires %type to apply only to nonterminal symbols, it appears
that both byacc and bison accept it for tokens too. And some
experienced users do actually expect this feature to group
symbols (terminal or not) by type ("On the other hand, it is generally
more useful IMHO to group terminals and non-terminals with the same
type tag together",
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00000.html).
Even Bison's own parser does this today (see CHAR).
Basically reverts 7928c3e6fb.
* src/scan-gram.l (%nterm): Dedeprecate, but issue a Wyacc warning.
* tests/input.at: Adjust expectations.
(Yacc warnings on symbols): New.
* src/symtab.c (symbol_class_set): Fix error introduced in
20b0746793.
I personally prefer 'non terminal', or 'non-terminal', but
'nonterminal' is the common spelling.
* data/glr.c, src/parse-gram.y, src/symtab.c, src/symtab.h,
* tests/input.at, doc/refcard.tex: here.
Currently our error messages include both "symbol redeclared" and
"symbol redefined", and they mean something different. This is
obscure, let's make this clearer.
I think the idea between 'definition' vs. 'declaration' is that in the
case of the nonterminals, the actual definition is its set of rules,
so %nterm would be about declaration. The case of %token is less
clear.
* src/symtab.c (complain_class_redefined): New.
(symbol_class_set): Use it.
Simplify the logic of this function to clearly skip its body when the
preconditions are not met.
* tests/input.at (Symbol class redefinition): New.
Convert some of the READMEs to Markdown, which is now more common, and
nicely displayed in some git hosting services.
Add missing READMEs and Makefiles. Generate XML, HTML and Dot files. Be
sure to ship the test files. Complete CLEANFILES to remove all generated
files.
* examples/calc++: Move into...
* examples/c++: here.
* examples/mfcalc, examples/rpcalc: Move into...
* examples/c: here.
* examples/README.md, examples/c++/calc++/Makefile, examples/c/local.mk,
* examples/c/mfcalc/Makefile, examples/c/rpcalc/Makefile,
* examples/d/README.md, examples/java/README.md:
New files.
* examples/test (medir): Be robust to deeper directory nesting.
Revamping the handling of the symbols is the grammar is much more
delicate than I anticipated. Let's first move things around for
clarity.
* src/symtab.c (symbol_make_alias): Don't accept to alias
non-terminals.
(symbol_user_token_number_set): Don't accept user token numbers
for non-terminals.
Don't do anything in case of redefinition, instead of trying to
update. The flow is eaier to follow this way.
When we extract the examples from the documentation, %require
"@value{VERSION}" is replaced with the current version. If we change
the git branch, without changing the documentation, the generated
examples will %require a version of Bison that differs from the actual
version.
* examples/local.mk (extracted.stamp): Depend on doc/version.texi.
* data/README: Convert to Markdown.
Start documenting some of the macros used in all our skeletons.
Simplify and fix the documentation of the macros in the skeletons.
Currently it is the front end that passes the symbol types to the
backend. For instance:
%token <ival> NUM
%type <ival> exp1 exp2
exp1: NUM { $$ = $1; }
exp2: NUM { $<ival>$ = $<ival>1; }
In both cases, $$ and $1 are passed to the backend as having type
'ival' resulting in code like `val.ival`. This is troublesome in the
case of api.value.type=union, since in that the case the code this:
%define api.value.type union
%token <int> NUM
%type <int> exp1 exp2
exp1: NUM { $$ = $1; }
exp2: NUM { $<int>$ = $<int>1; }
because in this case, since the backend does not know the symbol being
processed, it is forced to generate casts in both cases: *(int*)(&val)`.
This is unfortunate in the first case (exp1) where there is no reason
at all to use a cast instead of `val.NUM` and `val.exp1`.
So instead delegate the computation of the actual value type to the
backend: pass $<ival>$ as `symbol-number, ival` and $$ as
`symbol-number, MULL`, instead of passing `ival` before.
* src/scan-code.l (handle_action_dollar): Find the symbol the action
is about, not just its tyye. Pass both symbol-number, and explicit
type tag ($<tag>n when there is one) to b4_lhs_value and b4_rhs_value.
* data/bison.m4 (b4_symbol_action): adjust to the new signature to
b4_dollar_pushdef.
* data/c-like.m4 (_b4_dollar_dollar, b4_dollar_pushdef): Accept the
symbol-number as new argument.
* data/c.m4 (b4_symbol_value): Accept the symbol-number as new
argument, and use it.
(b4_symbol_value_union): Accept the symbol-number as new
argument, and use it to prefer ready a union member rather than
casting the union.
* data/yacc.c (b4_lhs_value, b4_rhs_value): Accept the new
symbol-number argument.
Adjust uses of b4_dollar_pushdef.
* data/glr.c (b4_lhs_value, b4_rhs_value): Adjust.
* data/lalr1.cc (b4_symbol_value_template, b4_lhs_value): Adjust
to the new symbol-number argument.
* data/variant.hh (b4_symbol_value, b4_symbol_value_template): Accept
the new symbol-number argument.
* data/java.m4 (b4_symbol_value, b4_rhs_data): New.
(b4_rhs_value): Use them.
* data/lalr1.java: Adjust to b4_dollar_pushdef, and use b4_rhs_data.
* data/bison.m4, data/c++.m4, data/glr.c, data/java.m4, data/lalr1.cc,
* data/yacc.c, src/scan-code.l:
Fix comments.
Prefer POS to denote the position of a symbol in a rule, since NUM
is also used to denote symbol numbers.
Instead of defining yy::variant<S> and then alias
yy::parser::semantic_type to variant<sizeof (union_type)>, directly
define yy::parser::semantic_type.
This model is more appropriate if we want to sit the storage on top of
unions in C++11.
* data/variant.hh (b4_variant_define): Specialize and inline the
definition into...
(b4_value_type_declare): Here.
Define union_type here.
* data/lalr1.cc: Adjust.
* examples/java/Calc.y: New, based on test 495: "Calculator
parse.error=verbose %locations".
* examples/java/Calc.test, examples/java/local.mk: New.
* configure.ac (ENABLE_JAVA): New.
* examples/test (prog): Be ready to run Java programs.
This was demanded several times. See for instance:
- David M. Warme
https://lists.gnu.org/archive/html/help-bison/2011-04/msg00003.html
- box12009
http://lists.gnu.org/archive/html/bug-bison/2016-10/msg00001.html
Basically, this reverts:
- commit 3d3bc1fe30
Get rid of (yy)rhs and (yy)prhs
- commit d333175f63
Avoid compiler warning.
Note that since these tables are not needed in the generated parsers,
no skeleton requests them. This change only brings back their
definition to M4, making it possible to user-defined skeletons to use
these tables.
* src/output.c (muscle_insert_item_number_table): Define.
(prepare_rules): Generate the rhs and prhs tables.
Reported by Rici Lake.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00000.html
* src/complain.h: Formatting change.
* src/parse-gram.y (id): Reject character literals used in a context
for non-terminals.
* tests/input.at (Invalid %nterm uses): Check that.
Reported by Uxio Prego.
https://lists.gnu.org/archive/html/help-bison/2018-11/msg00031.html
We also need to move the unreachable 'goto' to a reachable place,
otherwise clang complains about the code being unreachable anyway.
See also https://bugs.llvm.org/show_bug.cgi?id=39736.
Interestingly, we don't have to apply that trick to
`#define YYCDEBUG if (false) std::cerr`, clang does not warn when the
code comes from macro expansion.
* configure.ac: Use -Wunreachable-code when supported.
* data/lalr1.cc, data/yacc.c: Pacify clang's warning about `if (0)`
by using a macro.
Another possibility was to move this statement to a reachable place.
* tests/actions.at, tests/c++.at: Avoid generating unreachable code.
We should probably introduce some struct and functions to deal with
stack management, rather than open coding it. yyparse would be much
nicer to read, and a better model for possible other skeletons.
* data/yacc.c (yyparse::yysetstate): Avoid generating code when
neither yyoverflow nor YYSTACK_RELOCATE is defined.
Currently on a grammar such as
exp : a '1' | a '2' | a '3' | b '1' | b '2' | b '3'
a:
b:
we count only one rr-conflict on the `b:` rule, i.e., we expect:
b: %expect-rr 1
although there are 3 conflicts in total. That's because in the
conflicted state we count only a single conflict, not three (one for
each of the lookaheads: '1', '2', '3').
State 0
0 $accept: . exp $end
1 exp: . a '1'
2 | . a '2'
3 | . a '3'
4 | . b '1'
5 | . b '2'
6 | . b '3'
7 a: . %empty ['1', '2', '3']
8 b: . %empty ['1', '2', '3']
'1' reduce using rule 7 (a)
'1' [reduce using rule 8 (b)]
'2' reduce using rule 7 (a)
'2' [reduce using rule 8 (b)]
'3' reduce using rule 7 (a)
'3' [reduce using rule 8 (b)]
$default reduce using rule 7 (a)
exp go to state 1
a go to state 2
b go to state 3
See https://lists.gnu.org/archive/html/bison-patches/2013-02/msg00106.html.
* src/conflicts.c (rule_has_state_rr_conflicts): Rename as...
(count_rule_state_sr_conflicts): this.
DWIM.
(count_rule_rr_conflicts): Adjust.
* tests/conflicts.at (%expect-rr in grammar rules)
(%expect-rr too much in grammar rules)
(%expect-rr not enough in grammar rules): New.
On a grammar such as
exp: "num" | "num" | "num"
we currently report only one RR conflict, instead of two.
This bug is present since the origins of Bison
commit 08089d5d35
Author: David MacKenzie <[email protected]>
Date: Tue Apr 20 05:42:52 1993 +0000
Initial revision
and was preserved in
commit 676385e29c
Author: Paul Hilfinger <[email protected]>
Date: Fri Jun 28 02:26:44 2002 +0000
Initial check-in introducing experimental GLR parsing. See entry in
ChangeLog dated 2002-06-27 from Paul Hilfinger for details.
See
https://lists.gnu.org/archive/html/bison-patches/2018-11/msg00011.html
* src/conflicts.h, src/conflicts.c (count_state_rr_conflicts)
(count_rr_conflicts): Use only the correct count of conflicts.
* tests/glr-regression.at: Fix expectations.
Currently on a grammar such as
exp: "number" | exp "+" exp | exp "*" exp
we count only one sr-conflict for both binary rules, i.e., we expect:
exp: "number" | exp "+" exp %expect 1 | exp "*" exp %expect 1
although there are 4 conflicts in total. That's because in the states
in conflict, for instance that for the "+" rule:
State 6
2 exp: exp . "+" exp
2 | exp "+" exp . [$end, "+", "*"]
3 | exp . "*" exp
"+" shift, and go to state 4
"*" shift, and go to state 5
"+" [reduce using rule 2 (exp)]
"*" [reduce using rule 2 (exp)]
$default reduce using rule 2 (exp)
we count only a single conflict, although there are two (one on "+"
and another with "*").
See https://lists.gnu.org/archive/html/bison-patches/2013-02/msg00106.html.
* src/conflicts.c (rule_has_state_sr_conflicts): Rename as...
(count_rule_state_sr_conflicts): this.
DWIM.
(count_rule_sr_conflicts): Adjust.
* tests/conflicts.at (%expect in grammar rules): New.
This change allows one to document (and check) which rules participate
in shift/reduce and reduce/reduce conflicts. This is particularly
important GLR parsers, where conflicts are a normal occurrence. For
example,
%glr-parser
%expect 1
%%
...
argument_list:
arguments %expect 1
| arguments ','
| %empty
;
arguments:
expression
| argument_list ',' expression
;
...
Looking at the output from -v, one can see that the shift-reduce
conflict here is due to the fact that the parser does not know whether
to reduce arguments to argument_list until it sees the token AFTER the
following ','. By marking the rule with %expect 1 (because there is a
conflict in one state), we document the source of the 1 overall shift-
reduce conflict.
In GLR parsers, we can use %expect-rr in a rule for reduce/reduce
conflicts. In this case, we mark each of the conflicting rules. For
example,
%glr-parser
%expect-rr 1
%%
stmt:
target_list '=' expr ';'
| expr_list ';'
;
target_list:
target
| target ',' target_list
;
target:
ID %expect-rr 1
;
expr_list:
expr
| expr ',' expr_list
;
expr:
ID %expect-rr 1
| ...
;
In a statement such as
x, y = 3, 4;
the parser must reduce x to a target or an expr, but does not know
which until it sees the '='. So we notate the two possible reductions
to indicate that each conflicts in one rule.
See https://lists.gnu.org/archive/html/bison-patches/2013-02/msg00105.html.
* doc/bison.texi (Suppressing Conflict Warnings): Document %expect,
%expect-rr in grammar rules.
* src/conflicts.c (count_state_rr_conflicts): Adjust comment.
(rule_has_state_sr_conflicts): New static function.
(count_rule_sr_conflicts): New static function.
(rule_nast_state_rr_conflicts): New static function.
(count_rule_rr_conflicts): New static function.
(rule_conflicts_print): New static function.
(conflicts_print): Also use rule_conflicts_print to report on individual
rules.
* src/gram.h (struct rule): Add new fields expected_sr_conflicts,
expected_rr_conflicts.
* src/reader.c (grammar_midrule_action): Transfer expected_sr_conflicts,
expected_rr_conflicts to new rule, and turn off in current_rule.
(grammar_current_rule_expect_sr): New function.
(grammar_current_rule_expect_rr): New function.
(packgram): Transfer expected_sr_conflicts, expected_rr_conflicts
to new rule.
* src/reader.h (grammar_current_rule_expect_sr): New function.
(grammar_current_rule_expect_rr): New function.
* src/symlist.c (symbol_list_sym_new): Initialize expected_sr_conflicts,
expected_rr_conflicts.
* src/symlist.h (struct symbol_list): Add new fields expected_sr_conflicts,
expected_rr_conflicts.
* tests/conflicts.at: Add tests "%expect in grammar rule not enough",
"%expect in grammar rule right.", "%expect in grammar rule too much."
We also lack a consistent naming for directive implementations.
`directive_skeleton` is too long, `percent_skeleton` is not very nice
looking, `process_skeleton` looks ambiguous, `do_skeleton` is somewhat
ambiguous too, but seems a better track.
* src/parse-gram.y (version_check): Rename as...
(do_require): this.
(do_skeleton): New.
Use it.
We may generate code such as
basic_symbol (typename Base::kind_type t, YY_RVREF (std::pair<int,int>) v);
which, of course, breaks, because YY_RVREF sees two arguments. Let's
not play tricks with _VA_ARGS__, I'm unsure about it portability.
Anyway, I plan to change more things in this area.
Reported by Sébastien Villemot.
http://lists.gnu.org/archive/html/bug-bison/2018-11/msg00014.html
* data/variant.hh (b4_basic_symbol_constructor_declare)
(b4_basic_symbol_constructor_define): Don't use macro on user types.
* tests/types.at: Check that we support pairs.
In C++, [[noreturn]] must not be between "static" and the rest of the
function signature, it must precede it. C's _Noreturn does not seem
to have such a constraint, but it is therefore compatible with the C++
constraint. Since we #define _Noreturn as [[noreturn]] is modern C++,
be sure to push the _Noreturn first.
Unfortunately this was not caught by the test suite, because it always
loads config.h first, and config.h contains another definition of
_Noreturn that does not use [[noreturn]], and hides ours. That's
probably a sign we should avoid always loading config.h.
* data/glr.c (yyFail, yyMemoryExhausted): here.
* tests/local.at (AT_LANG_FOR_EACH_STD): New.
(AT_REQUIRE_CXX_VERSION): Rename as...
(AT_REQUIRE_CXX_STD): this.
Accept an argument for what to do when the requirement is not met.
* tests/types.at (api.value.type): Check all the C++ stds.
The following commit introduce even more compilations/runs than
before, and with ASAN on, we go beyond to 50min credit from Travis.
* .travis.yml (Clang 7 libc++ and ASAN): Split in two.
It is unfortunate that %error_verbose was properly diagnosed as
obsoleted by "%define parse.error verbose", but %error-verbose was
not.
* src/parse-gram.y (%error-verbose): Remove support.
* src/scan-gram.l: Do it here instead, with a warning.
* tests/input.at (Deprecated directives): Check it.
These tests are skipped with GCC:
"\"".c:1:5: error: function declaration isn't a prototype [-Werror=strict-prototypes]
int main() { return 0; }
^~~~
* tests/synclines.at: Stop writing C++ in C.
* tests/local.at: Formatting changes.
* src/parse-gram.y (api.value.type): Set to union.
Replace occurrences of %union with explicit %types.
* src/scan-gram.l: Adjust yylval's field names.
(RETURN_VALUE): No longer needs the Field argument.
Use it more.
Support for DJGPP was announced to be removed in the NEWS of Bison
3.1 (2018-08-27) unless someone expressed interest. There was no answer.
* djgpp: Remove.
* NEWS, Makefile.am, cfg.mk, po/POTFILES.in: Adjust.
Reported by Andre da Costa Barros.
https://savannah.gnu.org/patch/?9716
* examples/calc++/local.mk: We no longer generate position.hh and
stack.hh. Leaving them here triggers their concurrent generation,
which fails.
(%C%_calc___CPPFLAGS): Fix the extracted headers in the source tree.
* examples/mfcalc/local.mk (%C%_mfcalc_CPPFLAGS): Ditto.
Currently, the examples are extracted on the user's side.
Unfortunately, that requires that the user has Perl, which is
otherwise not needed for Bison. Let's ship the examples instead.
The examples were handled this way so that we could depend on
configure flags: if --enable-gcc-warnings is passed, it is understood
as "I'm a maintainer", so the examples are generated with `#line`s.
Regular users should not see them, so they are now unconditionally
removed when rolling a tarball.
Reported by Mike Frysinger.
https://lists.gnu.org/archive/html/bison-patches/2015-04/msg00000.html
* examples/local.mk: Ship all the extracted files.
(examples-unline): New.
Make sure that the generated tarballs do not contain the #lines.
Also, make sure that `make dist` generates a correct tarball even if
the C++ compiler does not work.
Reported by Nelson H. F. Beebe.
* m4/cxx.m4 (BISON_CXX_WORKS): Define to true/false instead of
true/exit 77. The latter is too dangerous to use (it directly quits).
(ENABLE_CXX): New name for the Automake conditional, for consistency
with ENABLE_CXX11 etc.
* tests/local.at (AT_COMPILE, AT_COMPILE_CXX): Adjust to the new
semantics of BISON_CXX_WORKS.
* examples/c++/local.mk: Skip the variant test if C++ does not work.
* examples/calc++/local.mk: Likewise.
On some systems (x86_64-pc-solaris2.11), with Developer Studio 12.5's
CC, we get:
".../include/CC/Cstd/vector.cc", line 127: Error: Cannot assign const yy::parser::stack_symbol_type to yy::parser::stack_symbol_type without "yy::parser::stack_symbol_type::operator=(const yy::parser::stack_symbol_type&)";.
".../include/CC/Cstd/vector", line 475: Where: While instantiating "std::vector<yy::parser::stack_symbol_type>::__insert_aux(yy::parser::stack_symbol_type*, const yy::parser::stack_symbol_type&)".
".../include/CC/Cstd/vector", line 475: Where: Instantiated from non-template code.
1 Error(s) detected.
Don't expect __cplusplus to be always defined. If it's not, consider
this is C++98.
Reported by Nelson H. F. Beebe.
* data/c++.m4, data/lalr1.cc, examples/c++/variant.yy, tests/local.at,
* tests/testsuite.h:
An undefined __cplusplus means pre C++11.
On some systems, we don't use our getopt. As a consequence the error
messages vary:
$ bison --skeleton
bison: option requires an argument -- skeleton
Try 'bison --help' for more information.
instead of
bison: option '--skeleton' requires an argument
Try 'bison --help' for more information.
Reported by Jannick and Nelson H. F. Beebe.
https://lists.gnu.org/archive/html/bison-patches/2018-10/msg00140.html
* tests/input.at (Invalid number of arguments): work around getopt
portability issues.
On Solaris, GCC 7.3 defines:
-std=c++14 -std=c++17
__cplusplus 201402L 201703L
__STDC_VERSION__ 199901L 201112L
So the current #definition of _Noreturn sees that 201112 <=
__STDC_VERSION__, i.e., that C11 is supported, so it expects _Noreturn
to be supported. Apparently it is not.
This is only for C++, the test suite works for C. However, the test
suite does not try several C standards, maybe we should...
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00064.html
* data/c.m4 (b4_attribute_define): Define _Noreturn as [[noreturn]] in
modern C++.
Does not work on Solaris 11.3 x86/64:
479. c++.at:1293: testing C++ GLR parser identifier shadowing ...
======== Testing with C++ standard flags: '-std=c++17'
./c++.at:1332: $BISON_CXX_WORKS
stderr:
stdout:
./c++.at:1332: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o input input.cc $LIBS
stderr:
input.cc:837:8: error: '_Noreturn' does not name a type
static _Noreturn void
^~~~~~~~~
input.cc:845:8: error: '_Noreturn' does not name a type
static _Noreturn void
^~~~~~~~~
Reported by Kiyoshi Kanazawa.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00051.html
* data/c.m4 (b4_attribute_define): Use the snippet which is currently
in gnulib's m4/gnulib-common.m4 (which seems a little more advanced
than lib/_Noreturn.h).
The "Report translation bugs to..." part of --help is issued only on
glibc systems. So if the tarball is not wrapped on such a system, and
used on such a system (or the converse), then bison.help will differ
on the user's system, and help2man will be called to update bison.1.
But help2man should not be a requirement.
Reported by Alexandre Duret-Lutz.
* doc/local.mk (doc/bison.help): Remove the possible doc about
translation bugs.
Pass LC_ALL=C, as reported in src/getargs.c's usage().
(doc/cross-options.texi): Use bison.help instead of calling bison
--help.
Some users rely on this sentence to know that the file can be ignored.
Reported by Alexandre Duret-Lutz.
* data/bison.m4 (b4_generated_by): New.
(b4_copyright): Use it.
* data/location.cc, data/stack.hh: Use it too, for the stub files
(position.hh and stack.hh).
* doc/bison.texi (Calc++ Scanner): Show how exception can be thrown
from auxiliary functions.
Clarify the meaning of the various flex %options we use.
Get rid of a warning.
(Calc++ Parsing Driver): Use the parser as a functor.
* doc/bison.texi: Simplify wording.
Fix Texinfo error.
(Complete Symbols): Handle the token EOF.
(Calc++ Parser): In the modern C++ world, prefer assignment to swap.
(Strings are Destroyed): Prefer an explicit 'continue' to a comment.
The CI is using Flex 2.5.35. And ICC is too picky for it. Let's stop
making these warnings errors. I wish I could disable them in the
source files using the ICC version and the Flex version, but ICC's
pragma support is unclear, and I'm tired of fighting it.
* configure.ac (FLEX_SCANNER_CXXFLAGS): Make warnings warnings.
* examples/c++/local.mk: Comment changes.
Suggested by Victor Khomenko.
http://lists.gnu.org/archive/html/bug-bison/2018-08/msg00037.html
* doc/bison.texi (A Simple C++ Example): New.
* examples/c++/local.mk, examples/c++/simple.test: New.
Extract, check, and install this new example.
* examples/local.mk: Adjust.
* examples/test: Adjust to the case where the dirname differs
from the test name.
* build-aux/local.mk, cfg.mk, examples/calc++/local.mk,
* examples/local.mk, examples/mfcalc/local.mk,
* examples/rpcalc/local.mk, lib/local.mk, src/local.mk,
* tests/local.mk:
Use Automake comments so that we don't get a copy of each in the
generated Makefile.
On the CI, both GCC and Clang report:
src/parse-gram.c: In function 'yy_lac':
src/parse-gram.c:1479:29: error: format '%hd' expects argument of type 'int',
but argument 3 has type 'yytype_int16 {aka long int}' [-Werror=format=]
YYDPRINTF ((stderr, " G%hd", yystate));
^
Although yytype_int16 is supposed to be a short int, not a long int.
This must be explored.
* data/yacc.c (yy_lac): Work around typing issue.
* data/glr.c: Don't use `foo |= bar` with foo and bar being yybool:
the result appears to be an int, not a yybool.
Use yybool where appropriate.
Add casts where needed.
Reported by Derek Clegg.
http://lists.gnu.org/archive/html/bug-bison/2018-10/msg00018.html
Rather than adding casts, we should be more careful with types. For
instance yystate should be a yytype_int16. But currently we can't: it
is also used sometimes for storing other things that state numbers.
* data/yacc.c (yyparse): Add missing casts.
The yacc.c skeleton is old, and was using many tricks to save
registers. Today's register allocators can do this themselves. Let's
keep the code simpler to read and let compilers do their job.
* data/yacc.c: Avoid using yystate for different types of content.
An inline function would be better, but doing this portably will be
a problem.
* data/lalr1.cc: Fix oldish comment.
* data/stack.hh: Prefer typename for type names.
Use size() instead of duplicating it.
* examples/variant-11.yy, examples/variant.yy (yylex): Use int,
as this is the type of the semantic value.
Currently we emit useless code for places where we might issue user
content, but there is none. This commit avoids this. Besides, some
of the comments looked like implementation details ("Copy the first
part of user declarations"), rather than made for the reader of the
result ("First part of user prologue").
On Bison's parse-gram.c we get:
@@ -76,10 +76,6 @@
#define yynerrs gram_nerrs
-/* Copy the first part of user declarations. */
-
-#line 82 "src/parse-gram.c" /* yacc.c:339 */
-
* data/bison.m4 (b4_define_user_code): Accept a comment to document
the section.
Do not emit any code if the content is empty.
Adjust callers to not emit the comment.
Do not
* data/glr.c, data/glr.cc, data/lalr1.cc, data/lalr1.java, data/yacc.c:
Adjust.
The previous name is too obscure, and the other macros for C++ use
CXX, not CC.
* tests/local.at (AT_SKEL_CC_IF, AT_SKEL_JAVA_IF): Rename as...
(AT_CXX_IF, AT_JAVA_IF): these.
Adjust callers.
When we introduced variants in Bison, C++ did not have the 'emplace'
functions, and we chose 'build'. Let's align with modern C++ and
promote 'emplace' rather than 'build'.
* data/lalr1.cc, data/variant.hh (emplace): New.
(build): Deprecate in favor of emplace.
* doc/bison.texi: Adjust.
As a matter of fact, I think it is wrong to gitignore generated files
that belong to the build tree. There should be the strict minimum,
and it's up to people that build in place to adjust their own
~/.gitignore.
* doc/.gitignore: here.
Remove files we no longer produce (thanks to texi2dvi).
In the previous commit we fixed a problem when the C++ stack was
resized. The test was using ints. Let's add a test with someone
quite touchy: unique_ptr
* examples/variant-11.yy: Accept an argument, which is the number of
numbers to send to the parser.
* examples/variant-11.test: Check with many numbers.
In some casing, once we moved a stack symbol, we forget to mark the
source stack symbol as emptied. As a consequence, it may be destroyed
a second time.
This happens when the stack has to be resized.
* data/lalr1.cc (stack_symbol_type::stack_symbol_type): Record that
the source was emptied.
(stack_symbol_type::operator=): Likewise.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Force the stack
to be resized. Check its content.
The files stack.hh and position.hh are deprecated. Rather than
devoting specify %define variables to discard them (api.position.file
and api.stack.file), and rather than having to use special rules when
api.location.file is used, let's simply decide that from %require
"3.2" onwards, these files will not be generated.
The only noticeable thing here is that, in order to be able to check
the behavior of %require "3.2", to have this version (which is still
3.1-*) to accept %require "3.2".
* src/gram.h, src/gram.c (required_version): New.
* src/parse-gram.y (version_check): Set it.
* src/output.c (prepare): Pass it m4.
* data/bison.m4 (b4_required_version_if): Receive it and use it.
* data/location.cc, data/stack.hh: Replace the api.*.file with only
required version comparison.
* tests/input.at: No longer check api.stack.file and api.position.file.
* NEWS, doc/bison.texi: Don't mention them.
Document the %require 3.2 behavior.
* tests/output.at: Use %require 3.2 instead.
Currently, in C, the default semantic action is implemented by being
always run before running the actual user semantic action. As a
consequence, when the user action is run, $$ is already set as $1.
In C++ with variants, we don't do that, since we cannot manipulate the
semantic value without knowing its exact type. When variants are
enabled, the only guarantee is that $$ is default contructed and ready
to the used.
Some users still would like the default action to be run with
variants. Frank Heckenbach's parser in
C++17 (http://lists.gnu.org/archive/html/bug-bison/2018-04/msg00011.html)
provides this feature, but relying on std::variant's dynamic typing,
which we forbid in lalr1.cc.
The simplest seems to be actually generating the default semantic
action (in all languages/skeletons). This makes the pre-action (that
sets $$ to $1) useless. But... maybe some users depend on this, in
spite of the comments that clearly warn againt this. So let's not
turn this off just yet.
* src/reader.c (grammar_rule_check_and_complete): Rename as...
(grammar_rule_check_and_complete): this.
Install the default semantic action when applicable.
* examples/variant-11.yy, examples/variant.yy, tests/calc.at:
Exercise the default semantic action, even with variants.
Currently we use "<dir><api.location.file>" as \file argument, and as
base for the CPP guard. This is not nice when <dir> is absolute, in
which case it is expected that the user will use api.location.include
to get something nicer. If defined, use that name instead.
* data/location.cc (b4_location_path): New.
Use it.
* tests/c++.at (Shared locations): Check the guard and Doxygen doc.
Now that distcheck no longer fails (see previous commit), let's
address the shortcomings.
* Makefile.am (CLEANDIRS, clean-local): New.
* doc/local.mk, examples/calc++/local.mk, examples/local.mk,
* examples/mfcalc/local.mk, examples/rpcalc/local.mk,
* src/local.mk
(CLEANDIRS): Get rid of Apple's *.dSYM directories.
(CLEANFILES): Get rid of *.output files.
* examples/variant-11.yy, examples/variant.yy: Don't generate
any of the auxiliary files (location.hh and the like).
`make update-po` runs:
package_gnu="$(PACKAGE_GNU)"; \
test -n "$$package_gnu" || { \
if { if (LC_ALL=C find --version) 2>/dev/null | grep GNU >/dev/null; then \
LC_ALL=C find -L $(top_srcdir) -maxdepth 1 -type f \
-size -10000000c -exec grep 'GNU @PACKAGE@' \
/dev/null '{}' ';' 2>/dev/null; \
else \
LC_ALL=C grep 'GNU @PACKAGE@' $(top_srcdir)/* 2>/dev/null; \
fi; \
} | grep -v 'libtool:' >/dev/null; then \
package_gnu=yes; \
else \
package_gnu=no; \
fi; \
}; \
and based on the result, put GNU or not in the following line from
bison.pot:
# This file is distributed under the same license as the GNU bison package.
It turns out that in my environment some log files had the 'GNU bison'
string (note the lower case), but in distcheck, these files are no
longer visible, so the generate bison.pot was different, and distcheck
failed because we try to update bison.pot, which is read only in
distcheck.
The heuristics should look accept 'GNU Bison', not just 'GNU bison'.
But let's please it to make sure we have our 'GNU'.
* README: Mention 'GNU bison'.
Users may want to generate the location file elsewhere, say
$top_srcdir/include/ast/location.hh. Yet, we should not generate
`#include "$top_srcdir/include/ast/location.hh"` but probably
something like `#include <ast/location.hh>`, or `#include
"ast/location.hh", or `#include <location.hh>`. It entirely depends
on the compiler flags (-I/-isystem) that are used. Bison cannot guess
what is expected, so let's give the user a means to tell how the
location file should be included.
* data/location.cc (b4_location_file): New.
* data/glr.cc, data/lalr1.cc: Use it.
In the case a user wants to create location.hh elsewhere, it can be
helpful to define api.location.file to some possibly absolute path
such as -Dapi.location.file='"$(top_srcdir)/include/ast/location.hh"'.
Currently this does not work with `-o foo/parser.cc`, as we join foo/
and $(top_srcdir) together, the latter starting with slash.
We should not try to do that in m4, manipulating file names is quite
complex when you through Windows file name in. Let m4 delegate this
to gnulib.
* src/scan-skel.l (at_output): Accept up to two arguments.
* data/bison.m4 (b4_output): Adjust.
* tests/skeletons.at (Fatal errors but M4 continues producing output):
Adjust to keep the error.
* data/location.cc, data/stack.hh: Leave the concatenation to @output.
* tests/output.at: Exercise api.location.file with an absolute path.
Make it easier to have fewer files.
* data/stack.hh: Don't generate stack.hh when api.location.file is
specified.
* tests/calc++.at, tests/output.at: Adjust tests.
Let's put the definition of position into location.hh, there's no real
value in keeping them separate: they are small, and share the same
requirements.
To help users transition to this new model, still generate position.hh
by default, but as a simple include to location.hh.
* data/location.cc (api.position.file): Accept only 'none' as possible
value.
(position.hh): Make it a stub.
(location.hh): Adjust.
(b4_position_define): Merge into...
(b4_location_define): this.
* data/glr.cc, data/lalr1.cc, tests/input.at, tests/output.at: Adjust.
* data/location.cc: Sort includes.
(b4_position_file, b4_location_file): New.
When there's a file for locations but not for positions, include the
definition of position in the location file.
* data/lalr1.cc (b4_shared_declarations): Include the
position/location file when it exists.
Otherwise, define the class.
* data/glr.cc: Likewise.
* tests/input.at (%define file variables): Check them.
* tests/output.at (C++ output): Check various cases with
api.position.file and api.location.file.
It was not a good idea to generate the file stack.hh. It never was.
But now we have to deal with backward compatibility: if we stop
generating it, the build system of some build system will probably
break.
So offer the user a means to (i) decide what the name of the output
file should be, and (ii) not generate this file at all (its content
will be inline where the parser is defined).
* data/lalr1.cc (b4_percent_define_check_file_complain)
(b4_percent_define_check_file): New.
* data/stack.hh: Generate the file only if api.stack.file is not
empty.
In that case, use it as file name.
* data/lalr1.cc: Adjust to include the right file, or to include
the definition of stack.
* tests/calc.at, tests/output.at: Exercise api.stack.file.
They are not the same concept. It appears that we still consider that
api.prefix is the default for api.namespace. We should stop that.
* tests/local.at (AT_BISON_OPTION_PUSHDEFS, AT_BISON_OPTION_POPDEFS):
Separate namespace and prefix.
Adjust dependencies.
* src/getargs.c (getargs): Don't display any argv other that argv[0]
when reporting a missing argument.
* tests/bison.in: Neutralize path differences in stderr.
* tests/input.at (Invalid number of arguments): New.
* src/getargs.c (usage): Rely on setlocale (LC_MESSAGES, NULL)
trick only on glibc, as POSIX does not specify the output
of setlocale in this case, and the Gnulib localename module
source code indicates that the trick works only on glibc.
C11 no longer requires support for variable-length arrays, and
VS2015 does not have them. Redo UNIQSTR_CONCAT to use a method
that is simpler and better anyway.
* src/uniqstr.c (uniqstr_vsprintf): Remove; no longer needed.
* src/uniqstr.h (UNIQSTR_GEN_FORMAT, UNIQSTR_GEN_FORMAT_):
* src/uniqstr.c (uniqstr_concat): New function.
* src/uniqstr.h (UNIQSTR_CONCAT): Use it instead of using
uniqstr_vsprintf.
* doc/bison.texi: Minor fixes in typography.
It is no longer require to pass --defines for C++ (it was addressed
long ago).
No longer refer to the `variant` define variable, it was replaced by
`api.value.type variant`.
Prefer nullptr to 0 for the null pointer.
Use deftypeop for constructors.
(Complete Symbols): Give the expected signature of yylex.
Don't document the symbol_type constructors, as we want users to focus
on make_TOKEN.
Also show the case without locations.
Bison's test 464 (Syntax error as exception) fails on the CI.
Do not use clang with asan on Ubuntu's libc++.
https://bugs.llvm.org/show_bug.cgi?id=17379
* .travis.yml (Clang 7 libc++ and ASAN): New.
(Clang 6 -O3 and libc++): Really use libc++.
(Clang 5): Don't use libc++, nor asan (does not work either, same
reason).
* .travis.yml: Don't define CC/CXX, it does not work.
Use `[[...]]` instead of `[...]`.
Show the compiler versions.
(Clang 3.3, Clang 3.4): Specify the path to avoid using
/usr/local/clang-5.0.0/bin's clang.
The message on configure is misleading:
checking whether the linker accepts -std=c++11... yes
checking whether the linker accepts -std=c++14... yes
checking whether the linker accepts -std=c++17... no
It is the compiler that we check, not just the linker.
* m4/ax_check_link_flag.m4: Remove.
* m4/bison-check-compiler-flag.m4: New.
* m4/bison-cxx-std.m4: Use it.
This has been bugging me for while. I was hard to reproduce: it
worked only on GNU/Linux, probably because libc++ implements the small
string optimization, while libstdc++ did not and actually allocated on
the heap for this small string.
See https://lists.gnu.org/archive/html/bison-patches/2018-09/msg00110.html.
* tests/types.at (api.value.type): Do not provide a semantic value to
EOF.
The forthcoming automove feature, to be properly checked, will require
that we can rely on the value of a moved-from string, which is not
something the C++ standard guarantees. So introduce our own wrapper.
Suggested by Frank Heckenbach.
https://lists.gnu.org/archive/html/bison-patches/2018-09/msg00111.html
* tests/c++.at (Variants): Introduce and use a new 'string' class.
The 'Variants' tests are well suited to check support for move, and in
particular for the forthcoming automove feature. But the tests were
written to show the best practice in C++98, using swap:
list "," item { std::swap ($$, $1); $$.push_back ($3); }
This cannot work with std::move. So, make this example simpler, based
on regular assignment instead of swap, which is a regression for
C++98 (as the new traces show), but will be an improvement for modern
C++ with automove.
* tests/c++.at (Variants): Stop using swap.
We don't generate a header file, so remove the 'require' code section.
Adjust expectations.
Several types of failures. First, unable to pass the file name
properly to the linker.
./synclines.at:416: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o \"\\\"\" \"\\\"\".c $LIBS
stderr:
ld: cannot open output file "/"": No such file or directory
stdout:
Unable to save under such a file name.
./synclines.at:421: $CXX $CXXFLAGS $CPPFLAGS -c $LDFLAGS -o \"\\\"\" \"\\\"\".cc $LIBS
stderr:
error: can't open file "/"" for write
compilation aborted for "\"".cc (code 1)
Spurious output because of warning flags is failed to reject as an
error during configure:
./headers.at:343: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS c-only.o cxx-only.o -o c-and-cxx ||
exit 77
--- /dev/null 2018-09-18 21:21:37.745649000 +0000
+++ /home/travis/build/akimd/bison/tests/testsuite.dir/at-groups/222/stderr 2018-09-18 21:28:17.291919519 +0000
@@ -0,0 +1,7 @@
+icpc: command line warning #10006: ignoring unknown option '-Wcast-align'
+icpc: command line warning #10006: ignoring unknown option '-fparse-all-comments'
+icpc: command line warning #10006: ignoring unknown option '-Wdocumentation'
+icpc: command line warning #10006: ignoring unknown option '-Wnull-dereference'
+icpc: command line warning #10006: ignoring unknown option '-Wnoexcept'
+icpc: command line warning #10006: ignoring unknown option '-fno-color-diagnostics'
+icpc: command line warning #10006: ignoring unknown option '-Wno-keyword-macro'
stdout:
* tests/local.at (AT_SKIP_IF_CANNOT_LINK_C_AND_CXX): Also ignore
stderr, as with ICC we get
* tests/synclines.at (syncline escapes): Don't link the output.
The code was already using midrule only, never mid_rule. This is
simpler to remember, and matches a similar change we made from
look-ahead to lookahead.
* NEWS, doc/bison.texi, src/reader.c, src/scan-code.h, src/scan-code.l
* tests/actions.at, tests/c++.at, tests/existing.at: here.
We use both styles, let's stick to a single one. Autoconf uses the
prefix one, let's do the same.
* data/bison.m4, data/c++.m4, data/c-like.m4, data/lalr1.cc,
* data/variant.hh, data/yacc.c: Rename all the b4_*_ macros
as _b4_*.
On the CI, we had failures such as:
./c++.at:401: $PREPARSER ./list
stderr:
./list: error while loading shared libraries: libc++.so.1:
cannot open shared object file: No such file or directory
because we accepted `-std=c++ -stdlib=libc++` although libc++ is not
installed on the machine.
* m4/ax_check_compile_flag.m4 (AX_CHECK_COMPILE_FLAG): Rewrite as...
* m4/bison-check-compile-flag.m4 (BISON_CHECK_COMPILE_FLAG): this, so
that we use AC_LINK_IFELSE to check the compiler (and its std lib)
instead of AC_COMPILE_IFELSE.
The CI is littered with
# -*- compilation -*-
423. regression.at:907: testing Dancer %glr-parser ...
./regression.at:907: bison -fno-caret -o dancer.c dancer.y
./regression.at:907: $BISON_C_WORKS
stderr:
stdout:
./regression.at:907: $CC $CFLAGS $CPPFLAGS $LDFLAGS -o dancer dancer.c $LIBS
stderr:
icc: command line warning #10006: ignoring unknown option '-Wcast-align'
icc: command line warning #10006: ignoring unknown option '-fparse-all-comments'
icc: command line warning #10006: ignoring unknown option '-Wdocumentation'
icc: command line warning #10006: ignoring unknown option '-Wnull-dereference'
icc: command line warning #10006: ignoring unknown option '-Wbad-function-cast'
icc: command line warning #10006: ignoring unknown option '-fno-color-diagnostics'
icc: command line warning #10006: ignoring unknown option '-Wno-keyword-macro'
dancer.c(755): error #1628: function declared with "noreturn" does return
}
^
dancer.c(761): error #1628: function declared with "noreturn" does return
}
^
compilation aborted for dancer.c (code 2)
ICC sees that `longjmp(buf, 1);` does not return, it sees that
`abort();` does not either, but fails to see it for
`longjmp(buf, 1); abort();`
* data/glr.c (YYLONGJMP): Be even clearer on the fact this does not
return.
Putting them in the env is useless. We don't want to pass
`CPPFLAGS="$CPPFLAGS"` to configure, as it means "set it to nothing"
when $CPPFLAGS is not set, which is not what we want.
This correctly started to use libc++, but it is not installed on the
Ubuntu. We will see later if we can use it.
* .travis.yml: Define CONFIGUREFLAGS, and pass it to configure.
* build-aux/install-icc.sh: New.
* .travis.yml (icc): New.
Use -k to get as many errors as possible from the start.
* src/complain.c (warnings_types): Use a more precise type.
On the CI, we have this spurious failure with clang 3.9 with
-std=c++17:
In file included from list.y:23:
In file included from /usr/include/c++/4.8/iostream:39:
In file included from /usr/include/c++/4.8/ostream:38:
In file included from /usr/include/c++/4.8/ios:42:
In file included from /usr/include/c++/4.8/bits/ios_base.h:41:
In file included from /usr/include/c++/4.8/bits/locale_classes.h:40:
In file included from /usr/include/c++/4.8/string:52:
In file included from /usr/include/c++/4.8/bits/basic_string.h:2815:
In file included from /usr/include/c++/4.8/ext/string_conversions.h:43:
/usr/include/c++/4.8/cstdio:120:11: error: no member named 'gets' in the global namespace
using ::gets;
~~^
This shows that our test, based on gl_WARN_ADD, is a joke. We have to
really check for at least a bit of C++.
* m4/ax_check_compile_flag.m4, m4/bison-cxx-std.m4: New.
* configure.ac: Use them to make sure the compiler actually works.
In C++ pre C++11 it is standard practice to use 0 for the null pointer.
But GCC pre 8 -std=c++98 with -Wzero-as-null-pointer-constant warns about
this.
So disable -Wzero-as-null-pointer-constant when compiling C++ pre 11.
Let's do this in AT_DATA_SOURCE_PROLOGUE (which is pasted on top of
all the test grammar files). Unfortunately, that shifts all the
locations in the expected error messages, which would be too noisy.
Instead, let's introduce testsuite.h, which can vary in length, and
include it in AT_DATA_SOURCE_PROLOGUE.
* tests/testsuite.h: New.
Disable -Wzero-as-null-pointer-constant's warning with GCC pre 8,
C++ pre 11.
* tests/local.at (AT_DATA_SOURCE_PROLOGUE): Use it.
* tests/atlocal.in (CPPFLAGS): Find it.
* tests/local.mk: Ship it.
* data/c.m4 (YY_NULLPTR): Prefer ((void*)0) to 0 in C.
For some reasons, the checkout on travis may not have any tags, so
`git describe` fails, so bootstrap fails.
* .travis.yml: If git describe fails, install some tag.
This is much of course more efficient than in the matrix of the CI (or
on our own machines), but a bit more tedious.
* configure.ac (CXX03_CXXFLAGS, CXX11_CXXFLAGS, CXX14_CXXFLAGS)
(CXX17_CXXFLAGS, CXX2A_CXXFLAGS, STDCXX_FLAGS): New.
* tests/atlocal.in: Receive them.
* tests/local.at (AT_FOR_EACH_CXX): New.
* tests/c++.at: Use AT_FOR_EACH_CXX.
Reported by Frank Heckenbach.
http://lists.gnu.org/archive/html/bug-bison/2018-03/msg00002.html
Actually the assignment operator should never be needed: the C++98
requirements for vector::push_back is CopyInsertable, which does not require
an assignment operator. However, libstdc++ shipped with GCC up to (and
including) 6 uses the assignment operator (which affects Clang on top of
libstdc++, but also ICC). So let's keep it for legacy C++.
See https://gcc.godbolt.org/z/q0XXmC.
* data/lalr1.cc (stack_symbol_type::operator=): Remove.
* data/c++.m4 (basic_symbol::operator=): Ditto.
* tests/c++.at (C++ Variant-based Symbols Unit Tests): Adjust.
Modern C++ (i.e., C++11 and later) introduced "move only" types: types such
as std::unique_ptr<T> that can never be duplicated. They must never be
copied (by assignments and constructors), they must be "moved". The
implementation of lalr1.cc used to copy symbols (including their semantic
values). This commit ensures that values are only moved in modern C++, yet
remain compatible with C++98/C++03.
Suggested by Frank Heckenbach, who provided a full implementation on
top of C++17's std::variant.
See http://lists.gnu.org/archive/html/bug-bison/2018-03/msg00002.html,
and https://lists.gnu.org/archive/html/bison-patches/2018-04/msg00002.html.
Symbols (terminal/non terminal) are handled by several functions that used
to take const-refs, which resulted eventually in a copy pushed on the stack.
With modern C++ (C++11 and later) the callers must use std::move, and the
callees must take their arguments as rvalue refs (foo&&). In order to avoid
duplicating these functions to support both legacy C++ and modern C++, let's
introduce macros (YY_MOVE, YY_RVREF, etc.) that rely on copy-semantics for
C++98/03, and move-semantics for modern C++.
That's easy for inner types, when the parser's functions pass arguments to
each other. Functions facing the user (make_NUMBER, make_STRING, etc.)
should support both rvalue-refs (for instance to support move-only types:
make_INT (std::make_unique<int> (1))), and lvalue-refs (so that we can pass
a variable: make_INT (my_int)). To avoid the multiplication of the
signatures (there is also the location), let's take the argument by value.
See:
https://lists.gnu.org/archive/html/bison-patches/2018-09/msg00024.html.
* data/c++.m4 (b4_cxx_portability): New.
(basic_symbol): In C++11, replace copy-ctors with move-ctors.
In C++11, replace copies with moves.
* data/lalr1.cc (stack_symbol_type, yypush_): Likewise.
Use YY_MOVE to avoid useless copies.
* data/variant.hh (variant): Support move-semantics.
(make_SYMBOL): In C++11, in order to support both read-only lvalues,
and rvalues, take the argument as a copy.
* data/stack.hh (yypush_): Use rvalue-refs in C++11.
* tests/c++.at: Use move semantics.
* tests/headers.at: Adjust to the new macros (YY_MOVE, etc.).
* configure.ac (CXX98_CXXFLAGS, CXX11_CXXFLAGS, CXX14_CXXFLAGS)
(CXX17_CXXFLAGS, ENABLE_CXX11): New.
* tests/atlocal.in: Receive them.
* examples/variant.yy: Don't define things in std.
* examples/variant-11.test, examples/variant-11.yy: New.
Check the support of move-only types.
* examples/README, examples/local.mk: Adjust.
This generates less code, which is nicer to read, but also takes less
chances with compilers such as G++ 4.8 that are too strict and check
"dead code" (templated code that is not instantiated).
* data/c++.m4 (b4_symbol_type_declare, b4_symbol_type_define): When
variants are used, don't generate code meant for non variants.
See ea0db44fed. We also need to disable
the warning in the examples (but don't want to clutter the
documentation with such details).
* doc/bison.texi (scanner.ll): Disable Clang's -Wdocumentation.
While at it, hide the other kludges.
That's the version on Ubuntu Precise.
See also 1dac131ec4.
* src/flex-scanner.h: Disable -Wdocumentation.
* doc/bison.texi: Turn off a warning triggered by Flex 2.6.4.
We have failures on Flex output, which are probably related to an old
release. Let's check.
In file included from src/scan-code-c.c:3:
src/scan-code.c:2198:21: error: empty paragraph passed to '@param' command
[-Werror,-Wdocumentation]
* @param line_number
~~~~~~~~~~~~~~~~~^
* .travis.yml: here.
Running all these tests might be overkill: it is very long, and don't
need full portability checks. Besides, some tests under Valgrind are
too slow and get killed by the CI (timeout of 10min without output).
* .travis.yml: here.
We have some maintainer-check-foo and some maintainer-foo-check. Keep
only the former.
* tests/local.mk (maintainer-push-check, maintainer-xml-check)
(maintainer-release-check): Rename as...
(maintainer-check-push, maintainer-check-xml)
(maintainer-check-release): these.
Most of our variables for C++ flags are named FOO_CXXFLAGS, not
CXXFLAGS_FOO.
* configure.ac, tests/atlocal.in, tests/calc.at
(NO_EXCEPTIONS_CXXFLAGS): Rename as...
(CXXFLAGS_NO_EXCEPTIONS): this.
Clang++ issues warnings when it's used to compile C. This make target
is precisely checking whether we can do that.
* configure.ac (NO_DEPRECATED_CXXFLAGS): New.
* tests/atlocal.in: Use it.
This completes 2d7e743802.
Some shells don't grok "local var=`cmd`" very well: they need the rhs
to be quoted.
./examples/test: 72: local: you.,: bad variable name
FAIL examples/variant.test (exit status: 2)
Reported by Étienne Renault.
* examples/test (run): Quote the values in 'local' assignments.
With GCC7 we have warnings (false positive):
x8.c: In function 'x8_parse':
x8.c:1233:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
yylval = *yypushed_val;
~~~~~~~^~~~~~~~~~~~~~~
x8.c: In function 'x8_pull_parse':
x8.c:1233:16: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
yylval = *yypushed_val;
~~~~~~~^~~~~~~~~~~~~~~
See also 9645a2b20e.
* tests/local.at (AT_PUSH_IF): New.
(AT_BISON_OPTION_POPDEFS): Pop it, and pop AT_PURE_IF.
* tests/headers.at (Several parsers, Several parsers): Disable these
warnings when in push parser.
Currently, in glr.cc, we emit the definitions of basic_symbol and
symbol_type, although there are not used.
* data/c++.m4 (b4_public_types_declare): Extract these definitions from
here, and move them...
(b4_symbol_type_declare): here.
(b4_public_types_declare): Also remove the definition of the symbol
constructors.
* data/lalr1.cc (b4_shared_declarations): Adjust: call
b4_symbol_type_declare and b4_symbol_constructor_declare.
Some shells don't grok `local var=$val` very well: they need the rhs
to be quoted.
./examples/test: 66: local: you.,: bad variable name
FAIL examples/variant.test (exit status: 2)
Reported by Étienne Renault.
* examples/test (run): Quote the values in 'local' assignments.
The header generated for variants with assertions but without
locations, is not self-contained. Prepare a check for this.
* tests/headers.at (Sane headers): New, extracted from...
(Several parsers): here.
Benchmarks show that it is more efficient to keep this copy
constructor, rather than forcing the use of the default constructor
and then assignment.
This reverts commit 7ab25ad020.
Currently, in bison's C++ parser template (`lalr.cc`), the `variant<>`
struct's `build()` method uses placement-new in the form `new (...) T`
to initialize a variant type. However, for POD variant types, this
will leave the memory space uninitialized. If we subsequently tries
to `::move` into a variant object in such state, the call can trigger
clang's undefined behavior sanitizer due to accessing the
uninitialized memory.
https://lists.gnu.org/archive/html/bison-patches/2018-08/msg00098.html
* data/variant.hh (build): Always initialize the stored value.
Signed-off-by: Akim Demaille <[email protected]>
* doc/bison.texi (A Complete C++ Example): Prefer throw exceptions
from the scanner.
Show the invalid characters.
Since the scanner sends exceptions, it no longer needs to report
errors, so we can get rid of the driver's routine to report error,
do it in yyerror.
Use @group/@end group to improve rendering.
This example uses the calcxx_ prefix for each class. That's uselessly
heavy.
* doc/bison.texi (A Complete C++ Example): Simplify the class names.
Since now 'driver' denotes the class, use 'drv' for the values.
Formatting changes.
On these tests, at -O2 and above, GCC 8 complains that yylval may be
uninitialized. But it seems wrong: it is initialized. Rather than
turning off the warning in the skeleton (hence possibility hiding
relevant warnings of user parsers), let's turn it off in the tests
only.
163: parse.error=verbose and consistent errors: FAILED (conflicts.at:625)
165: parse.error=verbose and consistent errors: lr.default-reduction=consistent FAILED (conflicts.at:635)
166: parse.error=verbose and consistent errors: lr.default-reduction=accepting FAILED (conflicts.at:641)
167: parse.error=verbose and consistent errors: lr.type=canonical-lr FAILED (conflicts.at:645)
168: parse.error=verbose and consistent errors: parse.lac=full FAILED (conflicts.at:650)
169: parse.error=verbose and consistent errors: parse.lac=full lr.default-reduction=accepting FAILED (conflicts.at:655)
We get:
input.c: In function 'yyparse':
input.c:980:9: error: 'yylval' may be used uninitialized in this function [-Werror=maybe-uninitialized]
YYSTYPE yylval YY_INITIAL_VALUE (= yyval_default);
^~~~~~
cc1: all warnings being treated as errors
See https://lists.gnu.org/archive/html/bison-patches/2018-08/msg00063.html.
* tests/conflicts.at (AT_CONSISTENT_ERRORS_CHECK): Disable
-Wmaybe-uninitialized.
Suggested by Rici Lake.
https://lists.gnu.org/archive/html/bug-bison/2018-08/msg00033.html
* doc/bison.texi: Complete description of the first node in the main
@menu.
(Push Decl): Remove the 'experimental' warnings about push parser.
Clarify that the push parser object can be reused in several parses.
Reported by Brooks Moses <[email protected]>
http://lists.gnu.org/archive/html/bison-patches/2018-02/msg00000.html
* data/lalr1.cc (YY_EXCEPTIONS): New.
Use it to disable try/catch clauses.
* doc/bison.texi (C++ Parser Interface): Document it.
* configure.ac (CXXFLAGS_NO_EXCEPTIONS): New.
* tests/atlocal.in: Receive it.
* tests/local.at (AT_FULL_COMPILE, AT_LANG_COMPILE):
Accept a new argument, extra compiler flags.
* tests/calc.at: Run the C++ calculator with exception support disabled.
Currently the examples are too dense, let's put empty lines where
'#line' would be issued. And also remove some spurious empty
lines (remains from @group, @end group, etc.).
* examples/extexi: Do that.
* examples/local.mk (extexiFLAGS): Rename as...
(EXTEXIFLAGS): this.
* examples/mfcalc/local.mk, examples/rpcalc/local.mk: Define the
programs in a more natural order, source, preproc, then linker.
* examples/test: Be ready to work on programs that are not in
a subdir.
* examples/variant.test: New.
* examples/local.mk: Use it.
* examples/variant.yy: Don't use 0 for nullptr.
Use a more natural output for a list of string.
Visual Studio issues a C4146 warning on '-static_cast<unsigned>(rhs)'.
The code is weird, probably to cope with INT_MIN. Let's go back to
using std::max (whose header is still included in position.hh...) like
originally, but with the needed casts.
Reported by 長田偉伸, and with help from Rici Lake.
See also
http://lists.gnu.org/archive/html/bug-bison/2013-02/msg00000.html
and commit 75ae829984.
* data/location.cc (position::add_): Take min as an int.
Use std::max.
While here, get rid of a couple of useless inlines.
* doc/bison.texi (Mid-Rule Actions): Restructure to insert...
(Typed Mid-Rule Actions): this new section.
Move the manual translation of mid-rule actions into regular actions
to...
(Mid-Rule Action Translation): here.
Reported by Jannick.
http://lists.gnu.org/archive/html/bug-bison/2017-05/msg00001.html
"Amusingly" enough, we have the same problem with %defines when the
parser file name has backslashes or quotes: we generate #includes with
an incorrect C string.
* src/output.c (prepare_symbol_definitions): Escape properly the file
names before passing them to M4.
* data/bison.m4, data/lalr1.cc: Don't simply put the file name between
two quotes (that should have been strong enough a smell...), expect
the string to be properly quoted.
* tests/synclines.at: New tests to check this.
Suggested by Paul Eggert.
* src/reader.c (find_start_symbol): Don't check 'res', we know it is
not null. That suffices to avoid the GCC warnings.
* bootstrap.conf: We don't need 'assume', which doesn't exist anyway.
In 0931d14728 I removed too many
initializations from some ctors: some were not about base ctors, but
about member variables. In fact, more of them were missing to please
GCC 8.
While at it, generate more natural code for C++ without variant:
instead of
template <typename Base>
parser::basic_symbol<Base>::basic_symbol (const basic_symbol& other)
: Base (other)
, value ()
{
value = other.value
}
generate
template <typename Base>
parser::basic_symbol<Base>::basic_symbol (const basic_symbol& other)
: Base (other)
, value (other.value)
{}
* data/c++.m4 (basic_symbol::basic_symbol): Always initialize 'value',
it might be a POD without a ctor.
* data/lalr1.cc (stack_symbol_type::stack_symbol_type): Likewise.
* data/variant.hh (variant::variant): Default initialize the buffer too.
GCC 8 issues warnings whose root cause was a bit hard to find.
calc.cc: In member function 'virtual int yy::parser::parse()':
calc.cc:810:18: warning: '*((void*)&<anonymous> +8)' may be used uninitialized in this function [-Wmaybe-uninitialized]
, location (l)
^
calc.cc: In member function 'void yy::parser::yypush_(const char*, yy::parser::stack_symbol_type&)':
calc.cc:810:18: warning: '*((void*)&<anonymous> +8)' may be used uninitialized in this function [-Wmaybe-uninitialized]
, location (l)
^
calc.cc: In member function 'void yy::parser::yypush_(const char*, yy::parser::state_type, yy::parser::symbol_type&)':
calc.cc:810:18: warning: '*((void*)&<anonymous> +8)' may be used uninitialized in this function [-Wmaybe-uninitialized]
, location (l)
^
The problem is with locations that don't have a constructor, such as
Span (in calc.cc) which is POD. It is POD on purpose: so that we can
use that structure to test glr.cc which cannot use non POD in its
(C) stacks.
* data/c++.m4 (basic_symbol): Also ensure that 'location' is
initialized.
With GCC 7.3.0 and Flex 2.6.4, we get warnings on all the generated
scanners:
examples/calc++/calc++-scanner.cc: In function 'void yyrestart(FILE*)':
examples/calc++/calc++-scanner.cc:1611:20: error: potential null pointer dereference [-Werror=null-dereference]
/* %endif */
~~~~~~~~~~~ ^
examples/calc++/calc++-scanner.cc:1607:19: error: potential null pointer dereference [-Werror=null-dereference]
/* %if-c-only */
~~~~~~~~~~~~~~~ ^
examples/calc++/calc++-scanner.cc:1611:20: error: potential null pointer dereference [-Werror=null-dereference]
/* %endif */
~~~~~~~~~~~ ^
examples/calc++/calc++-scanner.cc:1607:19: error: potential null pointer dereference [-Werror=null-dereference]
/* %if-c-only */
~~~~~~~~~~~~~~~ ^
cc1plus: all warnings being treated as errors
Obviously the lines are incorrect, and the warnings are emitted twice.
Still, let's get rid of these warnings.
* doc/bison.texi, src/flex-scanner.h: Disable these warnings in code
generated by Flex.
Commit 3df32101e7 introduced invalid C
code. Caught by GCC 7.3.0.
* bootstrap.conf (gnulib_modules): We need assume.
* src/reader.c (find_start_symbol): Fix the signature (too much C++,
sorry...).
Prefer 'assume' to 'assert', so that we don't have these warnings even
when NDEBUG is defined.
This file was meant to be shown as an example. Install it.
* README, data/README: Put Emacs metadata in the final section.
* examples/README: New.
* examples/variant.yy: Use %empty.
* examples/local.mk: Install both these files.
We currently generate copy constructors such as the following
one (taken from examples/variant.yy):
parser::stack_symbol_type::stack_symbol_type (const stack_symbol_type& that)
: super_type (that.state, that.location)
{
switch (that.type_get ())
{
case 3: // TEXT
case 8: // item
value.copy< ::std::string > (that.value);
break;
case 7: // list
value.copy< ::std::vector<std::string> > (that.value);
break;
case 4: // NUMBER
value.copy< int > (that.value);
break;
default:
break;
}
}
they are actually useless: we never need it.
* data/lalr1.cc: Don't generate the stack_symbol_type copy ctor.
Fix a typo so that instead of
basic_symbol::basic_symbol (typename Base::kind_type t, const int v)
we now generate
basic_symbol::basic_symbol (typename Base::kind_type t, const int& v)
* data/variant.hh (b4_basic_symbol_constructor_declare)
(b4_basic_symbol_constructor_define): Add missing reference.
Several features were flagged 'experimental' and waiting for user
feedback to 'stabilize', but i. AFAIK, no user ever reported anything
about them, ii. they'be been here long enough to prove they don't do
harm.
* doc/bison.texi: No longer experimental: default %printer and
%destructor (typed: <*> and untyped: <>), %define api.value.type union
and variant, Java parsers, XML output, LR family (lr, ielr, lalr),
semantic predicates (%?).
* tests/actions.at: Fix some log messages.
Prefer #error to fprintf: it fixes the invalid use of yyoutput in
%destructor, and it is an even stronger check: that the code is not
even emitted. The portability of #error is not really a problem here,
since the point is anyway to have the compilation fail.
Make sure that we cannot apply a type to the (main) action of a rule.
* src/reader.c (grammar_rule_check): Issue the warning.
* tests/input.at (Cannot type action): Check the warning.
* tests/input.at (_AT_UNUSED_VALUES_DECLARATIONS): Check
typed mid-rule actions.
* tests/report.at (Reports): Check that types of typed mid-rule
actions are reported.
* tests/actions.at (Typed mid-rule actions): Check that
the values of typed mid-rule actions are correct.
Prompted on Piotr Marcińczyk's message:
http://lists.gnu.org/archive/html/bug-bison/2017-06/msg00000.html.
See also http://lists.gnu.org/archive/html/bug-bison/2018-06/msg00001.html.
Because their type is unknown to Bison, the values of midrule actions are
not treated like the others: they don't have %printer and %destructor
support. In addition, in C++, (Bison) variants cannot work properly.
Typed midrule actions address these issues. Instead of:
exp: { $<ival>$ = 1; } { $<ival>$ = 2; } { $$ = $<ival>1 + $<ival>2; }
write:
exp: <ival>{ $$ = 1; } <ival>{ $$ = 2; } { $$ = $1 + $2; }
* src/scan-code.h, src/scan-code.l (code_props): Add a `type` field to
record the declared type of an action.
(code_props_rule_action_init): Add a type argument.
* src/parse-gram.y: Accept an optional type tag for actions.
* src/reader.h, src/reader.c (grammar_current_rule_action_append): Add
a type argument.
(grammar_midrule_action): When a mid-rule is typed, pass its type to
the defined dummy non terminal symbol.
grammar_current_rule_action_append was used in two different places:
for actual action (`{...}`), and for predicates (`%?{...}`). Let's
split this in two different functions.
* src/reader.h, src/reader.c (grammar_current_rule_predicate_append): New.
Extracted from...
(grammar_current_rule_action_append): here.
Remove arguments that don't apply.
Adjust dependencies.
Predicates with GLR are issued with synclines in the middle of C code:
case 2:
if (! (#line 6 "sempred.y" /* glr.c:816 */
new_syntax)) YYERROR;
#line 793 "sempred.tab.c" /* glr.c:816 */
break;
Reported by Rici Lake.
http://lists.gnu.org/archive/html/bug-bison/2018-05/msg00033.html
* data/c.m4 (b4_predicate_case): Be sure to start on column 0.
It would be nicer if b4_syncline could ensure this by itself
(that would avoid ugly code when synclines are disabled), but that's
way more work.
* tests/glr-regression.at (Predicates): Be a real end-to-end test.
This would have caught this error years ago...
For instance on test 99:
In file included from @@.cc:56:
@@.hh:409:26: error: definition of implicit copy constructor for
'stack_symbol_type' is deprecated because it
has a user-declared copy assignment operator
[-Werror,-Wdeprecated]
stack_symbol_type& operator= (const stack_symbol_type& that);
^
Reported by Derek Clegg.
https://lists.gnu.org/archive/html/bison-patches/2018-05/msg00036.html
* configure.ac (warn_tests): Add -Wdeprecated.
* data/lalr1.cc (stack_symbol_type): Add an explicit copy ctor.
We cannot rely on the explicit default implementation (`= default`)
as we support C++ 98.
With recent compilers:
input.yy:49:5: error: definition of implicit copy assignment
operator for 'Object' is deprecated because
it has a user-declared destructor
[-Werror,-Wdeprecated]
~Object ()
^
input.yy:130:35: note: in implicit copy assignment operator for
'Object' first required here
{ yylhs.value.as< Object > () = yystack_[0].value.as< Object > (); }
* tests/c++.at (Object): Add missing assignment operator.
sc_two_space_separator_in_usage complains about bootstrap:
two_space_separator_in_usage
/Users/akim/src/gnu/bison/bootstrap:905: --aux-dir $build_aux\
/Users/akim/src/gnu/bison/bootstrap:906: --doc-base $doc_base\
/Users/akim/src/gnu/bison/bootstrap:907: --lib $gnulib_name\
/Users/akim/src/gnu/bison/bootstrap:908: --m4-base $m4_base/\
/Users/akim/src/gnu/bison/bootstrap:909: --source-base $source_base/\
/Users/akim/src/gnu/bison/bootstrap:910: --tests-base $tests_base\
/Users/akim/src/gnu/bison/bootstrap:911: --local-dir $local_gl_dir\
maint.mk: help2man requires at least two spaces between an option and its description
* cfg.mk: Exclude bootstrap from this check.
At least GCC 7.3, with -O1 or -O2 (but not -O0 or -O3) generates
warnings with -Wnull-dereference when using yyformat: it fails to see
yyformat cannot be null.
Reported by Frank Heckenbach, https://savannah.gnu.org/patch/?9620.
* configure.ac: Use -Wnull-dereference if supported.
* data/glr.c, data/lalr1.cc, data/yacc.c: Define yyformat in such
a way that GCC cannot not see that yyformat is defined.
Using `default: abort();` also addresses the issue, but forces
the inclusion of `stdlib.h`, which we avoid.
Sometimes `inline` would be used in *.cc files on symbols that are not
exported (useless but harmless), and sometimes on exported symbols
such as the constructor of syntax_error (harmful: linking fails).
Reported several times, including:
- by Dennis T
http://lists.gnu.org/archive/html/bug-bison/2016-03/msg00002.html
- by Frank Heckenbach
https://savannah.gnu.org/patch/?9616
* data/c++.m4 (b4_inline): New: expands to `inline` or nothing.
Use it where appropriate.
* data/lalr1.cc: Use it where appropriate.
* tests/c++.at (Syntax error as exception): Put the scanner in another
compilation unit to exercise the constructor of syntax_error.
* data/lalr1.cc, doc/bison.texi, etc/bench.pl.in, examples/variant.yy,
* tests/actions.at, tests/atlocal.in, tests/c++.at, tests/headers.at,
* tests/local.at, tests/types.at:
Don't use std::endl, it flushes uselessly, and is considered bad
style.
* README-hacking: Commit before bootstrapping.
* bootstrap.conf: gnulib_mk is no longer defined by bootstrap.
* bootstrap, gnulib, lib/.gitignore, m4/.gitignore: Update/regen.
With Clang 6.0:
CC src/bison-getargs.o
bison/src/getargs.c:67:12: error: parameter 'option' not found in the
function declaration [-Werror,-Wdocumentation]
* \param option option being decoded.
^~~~~~
bison/src/getargs.c:67:12: note: did you mean 'rpl_option'?
* src/getargs.c: Don't use `option` as a documentation argument.
Currently lalr1.cc makes an out-of-bound access when trying to read @1
in rules with an empty rhs (i.e., when there is no @1) that raises an
error (YYERROR).
glr.c already gracefully handles this by using @$ as initial location
for the errors. Let's do that in yacc.c and lalr1.cc.
* data/lalr1.cc, data/yacc.c: Use @$ to initialize the error location.
* tests/actions.at: Check that case.
This style appears to be more traditional, at least in C++.
For instance in the standard, [facets.examples].
There are occurrences using "unsigned int" too though.
* data/lalr1.cc, data/location.cc, data/stack.hh: here.
We use this trick to write some test about internal details. But
since we use -Werror, clang++ 3.6 dies issueing a warning about it.
* configure.ac (warn_tests): Disable this warning.
Because musl supports UTF-8 with LC_ALL=C, gcc produces:
input.y: In function ‘yyparse’:
instead of:
input.y: In function 'yyparse':
Reported by Ferdinand Thiessen.
http://lists.gnu.org/archive/html/bug-bison/2015-02/msg00001.html
* tests/synclines.at (AT_SYNCLINES_COMPILE): Skip syncline tests when
we can't trust error messages issued about a function body.
Sun C 5.13 SunOS_sparc 2014/10/20 reports errors on tests 430-432.
Reported by Dennis Clarke.
<http://lists.gnu.org/archive/html/bug-bison/2015-01/msg00087.html>
* tests/c++.at (Variants): Be sure to emit operator<< before using it:
use "%code top" rather than "%code".
Prefer std::vector to std::list.
Do not define anything in std::, to avoid undefined behavior.
* origin/maint:
maint: post-release administrivia
version 3.0.4
gnulib: update
build: re-enable compiler warnings, and fix them
tests: c++: fix a C++03 conformance issue
tests: fix a title
c++: reserve 200 slots in the parser's stack
tests: be more robust to unrecognized synclines, and try to recognize xlc
tests: fix C++ conformance
build: fix some warnings
build: avoid infinite recursions on include_next
There are warnings (-Wextra) in generated C++ code:
ltlparse.cc: In member function 'ltlyy::parser::symbol_number_type
ltlyy::parser::by_state::type_get() const':
ltlparse.cc:452:33: warning: enumeral and non-enumeral type in
conditional expression
return state == empty_state ? empty_symbol : yystos_[state];
Reported by Alexandre Duret-Lutz.
It turns out that -Wall and -Wextra were disabled because of a stupid
typo.
* configure.ac: Fix the stupid typo.
* data/lalr1.cc, src/AnnotationList.c, src/InadequacyList.c,
* src/ielr.c, src/print.c, src/scan-code.l, src/symlist.c,
* src/symlist.h, src/symtab.c, src/tables.c, tests/actions.at,
* tests/calc.at, tests/cxx-type.at, tests/glr-regression.at,
* tests/named-refs.at, tests/torture.at:
Fix warnings, mostly issues about variables used only with assertions,
which are disabled with -DNDEBUG.
This fixes test 241 on xLC:
"input.y", line 42.11: 1540-0274 (S) The name lookup for "report" did not find a declaration.
"input.y", line 42.11: 1540-1292 (I) Static declarations are not considered for a function call if the function is not qualified.
where report is:
static void
report (std::ostream& yyo, int ival, float fval)
{
yyo << "ival: " << ival << ", fval: " << fval;
}
and line 42 is:
%printer { report (yyo, $$, $<fval>$); } <ival>;
It turns out that indeed this function must not be declared static,
<http://stackoverflow.com/a/17662745/1353549>. Let's put it into an
anonymous namespace.
Reported by Thomas Jahns.
http://lists.gnu.org/archive/html/bug-bison/2015-01/msg00059.html
* tests/actions.at (Qualified $$ in actions): Don't use "static",
prefer anonymous namespace.
This is consistent with what is done with yacc.c and glr.c. Because
it also avoids that the stack needs to be resized very soon, it should
help keeping tests about destructors more reliable.
Indeed, if the stack is created too small, very soon the C++ library
needs to enlarge it, which means creating a new one, copying the
elements from the initial one onto it, and then destroy the elements
of the initial stack: that would be a spurious call to a destructor.
Reported by Thomas Jahns.
http://lists.gnu.org/archive/html/bug-bison/2015-01/msg00059.html
* data/stack.hh (stack::stack): Reserve 200 slots.
* tests/c++.at: Remove traces of stack expansions.
Reported by Thomas Jahns.
http://lists.gnu.org/archive/html/bug-bison/2015-01/msg00059.html
* tests/synclines.at (AT_SYNCLINES_COMPILE): Rename as...
(_AT_SYNCLINES_COMPILE): this.
Try to recognize xlc locations.
(AT_SYNCLINES_COMPILE): New. Skips the test if we can't read the
synclines.
* origin/maint:
doc: minor fixes
gnulib: strtoul is considered obsolete and now useless
c++: avoid warnings when destructors don't use $$
maint: post-release administrivia
version 3.0.3
gnulib: update
On MacOS X 10.5 PPC with Apple's GCC 4.0.1:
% uname -a
Darwin aria.cielonegro.org 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:57:0
1 PDT 2009; root:xnu-1228.15.4~1/RELEASE_PPC Power Macintosh
% gcc --version
powerpc-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5493)
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
building in place enters into an infinite recursion on "#include_next":
% gmake V=1
[snip]
depbase=`echo lib/math.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
gcc -std=gnu99 -I. -Ilib -I. -I./lib -g -O2 -MT lib/math.o -MD -MP -MF $depbase.Tpo -c -o lib/math.o lib/math.c &&\
mv -f $depbase.Tpo $depbase.Po
In file included from lib/math.h:27,
from lib/math.h:27,
from lib/math.h:27,
from lib/math.h:27,
[snip]
from lib/math.h:27,
from lib/math.h:27,
from lib/math.c:3:
lib/math.h:27:23: error: #include nested too deeply
Makefile:3414: recipe for target 'lib/math.o' failed
gmake[2]: *** [lib/math.o] Error 1
Using -I./lib instead of -Ilib fixes the problem.
Reported by Pho.
<https://lists.gnu.org/archive/html/bison-patches/2014-01/msg00000.html>
* Makefile.am (AM_CPPFLAGS): Use -I./lib instead of -Ilib.
Currently on the following grammar:
%type <foo> foo
%%
start: foo | bar | "baz"
foo: foo
bar: bar
bison reports:
warning: 2 nonterminals useless in grammar [-Wother]
warning: 4 rules useless in grammar [-Wother]
1.13-15: warning: nonterminal useless in grammar: foo [-Wother]
%type <foo> foo
^^^
3.14-16: warning: nonterminal useless in grammar: bar [-Wother]
start: foo | bar | "baz"
^^^
[...]
i.e., the location of the first occurrence of a symbol is taken as its
definition point. In the case of nonterminals, the first occurrence
as a left-hand side of a rule makes more sense:
warning: 2 nonterminals useless in grammar [-Wother]
warning: 4 rules useless in grammar [-Wother]
4.1-3: warning: nonterminal useless in grammar: foo [-Wother]
foo: foo
^^^
5.1-3: warning: nonterminal useless in grammar: bar [-Wother]
bar: bar
^^^
[...]
* src/symtab.h, src/symtab.c (symbol::location_of_lhs): New.
(symbol_location_as_lhs_set): New.
* src/parse-gram.y (current_lhs): Use it.
* tests/reduce.at: Update locations.
In the following grammar, the 'exp' nonterminal is trivially useless.
So, of course, its rules are useless too.
%%
input: '0' | exp
exp: exp '+' exp | exp '-' exp | '(' exp ')'
Previously all the useless rules were reported, including those whose
left-hand side is the 'exp' nonterminal:
warning: 1 nonterminal useless in grammar [-Wother]
warning: 4 rules useless in grammar [-Wother]
2.14-16: warning: nonterminal useless in grammar: exp [-Wother]
input: '0' | exp
^^^
2.14-16: warning: rule useless in grammar [-Wother]
input: '0' | exp
^^^
! 3.6-16: warning: rule useless in grammar [-Wother]
! exp: exp '+' exp | exp '-' exp | '(' exp ')'
! ^^^^^^^^^^^
! 3.20-30: warning: rule useless in grammar [-Wother]
! exp: exp '+' exp | exp '-' exp | '(' exp ')'
! ^^^^^^^^^^^
! 3.34-44: warning: rule useless in grammar [-Wother]
! exp: exp '+' exp | exp '-' exp | '(' exp ')'
! ^^^^^^^^^^^
The interest of being so verbose is dubious. I suspect most of the
time nonterminals are not expected to be useless, so the user wants to
fix the nonterminal, not remove its rules. And even if the user
wanted to get rid of its rules, the position of these rules probably
does not help more that just having the name of the nonterminal.
This commit discard these messages, marked with '!', and keep the
others. In particular, we still report:
2.14-16: warning: rule useless in grammar [-Wother]
input: '0' | exp
^^^
All the useless rules (including the '!' ones) are still reported in
the reports (xml, text, etc.); only the diagnostics on stderr change.
* src/gram.c (grammar_rules_useless_report): Don't complain about
useless rules whose lhs is useless.
* src/reduce.h, src/reduce.c (reduce_nonterminal_useless_in_grammar):
Take a sym_content as argument.
Adjust callers.
* tests/reduce.at (Useless Rules, Underivable Rules, Reduced Automaton):
Adjust.
* origin/maint:
tests: split a large test case into several smaller ones
package: a bit of trouble shooting indications
doc: liby's main arms the internationalization
bison: avoid warnings from static code analysis
c++: fix the use of destructors when variants are enabled
style: tests: simplify the handling of some C++ tests
c++: symbols can be empty, so use it
c++: variants: don't leak the lookahead in error recovery
c++: provide a means to clear symbols
c++: clean up the handling of empty symbols
c++: comment and style changes
c++: variants: comparing addresses of typeid.name() is undefined
c++: locations: complete the API and fix comments
build: do not clean figure sources in make clean
* tests/conflicts.at (AT_CONSISTENT_ERRORS_CHECK): Move AT_SETUP/AT_CLEANUP
into it, so that we don't skip non Java tests following a test case in Java.
A static analysis tool reports that some callers of symbol_list_n_get
might get NULL and not handle it properly. This is not the case, yet
we can suppress this pattern.
Reported by Mike Sullivan.
<https://lists.gnu.org/archive/html/bug-bison/2013-12/msg00027.html>
* src/symlist.c (symbol_list_n_get): Actually it is never called
to return 0. Enforce this postcondition via aver.
(symbol_list_n_type_name_get): Simplify accordingly. In particular,
discards a (translated) useless error message.
* src/symlist.h: Adjust documentation.
* src/scan-code.l: Style change.
When using variants, destructors generate invalid code.
<http://lists.gnu.org/archive/html/bug-bison/2014-09/msg00005.html>
Reported by Michael Catanzaro.
* data/c++.m4 (~basic_symbol): b4_symbol_foreach works on yysym:
define it.
* tests/c++.at (Variants): Check it.
The previous patches ensure that symbols (symbol_type and
stack_symbol_type) can be empty, cleared, and their emptiness can be
checked. Therefore, yyempty, which codes whether yyla is empty or
not, is now useless.
In C skeletons (e.g., yacc.c), the fact that the lookahead is empty is
coded by "yychar = YYEMPTY", which is exactly what this patch
restores, since yychar/yytoken corresponds to yyla.type.
* data/lalr1.cc (yyempty): Remove.
Rather, depend on yyla.empty ().
During error recovery, when discarding the lookeahead, we don't
destroy it, which is caught by parse.assert assertions.
Reported by Antonio Silva Correia.
With an analysis and suggested patch from Michel d'Hooge.
<http://savannah.gnu.org/support/?108481>
* tests/c++.at (Variants): Strengthen the test to try syntax errors
with discarded lookahead.
The symbol destructor is currently the only means to clear a symbol.
Unfortunately during error recovery we might have to clear the
lookahead, which is a local variable (yyla) that has not yet reached
its end of scope.
Rather that duplicating the code to destroy a symbol, or rather than
destroying and recreating yyla, let's provide a means to clear a
symbol.
Reported by Antonio Silva Correia, with an analysis from Michel d'Hooge.
<http://savannah.gnu.org/support/?108481>
* data/c++.m4, data/lalr1.cc (basis_symbol::clear, by_state::clear)
(by_type::clear): New.
(basic_symbol::~basic_symbol): Use clear.
Instead of storing and comparing pointers to names of types, store
pointers to the typeids, and compares the typeids.
Reported by Thomas Jahns.
<http://lists.gnu.org/archive/html/bug-bison/2014-03/msg00001.html>
* data/variant.hh (yytname_): Replace with...
(yytypeid_): this.
There are no support for += between locations, and some comments are wrong.
Reported by Alexandre Duret-Lutz.
* data/location.cc: Fix.
* doc/bison.texi: Document.
* tests/c++.at: Check.
* origin/maint:
build: don't try to generate docs when cross-compiling
package: fix a reporter's name
%union: fix the support for named %union
package: bump to 2015
flex: don't trust YY_USER_INIT
yacc.c: fix broken union when api.value.type=union and %defines are used
doc: fix missing xref
gnulib: update
location: remove some ugly debugging code traces
build: use abort to pacify compiler errors
package: bump to 2014
doc: specify documentation encoding
Bison supports a union tag, for obscure reasons. But it does a poor
job at it, especially since Bison 3.0.
Reported by Stephen Cameron and Tobias Frost.
It did not ensure that the name was not given several times. An easy
way to do this is to make the %union tag be handled as a %define
variable, as they cannot be defined several times.
Since Bison 3.0, the synclines were wrongly placed, resulting in
invalid code. Addressing this issue, because of the way the union tag
was stored (as a code muscle), would have been tedious. Unless we
rather define the %union tag as a %percent variable, whose synclines
are easier to manipulate.
So replace the b4_union_name muscle by the api.value.union.name
%define variable, document, and check.
* data/bison.m4: Make sure that api.value.union.name has a keyword value.
* data/c++.m4: Make sure that api.value.union.name is not defined.
* data/c.m4 (b4_union_name): No longer use it, use api.value.union.name.
* doc/bison.texi (%define Summary): Document it.
* src/parse-gram.y (union_name): No longer define b4_uion_name, but
api.value.union.name.
* tests/input.at (Redefined %union name): New.
* tests/synclines.at (%union name syncline): New.
* tests/types.at: Check named %unions.
Reported by Rich Wilson.
* data/c.m4 (b4_symbol_type_register): Append to b4_union_members,
not b4_user_union_members.
The latter invokes the former, but it is the former which is reinitialized
to empty by b4_value_type_setup_union.
* tests/types.at: Check it.
This reveals another bug, this time in the case of glr.c parsers.
* data/glr.c: Generate the header file before the implementation file,
to be sure that the setup is run before what depends on it.
clang, with -DNDEBUG and -Werror fails on some functions that might
lack a return. This is because aver is just another assert, discarded
with -DNDEBUG. So use abort.
* src/muscle-tab.c, src/scan-skel.l: here.
* doc/bison.texi: Add '@documentencoding UTF-8'; needed since the
manual contains UTF-8 characters. This will cause the .info files
to contain UTF-8 quotes and the like, which should be OK nowadays.
Add @documentlanguage while we're at it.
This completes and fixes a728075710.
Reported by Valentin Tolmer.
Before it Bison used to put the properties of the symbols
(associativity, printer, etc.) in the 'symbol' structure. An
identifier-named token (FOO) and its string-named alias ("foo")
duplicated these properties, and symbol_check_alias_consistency()
checked that both had compatible properties and fused them, at the end
of the parsing of the grammar.
The commit a728075710 introduces a
sym_content structure that keeps all these properties, and ensures
that both aliases point to the same sym_content (instead of
duplicating). However, it removed symbol_check_alias_consistency,
which resulted in the non-fusion of *existing* properties:
%token FOO "foo"
%left FOO %left "foo"
was properly diagnosed as a redeclaration, but
%left FOO %left "foo"
%token FOO "foo"
was not, as the properties of FOO and "foo" were not checked before
fusion. It certainly also means that
%left "foo"
%token FOO "foo"
did not transfer properly the associativity to FOO.
The fix is simple: reintroduce symbol_check_alias_consistency (under a
better name, symbol_merge_properties) and call it where appropriate.
Also, that commit made USER_NUMBER_HAS_STRING_ALIAS useless, but left
it.
* src/symtab.h (USER_NUMBER_HAS_STRING_ALIAS): Remove, unused.
Adjust dependencies.
* src/symtab.c (symbol_merge_properties): New, based on the former
symbol_check_alias_consistency.
* tests/input.at: Re-enable tests that we now pass.
* origin/maint:
package: install the examples
package: install README and the like in docdir
diagnostics: fix the order of multiple declarations reports
symbol: provide an easy means to compare them in source order
Conflicts:
src/symtab.c
tests/input.at
* tests/input.at: Comment out a test that master currently does not
pass (because of a728075710).
Currently, we do not install the various examples extracted from the
documentation. Let's do it, as they are useful starting points.
* configure.ac: When --enable-gcc-warnings is set, enable ENABLE_GCC_WARNINGS.
* examples/extexi: No longer issue synclines by default.
* examples/local.mk: Except if ENABLE_GCC_WARNINGS.
* examples/calc++/local.mk, examples/mfcalc/local.mk,
* examples/rpcalc/local.mk: Install the example files.
* origin/maint: (43 commits)
maint: post-release administrivia
version 3.0.2
gnulib: update
output: do not generate source files when late errors are caught
output: record what generated files are source or report files
output: do not generate source files when early errors are caught
xml: also use "%empty" with html output
style: formatting changes
xml: also display %empty for empty right-hand sides
reports: display %empty in the generated pointed-rules
news: YYERROR vs variants
style: scope reduction in lalr.cc
lalr1.cc: formatting changes
lalr1.cc: fix the support of YYERROR with variants
tests: check $$'s destruction with variant, YYERROR, and no error recovery
tests: simplify useless obfuscation
skeletons: use better names when computing a "goto"
maint: post-release administrivia
version 3.0.1
aver: it is no longer "protected against NDEBUG"
...
Conflicts:
data/glr.c
Reported by Alexandre Duret-Lutz as "second problem" in:
http://lists.gnu.org/archive/html/bug-bison/2013-09/msg00015.html
* bootstrap.conf: We need the "unlink" module.
* src/files.h, src/files.c (unlink_generated_sources): New.
* src/output.c: Use it.
* tests/output.at: Check the case of late errors.
* src/files.h, src/files.c (output_file_name_check): Take an additional
argument to record whether a file is a source or report file.
* src/files.c (generated_file): New.
(file_names, file_names_count): Replace with...
(generated_files, generated_files_size): these.
* src/scan-skel.l: Adjust.
Reported by Alexandre Duret-Lutz as "second problem" in:
http://lists.gnu.org/archive/html/bug-bison/2013-09/msg00015.html
One problem is that some errors are caught early, before the
generation of output files, while others can only be detected
afterwards (since, for instance, skeletons can raise errors
themselves).
This will be addressed in two steps: early errors do not generate
source files at all, while later errors will remove the files that
have already been generated.
* src/scan-skel.l (yyout): Open to /dev/null when there are errors.
* tests/output.at (AT_CHECK_FILES): Factored out of...
(AT_CHECK_OUTPUT): this.
Fuse the "SHELLIO" argument in the "FLAGS" one.
Use $5 to denote the expected exit status.
Add a test case for early errors.
When variant are enabled, the yylhs variable (the left-hand side of
the rule being reduced, i.e. $$ and @$) is explicitly destroyed when
YYERROR is called. This is because before running the user code, $$
is initialized, so that the user can properly use it.
However, when quitting yyparse, yylhs is also reclaimed by the C++
compiler: the variable goes out of scope.
Instead of trying to be too smart, let the compiler do its job: reduce
the scope of yylhs to exactly the reduction. This way, whatever the
type of scope exit (regular, exception, return, goto...) this variable
will be properly reclaimed.
Reported by Paolo Simone Gasparello.
<http://lists.gnu.org/archive/html/bug-bison/2013-10/msg00003.html>
* data/lalr1.cc (yyparse): Reduce the scope of yylhs.
* tests/c++.at: We now pass this test.
When variant are enabled, the yylhs variable (the left-hand side of
the rule being reduced, i.e. $$ and @$) is explicitly destroyed when
YYERROR is called. This is because before running the user code, $$
is initialized, so that the user can properly use it.
However, when quitting yyparse, yylhs is also reclaimed by the C++
compiler: the variable goes out of scope.
This was not detected by the test suite because (i) the Object tracker
was too weak, and (ii) the problem does not show when there is error
recovery.
Reported by Paolo Simone Gasparello.
<http://lists.gnu.org/archive/html/bug-bison/2013-10/msg00003.html>
* tests/c++.at (Exception safety): Improve the objects logger to make
sure that we never destroy twice an object.
Also track copy-constructors.
Use a set instead of a list.
Display the logs before running the function body, this is more
useful in case of failure.
Generalize to track with and without error recovery.
* data/glr.c (yyLRgotoState): Name the symbol argument yysym, instead
of yylhs.
* data/lalr1.cc (yy_lr_goto_state_): Likewise.
* data/lalr1.java (yy_lr_goto_state_): New, modeled after the previous
two routines.
Use it.
Apply the same rules for aver as for assert: no side effects,
especially not important ones.
* src/AnnotationList.c, src/muscle-tab.c: Adjust aver uses to resist
to -DNDEBUG.
Building C++ parsers with -Wsuggest-attribute=const and
-Wsuggest-attribute=noreturn triggers warning in generated code.
* data/lalr1.cc: Call b4_attribute_define.
(debug_stream, debug_level): Flag as pure.
* tests/headers.at (Several parsers): There are now more YY macros
that "leak".
Some tests now fail when compiled with G++ 4.3 or 4.4 on MacPorts.
* tests/local.at (AT_SKIP_IF_EXCEPTION_SUPPORT_IS_POOR): New.
* tests/c++.at (Exception safety): Use it.
* configure.ac (ENABLE_YACC): New conditional.
(YACC_SCRIPT, YACC_LIBRARY): Remove.
* lib/local.mk, src/local.mk: Use the former instead of the latter.
* doc/local.mk: Use ENABLE_YACC to avoid installing yacc.1.
* src/scan-code.l (show_sub_message):
Redo initializations to work around a bogus Sun C 5.12 warning.
(parse_ref): Remove unreachable code that Sun C 5.12 complains about.
* src/uniqstr.h (uniqstr_vsprintf): Use
_GL_ATTRIBUTE_FORMAT_PRINTF (...) instead of __attribute__
((__format__ (__printf__, ...))). Otherwise, Sun C 5.12
complains about an unknown attribute.
* configure.ac (FLEX_CXX_WORKS): New AM_CONDITIONAL.
* examples/calc++/local.mk (examples/calc++/calc++):
Build if FLEX_CXX_WORKS, not BISON_CXX_WORKS.
Currently "-Werror -Wno-error=foo" still turns "foo" warnings into errors.
Reported by Alexandre Duret-Lutz.
See http://lists.gnu.org/archive/html/bug-bison/2013-09/msg00015.html.
* src/complain.c (errority, errority_flag): New.
(complain_init): Initialize the latter.
(warning_argmatch): Extract the loop iterating on the flag's bits.
Set and unset errority_flag here.
(warnings_argmatch): -Wno-error is not the same as -Wno-error=everything:
we must remember if category foo was explicitly turned in an error/warning
via -W(no-)error=foo.
(warning_severity): Use errority_flag.
* tests/input.at (Symbols): Just check --yacc, not -Wyacc, that's the
job of tests on -W.
(-Werror is not affected by -Wnone and -Wall): Rename as...
(-Werror combinations): this.
Tests more combinations of -W, -W(no-)error, and -W(no-)error=foo.
* tests/local.at (AT_BISON_CHECK_WARNINGS): Don't expect -Werror
to turn runs that issue warnings into runs with errors, as the
warnings might be enforced as warnings by -Wno-error=foo, in which
case -Werror does not change anything.
* doc/bison.texi (Bison Options): Try to be clearer about how
-W(no-)error and -W(no-)error=foo interact.
* origin/maint:
glr: more assertions
glr: shorten scopes
glr: formatting changes
glr: better use of tracing macros
examples: improve the output of the "variant" example
variant: remove useless assertion
tests: remove stray debugging traces
tests: do not use grep -q
build: don't require flex for ordinary builds
maint: update .gitignore
build: port to pre-5.8.7 perl
tests: minor change to make it easier to test other skeletons
uniqstr: fix assertion
The commit which introduces yyresolveLocations (commit
8710fc41aa) saves and restores the
look-ahead (type, value and location) for no clear reason. This
appears to be useless.
* data/glr.c (yyresolveLocations): Don't save/restore the current
look-ahead to call YYLLOC_DEFAULT.
Minor style changes.
Rather than having duplicate info in the symbol and the alias that has
to be resolved later on, both the symbol and the alias have a common
pointer to a separate structure containing this info.
* src/symtab.h (sym_content): New structure.
* src/symtab.c (sym_content_new, sym_content_free, symbol_free): New
* src/AnnotationList.c, src/conflicts.c, src/gram.c, src/gram.h,
* src/graphviz.c, src/ielr.c, src/output.c, src/parse-gram.y, src/print.c
* src/print-xml.c, src/print_graph.c, src/reader.c, src/reduce.c,
* src/state.h, src/symlist.c, src/symtab.c, src/symtab.h, src/tables.c:
Adjust.
* tests/input.at: Fix expectations (order changes).
We don't ship the *.txt files that are used to build the info
file.
Reported by Colin Daley.
* doc/figs/example.txt: New.
* doc/local.mk (bison.info): Depend on the txt files.
And ship them.
* tests/c++.at (Exception safety): In variant mode $$ is an instance
of Object. Assigning YY_NULL in C++98 is incorrect, but behaves ok,
as it assigns YY_NULL=0 using Object::operator= (char v). It is wrong
in C++11 as there is operator for "$$ = nullptr".
Again some issues with the fact that yylval is reported by GCC as
possibly not initialized in some cases. Here, the case at hand is the
%destructor.
I am still not convinced that it is worth going all the trouble of
using pragmas to disable temporarily some warnings, instead of just
initializing the looking symbol once for all, but that's what Paul
voted for, see
<http://lists.gnu.org/archive/html/bison-patches/2012-10/msg00050.html>.
* data/c.m4 (b4_attribute_define): Define
YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN, YY_IGNORE_MAYBE_UNINITIALIZED_END,
YY_INITIAL_VALUE here, as we will need them in the generation of the
destructor function, which is defined in yacc.c before yyparse, which
was in charge of defining these macros.
* data/yacc.c (b4_declare_scanner_communication_variables): Simplify:
trying to factor the definitions of the case pure and impure is
too complex.
Actually, it is not even clear that this macro should really exist,
as even the calls are complex.
Be careful not to issue a lone ";", as this is a statement, and C90
forbids declarations after statements ; so write
"YY_INITIAL_VALUE(Decl;)", not "YY_INITIAL_VALUE(Decl);".
There are possible conflicts between gnulib replacement functions (in
<stdio.h>) and their C++ wrappers (in <stream>). Trying to address
these in configure seems too hard, and I don't know how to fix the issue
in gnulib. Cowardly avoid the problem by skipping C++ tests when this
happens.
Reported by Stefano Lattarini.
http://lists.gnu.org/archive/html/bug-bison/2013-06/msg00001.html
* tests/atlocal.in (BISON_CXX_WORKS): Also set it to "skip" if we can't
compile a simple program using <stream>.
* tests/local.at: Comment changes.
# $SSH_PRIVATE_KEY is multiline. Use $'...' to register its value: $'-----BEGIN OPENSSH PRIVATE KEY-----\nXXXXX...\n...==\n-----END OPENSSH PRIVATE KEY-----'.
# We always use _deps and _libs instead of .deps and .libs, because
# the latter is an invalid name on 8+3 MS-DOS file system. This makes
# the generated Makefiles good for every DJGPP installation, not only
# the one where the package was configured (which could happen to be
# a Windows box, where leading dots in file names are allowed).
/^rmdir[ ]*\.tst/i\
am__leading_dot=_
# Replace (command) > /dev/null with `command > /dev/null`, since
# parenthesized commands always return zero status in the ported Bash,
# even if the named command doesn't exist
/if ([^|;`]*null/{
s,(,`,
s,),,
/null[ ]*2>&1/s,2>&1,&`,
/null.*null/s,null.*null,&`,
/null.*null/!{
/null[ ]*2>&1/!s,null,&`,
}
}
# DOS-style absolute file names should be supported as well
/\*) top_srcdir=/s,/\*,[\\\\/]* | ?:[\\\\/]*,
# The following two items are changes needed for configuring
# and compiling across partitions.
# 1) The given srcdir value is always translated from the
# "x:" syntax into "/dev/x" syntax while we run configure.
/^[ ]*-srcdir=\*.*$/a\
ac_optarg=`echo "$ac_optarg" | sed "s,^\\([A-Za-z]\\):,/dev/\\1,"`
/set X `ls -Lt \$srcdir/i\
if `echo $srcdir | grep "^/dev/" - > /dev/null`; then\
srcdir=`echo "$srcdir" | sed -e "s%^/dev/%%" -e "s%/%:/%"`\
fi
# Autoconf 2.52e generated configure scripts
# write absolute paths into Makefiles and bison.in
# making them useless for DJGPP installations for
# which the package has not been configured for.
/MISSING=/,/^$/{
/^fi$/a\
am_missing_run=`echo "$am_missing_run" | sed 's%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%'`
}
/^install_sh=/a\
install_sh=`echo "$install_sh" | sed 's%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%${top_srcdir}%'`
# This will only work if the assumption that the
# testsuite is ran from the following path:
# ${top_srcdir}/tests/testsuite.dir
# holds. The explicit names are of no importance.
# ../../. == ${top_srcdir}
#/^esac$/,/^ac_abs_srcdir=/ {
/^ac_abs_srcdir=/i\
ac_abs_builddir=`echo "$ac_abs_builddir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%"`\
ac_abs_top_builddir=`echo "$ac_abs_top_builddir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%"`\
ac_abs_top_srcdir=`echo "$ac_abs_top_srcdir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../..%"`
#ac_abs_srcdir=`echo "$ac_abs_srcdir" | sed "s%/dev/.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%;s%.:.*/bison[-_0-9]\\{1,1\\}[-.0-9A-Za-z+]*%../../.%"`
/^ac_abs_srcdir=/a\
for dj_dir in $ac_abs_builddir $ac_abs_top_builddir $ac_abs_top_srcdir $ac_abs_srcdir\
do\
ac_abs_builddir=`echo "$ac_abs_builddir" | sed "s%//%/%g"`\
ac_abs_top_builddir=`echo "$ac_abs_top_builddir" | sed "s%//%/%g"`\
ac_abs_top_srcdir=`echo "$ac_abs_top_srcdir" | sed "s%//%/%g"`\
ac_abs_srcdir=`echo "$ac_abs_srcdir" | sed "s%//%/%g"`\
done
# Add DJGPP version information.
/^#define VERSION/s/\$VERSION/& (DJGPP port (r1))/
# We need makeinfo to make the html formated docs.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Akim Demaille
Bruno HaibleandAkim Demaille
Adrian VogelsgesangandAkim Demaille
Victor Morales CayuelaandAkim Demaille
Jim MeyeringandAkim Demaille
Paul Eggert
kaneko yandAkim Demaille
László VáradyandAkim Demaille
Theophile RanquetandAkim Demaille
H. S. TeohandAkim Demaille
Eric S. RaymondandAkim Demaille
Kiyoshi KanazawaandAkim Demaille
Eduard StaniloiuandAkim Demaille
Akim DemailleandAkim Demaille
Paul HilfingerandAkim Demaille
JannickandAkim Demaille
Adam SampsonandAkim Demaille
Josh SorefandAkim Demaille
Jiahao LiandAkim Demaille
Nate GuerinandAkim Demaille
Andreas SchwabandAkim Demaille
Valentin TolmerandAkim Demaille
Valentin TolmerandAkim Demaille