Work on some other areas of Bison revealed that some macros expanded
to be expanded only once were actually expanded several times. This
was due to the fact that changecom was not properly restored each
time, and macro names appearing in comments were then expanded.
Introduce begin/end macros which are easier to match that
changecom()/changecom(#).
* data/bison.m4 (b4_output_begin, b4_output_end): New.
* data/glr.c, data/glr.cc, data/lalr1.cc, data/lalr1.java,
* data/location.cc, data/stack.hh, data/yacc.c:
Use them.
This makes the interface for yyerror() pure without the need for a spurious
parse_param.
* data/yacc.c (b4_pure_if, b4_pure_flag): New definition, accept three states.
(b4_yacc_pure_if): Rename as...
(b4_yyerror_arg_loc_if): This, and use b4_pure_flag.
* tests/actions.at (%define api.pure): Modernize.
* test/calc.at (Simple LALR Calculator): Modernize.
* tests/local.at (AT_YYERROR_ARG_LOC_IF): Adjust.
The initial location might be used if the parser starts by an empty
reduction, so really ensure proper initialization of the initial
location. The previous approach fails for PostgreSQL, which uses
Reported by Peter Eisentraut.
http://lists.gnu.org/archive/html/bug-bison/2012-11/msg00023.html
With help from Théophile Ranquet.
* data/yacc.c (b4_declare_scanner_communication_variables): Be sure
to initialize yylloc, even when its structure is unknown.
(yyparse): Simplify the call to b4_dollar_pushdef.
* tests/actions.at (Initial location): Check of similar pattern
as in the case of PostgreSQL.
There are several issues to address here. One is that yylloc should
be initialized when possible. Another is that the push parser needs
to update yypushed_loc when the user modified it. And if the parser
starts by a reduction of an empty, it uses the first location on the
stack, which, therefore, must also be initialized to this initial
location.
This is getting complex, especially since because initializing a
global (impure interface) is different from initializing a local
variable. To simplify, the local yylloc is not initialized during its
definition.
* data/c.m4 (b4_yyloc_default_define): Replace by...
(b4_yyloc_default): this.
Adjust dependencies.
* data/glr.cc: Initialize yylloc.
* data/yacc.c (b4_declare_scanner_communication_variables):
Initialize yylloc during its definition.
Don't define yyloc_default.
(yypush_parse): The location formal is not const, as we might
initialize it.
(yyparse): Define yyloc_default.
Use it before running the user initial action.
Possibly update the first location on the stack, and the pushed
location after the user initial action.
* tests/actions.at (Initial locations): Check that the initial
location is correct.
* data/c.m4 (b4_yy_location_print_define): New.
Now issues "short" locations, e.g., "1.1" instead of "1.1-1.1".
Was initially a function, but then we face "static but unused"
warnings.
Simpler as a macro.
* tests/local.at, data/glr.c, data/yacc.c: Use it instead of duplicating.
* tests/actions.at: Adjust expectations.
* data/yacc.c: Fuse the initializations of yyssp, yyss and the like.
Remove an obsolete comment: we do initialize these initial stack
members (in some cases).
When generating a pure push parser, the initialization of yylval and
yylloc may not be visible to the compiler. With warnings enabled, GCC
4.3.6, 4.4.7, 4.5.4, and 4.6.3 report uninitialized uses of
yylval/yylloc. Using local pragmas to disable these warnings is not
supported before 4.6, and 4.6 does not support it properly. So
initialize yylval and yylloc at their definition. Reported by Peter
Simons. See
http://lists.gnu.org/archive/html/bison-patches/2012-10/msg00133.html
* data/c.m4 (b4_yyloc_default_define): New.
* data/yacc.c: Use it when locations are requested.
(YYLVAL_INITIALIZE): Replace by...
(YY_INITIAL_VALUE): this.
(yyparse): Initialize yylloc and yylval.
Therefore, remove the initialization of yylloc's field.
* data/glr.c: Likewise.
See http://lists.gnu.org/archive/html/bison-patches/2012-08/msg00024.html
(spreading over September and October).
* data/yacc.c (YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN)
(YY_IGNORE_MAYBE_UNINITIALIZED_END, YYLVAL_INITIALIZE):
New macros. Use them to suppress an unwanted GCC diagnostic.
When no header was to be generated, Bison would issue:
/* In a future release of Bison, this section will be replaced
by #include "". */
#ifndef YY_
# define YY_
It now properly generates nothing.
* data/c.m4 (b4_cpp_guard_open, b4_cpp_guard_close): Issue nothing when
the file name is empty.
* data/yacc.c: Do not generate the above comment when there is no header
to generate.
* NEWS: Update.
There are still spurious spaces at the end of some lines. But this is
addressed in the master branch, and I am reluctant to try to backport
this.
* data/yacc.c, data/glr.c, data/lalr1.cc, data/glr.cc: here.
* tests/calc.at (AT_CHECK_SPACES): New.
Use it.
Be sure not to introduce trailing empty lines in the *.y files.
* NEWS: Doc it.
* cfg.mk (syntax-check): Remove the exception.
scan-code.l is already passing argument to b4_dollar_dollar for the
initial acton, but its definition (of b4_dollar_dollar) does not use
this argument.
Generalize this definition, and use it for the %initial-action too.
* data/c.m4 (b4_dollar_dollar_, b4_dollar_pushdef, b4_dollar_popdef):
Instead of expecting a pointer, require a value, and use ".".
Since they are now generic enough, move to...
* data/c-like.m4: this new file.
* data/c.m4, data/java.m4: Load it.
* data/glr.c, data/lalr1.cc, data/lalr1.java, data/yacc.c: Use
b4_dollar_pushdef for the %initial-action.
* tests/actions.at: Check that.
* data/Makefile.am: Adjust.
* NEWS, doc/bison.texi: Document.
* data/c++.m4, data/c.m4 (b4_yylloc_default_define): New.
* data/glr.c, data/glr.cc, data/lalr1.cc, data/yacc.c: Use it.
* data/glr.cc: Do not define YYLLOC_DEFAULT in the header file,
but in the implementation one.
The following mixture is insane:
#define YYSTYPE_IS_TRIVIAL PREFIX_STYPE_IS_TRIVIAL
#if (defined YYSTYPE_IS_TRIVIAL && YYSTYPE_IS_TRIVIAL)
since, of course YYSTYPE_IS_TRIVIAL is defined. Instead we could
define YYSTYPE_IS_TRIVIAL as PREFIX_STYPE_IS_TRIVIAL only when the
later is defined, but let's avoid stacking CPP on top of M4: rather, use
#if (defined PREFIX_STYPE_IS_TRIVIAL && PREFIX_STYPE_IS_TRIVIAL)
* data/glr.c, data/yacc.c: Use YYSTYPE_IS_TRIVIAL, YYSTYPE_IS_DECLARED,
YYLTYPE_IS_TRIVIAL and YYLTYPE_IS_DECLARED under their api.prefix-renamed
name.
The testsuite in master has shown weird errors for the "Mulitple
Parsers" tests: the caller of p5.parse() received some apparently
random value, while tracing p5.parse() showed that the function was
consistently returning 0.
It happens when mixing several parser headers, some generated without
%debug, others with. In particular the C++ parser was generated with
%debug, i.e., with:
#ifndef YYDEBUG
# define YYDEBUG 1
#endif
and compiled separatedly. Yet, its header was included after the one
of another parser, this time without %debug, i.e., with
#ifndef YYDEBUG
# define YYDEBUG 0
#endif
in its header. As a result, the parser was compiled with YYDEBUG set,
but its header was used without. Since the layout of the objects are
then completely different, boom.
Therefore, do not change the value of YYDEBUG. Rather, use it as a
default value for <API.PREFIX>DEBUG.
* data/c.m4 (b4_YYDEBUG_define): New.
(b4_declare_yydebug): Rename as...
(b4_yydebug_declare): this, for consistency.
* data/glr.c, data/glr.cc, data/lalr1.cc, data/yacc.c: Use it.
* NEWS: Document it.
* data/yacc.c (b4_shared_declarations): Include the header guards.
Do not include the header in the *.c file, duplicate it.
* NEWS (Future Changes): Extend, and announce the forthcoming change
about the use of the parser header.
There is one difference: now, even without --defines, we generate
extern declarations for these variables. The factoring is worth it.
* data/c.m4 (b4_declare_yylstype): Declare them.
* data/glr.c, data/yacc.c: Adjust.
* data/c.m4 (b4_declare_yydebug): New.
* data/glr.c, data/yacc.c (b4_shared_declarations): Use it.
Remove the corresponding code from the parser body.
* NEWS: Doc this.
yacc.c used to include two almost identical sections: one for the *.h
file, and another for the *.c file. The main difference is that in
the *.c file we used the yy* names (as %name-prefix is handled by
"#define yy* <prefix>*" before), while the *.hh used <prefix>* names.
Keep only the later. If this is troublesome, b4_shared_declarations
can easily take the desired prefix as argument.
* data/yacc.c (b4_shared_declarations): New.
Use it to factor duplicated declarations.
yacc.c and glr.c share common declarations. Their YYLTYPE are exactly
equal, and their YYSTYPE are sufficiently alike to be fused (its
declaration was protected by YYSTYPE_IS_DECLARED in yacc.c, but not in
glr.c). Besides, yacc.c duplicated the definitions of YYLTYPE and
YYSTYPE (*.h/*.c).
* data/c.m4 (b4_declare_yylstype): New.
* data/yacc.c, data/glr.c: Use it.
Using http://github.com/lyda/misspell-check, massage its
output into sed commands to perform the suggested changes.
Initially, I filtered out the THRU->Through changes, because
that failed to retain capitalization in the grammar token.
Instead, do this manually, beforehand:
sed -i s/THRU/THROUGH/ tests/existing.at
git ls-files|misspellings -f -|perl -nl \
-e '/^(.*?)\[(\d+)\]: (\w+) -> "(.*?)"$/ or next;' \
-e '($file,$n,$l,$r)=($1,$2,$3,$4); $q="'\''"; $r=~s/$q/$q\\$q$q/g;'\
-e 'print "sed -i $q${n}s!$l!$r!$q $file"'|bash
C++11 introduces "nullptr" which plays the role of C's NULL, in
replacement of "0". Fix the C++ skeletons to avoid warnings about
uses of "0" in place of "nullptr", and improve C skeletons to also use
this "nullptr" when compiled with a C++11 compiler.
* configure.ac: More C++ warnings.
* NEWS (2.5.1): Document this.
* data/c++.m4, data/c.m4 (b4_null_define): New.
(b4_null): Use YY_NULL instead of 0.
* data/glr.c, data/lalr1.cc, data/location.cc, data/yacc.c:
Call b4_null_define/b4_null where appropriate.
Use YY_NULL instead of NULL.
* data/location.cc (initialize): Accept a default argument,
YY_NULL.
* tests/actions.at, tests/calc.at: Adjust.
* data/glr.c, lib/libiberty.h, src/system.h (__attribute__):
Do not disable it when __STRICT_ANSI__ is defined, as, for
instance, it disables the __attribute__((unused)) which
protects us from some compiler warnings.
This was already done elsewhere in Bison, in 2001, see
4a0d893695.
* tests/regression.at: Adjust output.
* data/yacc.c, doc/bison.texinfo: Reword to avoid having to
disable that check.
* cfg.mk: No longer skip this test.
(cherry picked from commit 762caaf639)
Conflicts:
data/yacc.c
* cfg.mk: No longer skip it, except where EXIT_SUCCESS is used
as a witness for stdlib.h.
Skip this test when appropriate.
* data/yacc.c: Drop a note about why EXIT_SUCCESS is defined here.
Conflicts:
cfg.mk
Reported by David Kastrup:
https://lists.gnu.org/archive/html/bug-bison/2011-10/msg00002.html.
* data/yacc.c (YYBACKUP): Accept rhs size.
Restore the proper state value.
* TODO (YYBACKUP): Make it...
* tests/actions.at: a new test case.
* NEWS, THANKS: Update.
(cherry picked from commit d115aad911)
Conflicts:
TODO
data/yacc.c
* data/glr.c, data/yacc.c: Do not use apostrophes in '#' comments,
as they confuse xgettext, which tries to parse them as C character
constants in a preprocessor directive.
* data/yacc.c (yy_lac): Don't use printf %d format on *yyesp, as
that expression might not promote to int on some platforms.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
(cherry picked from commit 3c9b20ff5a)
Conflicts:
data/yacc.c
src/parse-gram.c
src/parse-gram.h
* data/yacc.c: Rename %define variable parse.lac.es-capacity to
parse.lac.es-capacity-initial. Accept parse.lac.memory-trace
with values of "failures" (default) or "full".
(b4_declare_parser_state_variables): Add yyesa, yyes, and
yyes_capacity variables.
(YYSTACK_USE_ALLOCA): Ignore it if LAC requested.
(YYSTACK_ALLOC, YYSTACK_FREE, YYSTACK_ALLOC_MAXIMUM): Define if
LAC requested.
(YYCOPY_NEEDED): New cpp macro.
(YYCOPY): Define if LAC requested.
(yy_lac_stack_realloc): New function implementing stack
reallocations. Use YYMAXDEPTH for maximum stack size given that
the stack should never need to grow larger than the main state
stack needs to grow without LAC.
(YY_LAC_ESTABLISH): Update yy_lac invocation.
(yy_lac): Add arguments for exploratory stack memory data
recorded in the main parser. Invoke yy_lac_stack_realloc when
reallocation is necessary.
(yysyntax_error): Add the same new arguments and pass them to
yy_lac.
(yypstate_delete): Free yyes if necessary.
(yyesa, yyes, yyes_capacity): #define these to yypstate members
in the case of push parsing.
(yyparse, yypush_parse): Initialize yyes and yyes_capacity.
Update yysyntax_error invocations. At yyreturn, free yyes if
necessary.
* src/parse-gram.y: %define parse.lac full.
* tests/input.at (LAC: errors for %define): Extend for
parse.lac-memory-trace.
* tests/regression.at (LAC: Exploratory stack): Extend to check
that stack reallocs happen when expected.
(LAC: Memory exhaustion): Update to use YYMAXDEPTH and
parse.lac.es-capacity-initial.
(cherry picked from commit 107844a3ee)
Conflicts:
src/parse-gram.c
src/parse-gram.h
src/parse-gram.y
LAC = lookahead correction. See discussion at
<http://lists.gnu.org/archive/html/bison-patches/2009-09/msg00034.html>.
However, one point there must be corrected: because of %nonassoc,
LAC is *not* always redundant for lr.type=canonical-lr.
* data/yacc.c: Accept values of "none" (default) or "full" for
parse.lac. Accept %define parse.lac.es-capacity to specify
capacity of LAC's temporary exploratory stack. It defaults to 20
and, for now, will not grow dynamically.
(b4_lac_flag, b4_lac_if): New m4 macros. Evaluate as true for
parse.lac!=none.
(YYBACKUP): Invoke YY_LAC_DISCARD.
(YY_LAC_ESTABLISH, YY_LAC_DISCARD): New cpp macros that invoke
yy_lac and track when it needs to be invoked
(yy_lac): New function that, given the current stack, determines
whether a token can eventually be shifted. Return status mimics
yyparse return status.
(yysyntax_error): Change yystate argument to yyssp so stack top
can be passed to yy_lac. If LAC is requested, build expected
token list by invoking yy_lac for every token instead of just
checking the current state for lookaheads. Return 2 if yy_lac
exhausts memory.
(yyparse, yypush_parse): Use local variable yy_lac_established and
cpp macros YY_LAC_ESTABLISH and YY_LAC_DISCARD to implement LAC.
Update yysyntax_error invocation. Add yyexhaustedlab code if LAC
is requested.
* tests/conflicts.at (%nonassoc and eof): Extend to check the
effect of each of -Dlr.type=canonical-lr and -Dparse.lac=full.
(%error-verbose and consistent errors): Likewise.
(LAC: %nonassoc requires splitting canonical LR states): New test
group demonstrating how LAC can fix canonical LR.
* tests/input.at (LAC: Errors for %define): New test group.
* tests/regression.at (LAC: Exploratory stack): New test group.
(LAC: Memory exhaustion): New test group.
(cherry picked from commit bf35c71c58)
Conflicts:
src/parse-gram.c
src/parse-gram.h
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it. With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage. Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (%error-verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general. Test
yacc.c, glr.c, lalr1.cc, and lalr1.java. No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
(cherry picked from commit d2060f0634)
Conflicts:
data/lalr1.cc
data/lalr1.java
src/parse-gram.c
src/parse-gram.h
tests/java.at
These are purely whitespace changes that result in ugly code
but that make the next couple of patches much easier to read.
* data/glr.c (yyreportSyntaxError): Reindent.
* data/lalr1.cc (yy::parser::yysyntax_error_): Reindent.
* data/lalr1.java (YYParser::yysyntax_error): Reindent.
* data/yacc.c (yysyntax_error): Reindent.
(cherry picked from commit b4bbc4a03c)
Conflicts:
src/parse-gram.c
src/parse-gram.h
* data/yacc.c (yyparse, yypush_parse): For yysyntax_error
invocation, get rid of the while loop, which is misleading
because there are really at most two iterations.
(cherry picked from commit 34db451c37)
