LAC = lookahead correction. See discussion at
<http://lists.gnu.org/archive/html/bison-patches/2009-09/msg00034.html>.
However, one point there must be corrected: because of %nonassoc,
LAC is *not* always redundant for lr.type=canonical-lr.
* data/yacc.c: Accept values of "none" (default) or "full" for
parse.lac. Accept %define parse.lac.es-capacity to specify
capacity of LAC's temporary exploratory stack. It defaults to 20
and, for now, will not grow dynamically.
(b4_lac_flag, b4_lac_if): New m4 macros. Evaluate as true for
parse.lac!=none.
(YYBACKUP): Invoke YY_LAC_DISCARD.
(YY_LAC_ESTABLISH, YY_LAC_DISCARD): New cpp macros that invoke
yy_lac and track when it needs to be invoked
(yy_lac): New function that, given the current stack, determines
whether a token can eventually be shifted. Return status mimics
yyparse return status.
(yysyntax_error): Change yystate argument to yyssp so stack top
can be passed to yy_lac. If LAC is requested, build expected
token list by invoking yy_lac for every token instead of just
checking the current state for lookaheads. Return 2 if yy_lac
exhausts memory.
(yyparse, yypush_parse): Use local variable yy_lac_established and
cpp macros YY_LAC_ESTABLISH and YY_LAC_DISCARD to implement LAC.
Update yysyntax_error invocation. Add yyexhaustedlab code if LAC
is requested.
* tests/conflicts.at (%nonassoc and eof): Extend to check the
effect of each of -Dlr.type=canonical-lr and -Dparse.lac=full.
(parse.error=verbose and consistent errors): Likewise.
(LAC: %nonassoc requires splitting canonical LR states): New test
group demonstrating how LAC can fix canonical LR.
* tests/input.at (LAC: Errors for %define): New test group.
* tests/regression.at (LAC: Exploratory stack): New test group.
(LAC: Memory exhaustion): New test group.
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it. With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage. Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (%error-verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general. Test
yacc.c, glr.c, lalr1.cc, and lalr1.java. No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
(cherry picked from commit d2060f0634)
Conflicts:
data/lalr1.cc
data/lalr1.java
src/parse-gram.c
src/parse-gram.h
tests/java.at
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it. With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage. Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
pass yyempty_ not yyla.type to yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (parse.error=verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general. Test
yacc.c, glr.c, lalr1.cc, and lalr1.java. No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
These are purely whitespace changes that result in ugly code
but that make the next couple of patches much easier to read.
* data/glr.c (yyreportSyntaxError): Reindent.
* data/lalr1.cc (yy::parser::yysyntax_error_): Reindent.
* data/lalr1.java (YYParser::yysyntax_error): Reindent.
* data/yacc.c (yysyntax_error): Reindent.
(cherry picked from commit b4bbc4a03c)
Conflicts:
src/parse-gram.c
src/parse-gram.h
These are purely whitespace changes that result in ugly code
but that make the next couple of patches much easier to read.
* data/glr.c (yyreportSyntaxError): Reindent.
* data/lalr1.cc (yy::parser::yysyntax_error_): Reindent.
* data/lalr1.java (YYParser::yysyntax_error): Reindent.
* data/yacc.c (yysyntax_error): Reindent.
* data/yacc.c (yyparse, yypush_parse): For yysyntax_error
invocation, get rid of the while loop, which is misleading
because there are really at most two iterations.
(cherry picked from commit 34db451c37)
* data/yacc.c (yyparse, yypush_parse): For yysyntax_error
invocation, get rid of the while loop, which is misleading
because there are really at most two iterations.
Jim Meyering reported this in
<http://lists.gnu.org/archive/html/bug-bison/2010-10/msg00007.html>.
* data/yacc.c: Use EXIT_SUCCESS as a witness for stdlib.h,
not _STDLIB_H. EXIT_SUCCESS has been defined by the standard
for quite some time.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
* tests/regression.at: Tamper with the renamed witness.
(cherry picked from commit 8ff146cd31)
Conflicts:
ChangeLog
src/parse-gram.c
src/parse-gram.h
Jim Meyering reported this in
<http://lists.gnu.org/archive/html/bug-bison/2010-10/msg00007.html>.
* data/yacc.c: Use EXIT_SUCCESS as a witness for stdlib.h,
not _STDLIB_H. EXIT_SUCCESS has been defined by the standard
for quite some time.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
* tests/regression.at: Tamper with the renamed witness.
Reported by Tys Lefering at
<http://lists.gnu.org/archive/html/bug-bison/2010-03/msg00035.html>.
This fix is already implemented in glr.c and does not apply to
lalr1.java.
* data/lalr1.cc (yy::parser::parse): Increase size of
yyerror_range and adjust subscripting so you don't have to
subtract one from the beginning of the array.
* data/yacc.c (b4_declare_parser_state_variables,
yyparse, yypush_parse): Likewise.
(cherry picked from commit 48f4100a82)
Conflicts:
data/lalr1.cc
data/yacc.c
src/parse-gram.c
src/parse-gram.h
Reported by Tys Lefering at
<http://lists.gnu.org/archive/html/bug-bison/2010-03/msg00035.html>.
This fix is already implemented in glr.c and does not apply to
lalr1.java.
* data/lalr1.cc (yy::parser::parse): Increase size of
yyerror_range and adjust subscripting so you don't have to
subtract one from the beginning of the array.
* data/yacc.c (b4_declare_parser_state_variables,
yyparse, yypush_parse): Likewise.
(cherry picked from commit 48f4100a82)
Conflicts:
data/yacc.c
src/parse-gram.c
src/parse-gram.h
The problem was that yacc.c didn't always update the yychar
translation afterwards. However, other skeletons appear to be
fine. glr.c appears to already translate yychar before every
use. lalr1.cc does not define yychar and does not document its
replacement, yyla, for users. It does provide yyclearin, but
that does not manipulate yyla and thus requires no translation
update. In lalr1.java, yychar is out of scope during semantic
actions.
* NEWS (2.5): Document.
* data/yacc.c (YYBACKUP): Don't bother translating yychar into
yytoken here.
(yyparse, yypush_parse): Instead, translate before every use of
yytoken, and add comments explaining this approach.
* tests/actions.at (Destroying lookahead assigned by semantic
action): New test group checking that translation happens before
lookahead destructor calls at parser return. Previously,
incorrect destructors were called.
* tests/conflicts.at (%error-verbose and consistent
errors): New test group checking that translation happens at
syntax error detection before the associated verbose error
message and the associated lookahead destructor calls. While
the destructor call is fixed by this patch, the verbose error
message is currently incorrect due to another bug (see
comments in test group), so this is an expected failure for now.
(cherry picked from commit df222dfaf4)
Conflicts:
src/parse-gram.c
src/parse-gram.h
The problem was that yacc.c didn't always update the yychar
translation afterwards. However, other skeletons appear to be
fine. glr.c appears to already translate yychar before every
use. lalr1.cc does not define yychar and does not document its
replacement, yyla, for users. It does provide yyclearin, but
that does not manipulate yyla and thus requires no translation
update. In lalr1.java, yychar is out of scope during semantic
actions.
* NEWS (2.5): Document.
* data/yacc.c (YYBACKUP): Don't bother translating yychar into
yytoken here.
(yyparse, yypush_parse): Instead, translate before every use of
yytoken, and add comments explaining this approach.
* tests/actions.at (Destroying lookahead assigned by semantic
action): New test group checking that translation happens before
lookahead destructor calls at parser return. Previously,
incorrect destructors were called.
* tests/conflicts.at (parse.error=verbose and consistent
errors): New test group checking that translation happens at
syntax error detection before the associated verbose error
message and the associated lookahead destructor calls. While
the destructor call is fixed by this patch, the verbose error
message is currently incorrect due to another bug (see
comments in test group), so this is an expected failure for now.
* data/yacc.c: Do not check YYLTYPE_IS_TRIVIAL if location
tracking is not enabled. Instead, unconditionally define
YY_LOCATION_PRINT as a no-op for backward compatibility.
(cherry picked from commit fcd32abd3a)
* data/glr.c: Check if YYENABLE_NLS and YYLTYPE_IS_TRIVIAL are
defined before using them.
* data/lalr1.cc: Likewise.
* data/yacc.c: Likewise.
(cherry picked from commit ae93128c96)
* data/yacc.c: Do not check YYLTYPE_IS_TRIVIAL if location
tracking is not enabled. Instead, unconditionally define
YY_LOCATION_PRINT as a no-op for backward compatibility.
Except when memory reallocation is required, this change
eliminates the need to invoke yysyntax_error twice and thus to
repeat the collection of lookaheads. It also prepares for
future extensions that will make those repetitions more
expensive and that will require additional memory management in
yysyntax_error. Finally, it fixes an obscure bug already
exercised in the test suite.
* data/yacc.c (yysyntax_error): Add arguments for message
buffer variables stored in the parser. Instead of size, return
status similar to yyparse status but indicating success of
message creation. Other than the actual reallocation of the
message buffer, import and clean up memory management code
from...
(yyparse, yypush_parse): ... here.
* tests/regression.at (%error-verbose overflow): No longer an
expected failure.
(cherry picked from commit 45319f1365)
Conflicts:
src/parse-gram.c
src/parse-gram.h
* data/yacc.c (yysyntax_error): Take the transated token instead
of the raw number.
Adjust callers.
(cherry picked from commit 84eedf86fe)
Conflicts:
TODO
* data/yacc.c (yysyntax_error): Rewrite, using a switch instead
of building dynamically the format string.
(cherry picked from commit eeb2942237)
Conflicts:
data/lalr1.cc
Except when memory reallocation is required, this change
eliminates the need to invoke yysyntax_error twice and thus to
repeat the collection of lookaheads. It also prepares for
future extensions that will make those repetitions more
expensive and that will require additional memory management in
yysyntax_error. Finally, it fixes an obscure bug already
exercised in the test suite.
* data/yacc.c (yysyntax_error): Add arguments for message
buffer variables stored in the parser. Instead of size, return
status similar to yyparse status but indicating success of
message creation. Other than the actual reallocation of the
message buffer, import and clean up memory management code
from...
(yyparse, yypush_parse): ... here.
* tests/regression.at (parse.error=verbose overflow): No longer
an expected failure.
* data/yacc.c: Clean up M4 for readability, and make output
whitespace more consistent. For the main parse function
comment, instead of saying "yyparse or yypush_parse", say either
"yyparse" or "yypush_parse" depending on which it actually is.
(cherry picked from commit bb31eb56ab)
Conflicts:
src/parse-gram.c
src/parse-gram.h
* data/yacc.c: Clean up M4 for readability, and make output
whitespace more consistent. For the main parse function
comment, instead of saying "yyparse or yypush_parse", say either
"yyparse" or "yypush_parse" depending on which it actually is.
* data/glr.c, data/lalr1.cc, data/lalr1.java, data/yacc.c: Don't
mention zero values in the YYTABLE comments.
* data/glr.c (yytable_value_is_error): Don't check for zero
value.
* data/lalr1.cc (yy_table_value_is_error_): Likewise.
* data/yacc.c (yytable_value_is_error): Likewise.
* data/lalr1.java (yy_table_value_is_error_): Likewise.
(yysyntax_error): Fix typo in code: use yytable_ not yycheck_.
* src/tables.h: In header comments, explain why it's useless to
check for a zero value in yytable.
(cherry picked from commit aa0cb40d61)
Conflicts:
data/bison.m4
data/lalr1.java
* data/bison.m4 (b4_integral_parser_tables_map): Don't mention
zero values in the YYTABLE comments.
* data/glr.c (yytable_value_is_error): Don't check for zero
value.
* data/lalr1.cc (yy_table_value_is_error_): Likewise.
* data/yacc.c (yytable_value_is_error): Likewise.
* data/lalr1.java (yy_table_value_is_error_): Likewise.
(yysyntax_error): Fix typo in code: use yytable_ not yycheck_.
* src/tables.h: In header comments, explain why it's useless to
check for a zero value in yytable.
* data/c.m4 (b4_table_value_equals): Comment that YYID must be
defined.
* data/glr.c, data/lalr1.cc, data/lalr1.java, data/yacc.c: Fix
yytable comments: zero indicates syntax error not default
action.
* data/glr.c (yyis_pact_ninf): Rename to...
(yypact_value_is_default): ... this.
(yyisDefaultedState): Update for rename.
(yyis_table_ninf): Rename to...
(yytable_value_is_error): ... this, and check for value zero
besides just YYTABLE_NINF.
(yygetLRActions): Check for default value from yypact. It
appears that this check is always performed before this function
is invoked, and so adding the check here is probably redundant.
However, the code may evolve after this subtlety is forgotten.
Also, update for rename to yytable_value_is_error. Because that
macro now checks for zero, a different but equivalent branch of
the if-then-else here is evaluated.
(yyreportSyntaxError): Update for rename to
yytable_value_is_error. The zero condition was mishandled
before.
(yyrecoverSyntaxError): Update for renames. No behavioral
changes.
* data/lalr1.cc, data/lalr1.java (yy_pact_value_is_default_):
New function.
(yy_table_value_is_error_): New function.
(parse): Use new functions where possible. No behavioral
changes.
(yysyntax_error_, yysyntax_error): Use yy_table_value_is_error_.
The zero condition was mishandled before.
* data/yacc.c (yyis_pact_ninf): Rename to...
(yypact_value_is_default): ... this.
(yyis_table_ninf): Rename to...
(yytable_value_is_error): ... this, and check for value zero
besides just YYTABLE_NINF.
(yysyntax_error): Update for rename to yytable_value_is_error.
The zero condition was mishandled before.
(yyparse): Update for renames. No behavioral changes.
* src/tables.h: Improve comments about yypact, yytable, etc.
more. Most importantly, say yytable value of zero means syntax
error not default action.
(cherry picked from commit f2b30bdf37)
Conflicts:
data/bison.m4
data/lalr1.cc
data/lalr1.java
data/yacc.c
src/parse-gram.c
src/parse-gram.h
* data/bison.m4 (b4_integral_parser_tables_map): Fix YYTABLE
comments: zero indicates syntax error not default action.
* data/c.m4 (b4_table_value_equals): Comment that YYID must be
defined.
* data/glr.c (yyis_pact_ninf): Rename to...
(yypact_value_is_default): ... this.
(yyisDefaultedState): Update for rename.
(yyis_table_ninf): Rename to...
(yytable_value_is_error): ... this, and check for value zero
besides just YYTABLE_NINF.
(yygetLRActions): Check for default value from yypact. It
appears that this check is always performed before this function
is invoked, and so adding the check here is probably redundant.
However, the code may evolve after this subtlety is forgotten.
Also, update for rename to yytable_value_is_error. Because that
macro now checks for zero, a different but equivalent branch of
the if-then-else here is evaluated.
(yyreportSyntaxError): Update for rename to
yytable_value_is_error. The zero condition was mishandled
before.
(yyrecoverSyntaxError): Update for renames. No behavioral
changes.
* data/lalr1.cc, data/lalr1.java (yy_pact_value_is_default_):
New function.
(yy_table_value_is_error_): New function.
(parse): Use new functions where possible. No behavioral
changes.
(yysyntax_error_, yysyntax_error): Use yy_table_value_is_error_.
The zero condition was mishandled before.
* data/yacc.c (yyis_pact_ninf): Rename to...
(yypact_value_is_default): ... this.
(yyis_table_ninf): Rename to...
(yytable_value_is_error): ... this, and check for value zero
besides just YYTABLE_NINF.
(yysyntax_error): Update for rename to yytable_value_is_error.
The zero condition was mishandled before.
(yyparse): Update for renames. No behavioral changes.
* src/tables.h: Improve comments about yypact, yytable, etc.
more. Most importantly, say yytable value of zero means syntax
error not default action.
For now, just api.push-pull and lr.keep-unreachable-states.
Maintain old names for backward compatibility.
* NEWS (2.5): Document.
* data/c.m4 (b4_identification): Update comment.
* data/yacc.c: Update access.
* doc/bison.texinfo: Update.
* etc/bench.pl.in (bench_push_parser): Update use.
* src/files.c (tr): Move to...
* src/getargs.c, src/getargs.h (tr): ... here because I can't
think of a better place to expose it. My logic is that, for all
uses of tr so far, command-line arguments can be involved, and
getargs.h is already included.
* src/main.c (main): Update access.
* src/muscle_tab.c (muscle_percent_define_insert): Convert old
variable names to new variable names before assigning value.
* src/reader.c (reader): Update setting default.
* tests/calc.at: Update uses.
* tests/conflicts.at (Unreachable States After Conflict
Resolution): Update use.
* tests/input.at (%define enum variables): Update use.
(%define backward compatibility): New test group.
* tests/push.at: Update uses.
* tests/reduce.at: Update uses.
* tests/torture.at: Update uses.
(cherry picked from commit 812775a039)
Conflicts:
data/c.m4
etc/bench.pl.in
src/parse-gram.c
src/parse-gram.h
tests/conflicts.at
For now, just api.push-pull and lr.keep-unreachable-states.
Maintain old names for backward compatibility.
* NEWS (2.5): Document.
* data/c.m4 (b4_identification): Update comment.
* data/yacc.c: Update access.
* doc/bison.texinfo: Update.
* etc/bench.pl.in (bench_grammar): Update use.
* src/files.c (tr): Move to...
* src/getargs.c, src/getargs.h (tr): ... here because I can't
think of a better place to expose it. My logic is that, for all
uses of tr so far, command-line arguments can be involved, and
getargs.h is already included.
* src/main.c (main): Update access.
* src/muscle_tab.c (muscle_percent_define_insert): Convert old
variable names to new variable names before assigning value.
* src/reader.c (reader): Update setting default.
* tests/calc.at: Update uses.
* tests/conflicts.at (Unreachable States After Conflict
Resolution): Update use.
* tests/input.at (%define enum variables): Update use.
(%define backward compatibility): New test group.
* tests/push.at: Update uses.
* tests/reduce.at: Update uses.
* tests/torture.at: Update uses.
This allows to pass -Derror_verbose on the command line. Better yet, it
allows to pass -Derror_verbose=$(ERROR_VERBOSE), with ERROR_VERBOSE being
defined as false or true.
* data/bison.m4 (b4_percent_define_if_define): Instead of relying
on b4_percent_define_ifdef, for does not check the defined value,
but only whether the symbol is defined, rely on
b4_percent_define_flag_if, so that a value of "false" is processed
as a false.
If not defined, define the flag to "false".
(b4_error_verbose_if): New.
* data/glr.c, data/lalr1.cc, data/yacc.c: Use it instead of
b4_error_verbose_flag.
* src/getargs.h, src/getargs.c (error_verbose_flag): Remove.
* src/output.c (prepare): Don't output it.
* src/parse-gram.y (%error-verbose): Treat as %define error_verbose.
