There is too much code duplication between %printer and %destructor.
We used to have two functions for each action: the first one for
destructors, the second one for printers. Factor using a
'code_props_type', and an array of code_props instead of two
members.
* src/symlist.h, src/symlist.c (symbol_list_destructor_set)
(symbol_list_printer_set): Fuse into...
(symbol_list_code_props_set): this.
* src/symtab.h, src/symtab.c (default_tagged_destructor)
(default_tagged_printer): Fuse into...
(default_tagged_code_props): this.
(default_tagless_destructor, default_tagless_printer)
(default_tagless_code_props): Likewise.
(code_props_type_string): new.
(symbol_destructor_set, symbol_destructor_get, semantic_type_destructor_set)
(default_tagged_destructor_set, default_tagless_destructor_set)
(symbol_printer_set, symbol_printer_get, semantic_type_printer_set)
(default_tagged_printer_set, default_tagless_printer_set): Replace by...
(symbol_code_props_set, symbol_code_props_get, semantic_type_code_props_set)
(default_tagged_code_props_set, default_tagless_code_props_set): these.
* src/parse-gram.y (grammar_declaration): Adjust.
* src/output.c (CODE_PROP, grammar_declaration): Ditto.
* src/reader.c (symbol_should_be_used): Ditto.
Basically, revert ba60c39547a445dee3e07920931b4d7a81843868's move to
mbs* functions, which was prompted by -DGNULIB_POSIXCHECK. See
<http://lists.gnu.org/archive/html/bison-patches/2012-05/msg00052.html>
and following.
* bootstrap.conf: No longer ask for them.
* src/files.c, src/getargs.c, src/location.c,
* src/parse-gram.c, src/parse-gram.y, src/scan-gram.l,
* src/symtab.c: s/mbs(r?chr)/str$1/g.
Using http://github.com/lyda/misspell-check, massage its
output into sed commands to perform the suggested changes.
Initially, I filtered out the THRU->Through changes, because
that failed to retain capitalization in the grammar token.
Instead, do this manually, beforehand:
sed -i s/THRU/THROUGH/ tests/existing.at
git ls-files|misspellings -f -|perl -nl \
-e '/^(.*?)\[(\d+)\]: (\w+) -> "(.*?)"$/ or next;' \
-e '($file,$n,$l,$r)=($1,$2,$3,$4); $q="'\''"; $r=~s/$q/$q\\$q$q/g;'\
-e 'print "sed -i $q${n}s!$l!$r!$q $file"'|bash
This change was made by applying emacs' untabify function to
nearly all files in Bison's repository. Required tabs in make
files, ChangeLog, regexps, and test code were manually skipped.
Other notable exceptions and changes are listed below.
* bootstrap: Skip because we sync this with gnulib.
* data/m4sugar/foreach.m4
* data/m4sugar/m4sugar.m4: Skip because we sync these with
Autoconf.
* djgpp: Skip because I don't know how to test djgpp properly, and
this code appears to be unmaintained anyway.
* README-hacking (Hacking): Specify that tabs should be avoided
where not required.
* bootstrap.conf (gnulib_modules): Add all modules suggested by
-DGNULIB_POSIXCHECK.
* src/files.c (file_name_split)
* src/getargs.c (getargs)
* src/location.c (boundary_set_from_string)
* src/output.c (output_skeleton)
* src/parse-gram.y (prologue_declaration)
* src/scan-gram.l (handle_syncline)
* src/symtab.c (symbol_new): Use mbschr and mbsrchr instead of
strchr and strrchr. In the cases of command-line options, file
names, and thus locations, functionality may be improved. In the
case of symbol names, there should be no functional difference as
all characters are ASCII, so the intended benefit is just warning
suppression.
(cherry picked from commit d143e9c33f)
Conflicts:
src/output.c
src/parse-gram.c
src/parse-gram.h
* bootstrap.conf (gnulib_modules): Add all modules suggested by
-DGNULIB_POSIXCHECK.
* src/files.c (file_name_split)
* src/getargs.c (getargs)
* src/location.c (boundary_set_from_string)
* src/output.c (token_definitions_output, output_skeleton)
* src/parse-gram.y (prologue_declaration)
* src/scan-gram.l (handle_syncline)
* src/symtab.c (symbol_new): Use mbschr and mbsrchr instead of
strchr and strrchr. In the cases of command-line options, file
names, and thus locations, functionality may be improved. In the
case of symbol names, there should be no functional difference as
all characters are ASCII, so the intended benefit is just warning
suppression.
* data/glr.c, data/yacc.c: Do not use apostrophes in '#' comments,
as they confuse xgettext, which tries to parse them as C character
constants in a preprocessor directive.
* data/yacc.c (yy_lac): Don't use printf %d format on *yyesp, as
that expression might not promote to int on some platforms.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
(cherry picked from commit 3c9b20ff5a)
Conflicts:
data/yacc.c
src/parse-gram.c
src/parse-gram.h
* data/glr.c, data/yacc.c: Do not use apostrophes in '#' comments,
as they confuse xgettext, which tries to parse them as C character
constants in a preprocessor directive.
* data/yacc.c (yy_lac): Don't use printf %d format on *yyesp, as
that expression might not promote to int on some platforms.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
* data/yacc.c: Rename %define variable parse.lac.es-capacity to
parse.lac.es-capacity-initial. Accept parse.lac.memory-trace
with values of "failures" (default) or "full".
(b4_declare_parser_state_variables): Add yyesa, yyes, and
yyes_capacity variables.
(YYSTACK_USE_ALLOCA): Ignore it if LAC requested.
(YYSTACK_ALLOC, YYSTACK_FREE, YYSTACK_ALLOC_MAXIMUM): Define if
LAC requested.
(YYCOPY_NEEDED): New cpp macro.
(YYCOPY): Define if LAC requested.
(yy_lac_stack_realloc): New function implementing stack
reallocations. Use YYMAXDEPTH for maximum stack size given that
the stack should never need to grow larger than the main state
stack needs to grow without LAC.
(YY_LAC_ESTABLISH): Update yy_lac invocation.
(yy_lac): Add arguments for exploratory stack memory data
recorded in the main parser. Invoke yy_lac_stack_realloc when
reallocation is necessary.
(yysyntax_error): Add the same new arguments and pass them to
yy_lac.
(yypstate_delete): Free yyes if necessary.
(yyesa, yyes, yyes_capacity): #define these to yypstate members
in the case of push parsing.
(yyparse, yypush_parse): Initialize yyes and yyes_capacity.
Update yysyntax_error invocations. At yyreturn, free yyes if
necessary.
* src/parse-gram.y: %define parse.lac full.
* tests/input.at (LAC: errors for %define): Extend for
parse.lac-memory-trace.
* tests/regression.at (LAC: Exploratory stack): Extend to check
that stack reallocs happen when expected.
(LAC: Memory exhaustion): Update to use YYMAXDEPTH and
parse.lac.es-capacity-initial.
(cherry picked from commit 107844a3ee)
Conflicts:
src/parse-gram.c
src/parse-gram.h
src/parse-gram.y
LAC = lookahead correction. See discussion at
<http://lists.gnu.org/archive/html/bison-patches/2009-09/msg00034.html>.
However, one point there must be corrected: because of %nonassoc,
LAC is *not* always redundant for lr.type=canonical-lr.
* data/yacc.c: Accept values of "none" (default) or "full" for
parse.lac. Accept %define parse.lac.es-capacity to specify
capacity of LAC's temporary exploratory stack. It defaults to 20
and, for now, will not grow dynamically.
(b4_lac_flag, b4_lac_if): New m4 macros. Evaluate as true for
parse.lac!=none.
(YYBACKUP): Invoke YY_LAC_DISCARD.
(YY_LAC_ESTABLISH, YY_LAC_DISCARD): New cpp macros that invoke
yy_lac and track when it needs to be invoked
(yy_lac): New function that, given the current stack, determines
whether a token can eventually be shifted. Return status mimics
yyparse return status.
(yysyntax_error): Change yystate argument to yyssp so stack top
can be passed to yy_lac. If LAC is requested, build expected
token list by invoking yy_lac for every token instead of just
checking the current state for lookaheads. Return 2 if yy_lac
exhausts memory.
(yyparse, yypush_parse): Use local variable yy_lac_established and
cpp macros YY_LAC_ESTABLISH and YY_LAC_DISCARD to implement LAC.
Update yysyntax_error invocation. Add yyexhaustedlab code if LAC
is requested.
* tests/conflicts.at (%nonassoc and eof): Extend to check the
effect of each of -Dlr.type=canonical-lr and -Dparse.lac=full.
(%error-verbose and consistent errors): Likewise.
(LAC: %nonassoc requires splitting canonical LR states): New test
group demonstrating how LAC can fix canonical LR.
* tests/input.at (LAC: Errors for %define): New test group.
* tests/regression.at (LAC: Exploratory stack): New test group.
(LAC: Memory exhaustion): New test group.
(cherry picked from commit bf35c71c58)
Conflicts:
src/parse-gram.c
src/parse-gram.h
* data/yacc.c: Rename %define variable parse.lac.es-capacity to
parse.lac.es-capacity-initial. Accept parse.lac.memory-trace
with values of "failures" (default) or "full".
(b4_declare_parser_state_variables): Add yyesa, yyes, and
yyes_capacity variables.
(YYSTACK_USE_ALLOCA): Ignore it if LAC requested.
(YYSTACK_ALLOC, YYSTACK_FREE, YYSTACK_ALLOC_MAXIMUM): Define if
LAC requested.
(YYCOPY_NEEDED): New cpp macro.
(YYCOPY): Define if LAC requested.
(yy_lac_stack_realloc): New function implementing stack
reallocations. Use YYMAXDEPTH for maximum stack size given that
the stack should never need to grow larger than the main state
stack needs to grow without LAC.
(YY_LAC_ESTABLISH): Update yy_lac invocation.
(yy_lac): Add arguments for exploratory stack memory data
recorded in the main parser. Invoke yy_lac_stack_realloc when
reallocation is necessary.
(yysyntax_error): Add the same new arguments and pass them to
yy_lac.
(yypstate_delete): Free yyes if necessary.
(yyesa, yyes, yyes_capacity): #define these to yypstate members
in the case of push parsing.
(yyparse, yypush_parse): Initialize yyes and yyes_capacity.
Update yysyntax_error invocations. At yyreturn, free yyes if
necessary.
* src/parse-gram.y: %define parse.lac full.
* tests/input.at (LAC: errors for %define): Extend for
parse.lac-memory-trace.
* tests/regression.at (LAC: Exploratory stack): Extend to check
that stack reallocs happen when expected.
(LAC: Memory exhaustion): Update to use YYMAXDEPTH and
parse.lac.es-capacity-initial.
LAC = lookahead correction. See discussion at
<http://lists.gnu.org/archive/html/bison-patches/2009-09/msg00034.html>.
However, one point there must be corrected: because of %nonassoc,
LAC is *not* always redundant for lr.type=canonical-lr.
* data/yacc.c: Accept values of "none" (default) or "full" for
parse.lac. Accept %define parse.lac.es-capacity to specify
capacity of LAC's temporary exploratory stack. It defaults to 20
and, for now, will not grow dynamically.
(b4_lac_flag, b4_lac_if): New m4 macros. Evaluate as true for
parse.lac!=none.
(YYBACKUP): Invoke YY_LAC_DISCARD.
(YY_LAC_ESTABLISH, YY_LAC_DISCARD): New cpp macros that invoke
yy_lac and track when it needs to be invoked
(yy_lac): New function that, given the current stack, determines
whether a token can eventually be shifted. Return status mimics
yyparse return status.
(yysyntax_error): Change yystate argument to yyssp so stack top
can be passed to yy_lac. If LAC is requested, build expected
token list by invoking yy_lac for every token instead of just
checking the current state for lookaheads. Return 2 if yy_lac
exhausts memory.
(yyparse, yypush_parse): Use local variable yy_lac_established and
cpp macros YY_LAC_ESTABLISH and YY_LAC_DISCARD to implement LAC.
Update yysyntax_error invocation. Add yyexhaustedlab code if LAC
is requested.
* tests/conflicts.at (%nonassoc and eof): Extend to check the
effect of each of -Dlr.type=canonical-lr and -Dparse.lac=full.
(parse.error=verbose and consistent errors): Likewise.
(LAC: %nonassoc requires splitting canonical LR states): New test
group demonstrating how LAC can fix canonical LR.
* tests/input.at (LAC: Errors for %define): New test group.
* tests/regression.at (LAC: Exploratory stack): New test group.
(LAC: Memory exhaustion): New test group.
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it. With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage. Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (%error-verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general. Test
yacc.c, glr.c, lalr1.cc, and lalr1.java. No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
(cherry picked from commit d2060f0634)
Conflicts:
data/lalr1.cc
data/lalr1.java
src/parse-gram.c
src/parse-gram.h
tests/java.at
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it. With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage. Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
pass yyempty_ not yyla.type to yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (parse.error=verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general. Test
yacc.c, glr.c, lalr1.cc, and lalr1.java. No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
These are purely whitespace changes that result in ugly code
but that make the next couple of patches much easier to read.
* data/glr.c (yyreportSyntaxError): Reindent.
* data/lalr1.cc (yy::parser::yysyntax_error_): Reindent.
* data/lalr1.java (YYParser::yysyntax_error): Reindent.
* data/yacc.c (yysyntax_error): Reindent.
(cherry picked from commit b4bbc4a03c)
Conflicts:
src/parse-gram.c
src/parse-gram.h
These are purely whitespace changes that result in ugly code
but that make the next couple of patches much easier to read.
* data/glr.c (yyreportSyntaxError): Reindent.
* data/lalr1.cc (yy::parser::yysyntax_error_): Reindent.
* data/lalr1.java (YYParser::yysyntax_error): Reindent.
* data/yacc.c (yysyntax_error): Reindent.
Jim Meyering reported this in
<http://lists.gnu.org/archive/html/bug-bison/2010-10/msg00007.html>.
* data/yacc.c: Use EXIT_SUCCESS as a witness for stdlib.h,
not _STDLIB_H. EXIT_SUCCESS has been defined by the standard
for quite some time.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
* tests/regression.at: Tamper with the renamed witness.
(cherry picked from commit 8ff146cd31)
Conflicts:
ChangeLog
src/parse-gram.c
src/parse-gram.h
Jim Meyering reported this in
<http://lists.gnu.org/archive/html/bug-bison/2010-10/msg00007.html>.
* data/yacc.c: Use EXIT_SUCCESS as a witness for stdlib.h,
not _STDLIB_H. EXIT_SUCCESS has been defined by the standard
for quite some time.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
* tests/regression.at: Tamper with the renamed witness.
These changes allow users to prefix an action with %? to indicate that it
is a semantic predicate---an expression that is evaluated immediately (not
deferred, even in GLR nondeterministic mode) and causes a syntax error if
false. In GLR parsers, this has the effect of killing one of a set of
split-off parses, just as would an ordinary syntax error.
Changelog:
* NEWS: Describe new semantic-predicate feature.
* data/c.m4 (b4_predicate_case): New definition.
* data/java.m4 (b4_predicate_case): New definition.
* data/glr.c (yyimmediate): Add definition.
(yydoAction): Remove comment, now obsolete.
Do YY_REDUCE_PRINT here.
(yyglrReduce): Alter comment to indicate that semantic values
need not be deferred.
Remove YY_REDUCE_PRINT from here; done in yydoAction.
(yyprocessOneStack): Pass immediate flag.
Delete stacks rejected by predicates in newly split-off parsers.
Change handling of yyerr so that only current stack gets deleted
when semantic predicate fails.
(yyfillin): Don't crash if a semantic value is unresolved (as may
happen in predicate rules).
Copy lr state as well in debugging mode.
Update comment on setting of yysval to include yyloc as well.
(yy_reduce_print): Add yynormal argument. Perform fillin properly.
Report unresolved RHS values.
(yyimmediate): New table.
* src/gram.h (struct rule): Add is_predicate field.
* src/output.c (user_actions_output): Use b4_predicate_case for
predicates.
(prepare_symbols): Output yyimmediate.
* src/scan-gram.l: Add %? token, SC_PREDICATE state.
* src/scan-code.l (code_props_rule_action_init): Add is_predicate
argument.
* src/scan-code.h (struct code_props): Add is_predicate field.
(code_props_rule_action_init): New interface.
* src/parse-gram.y (%?{...}): New token.
(rhs): Add %?{...} rule.
* src/parse-gram.c: Regenerate.
* src/parse-gram.h: Regenerate.
* src/reader.c (grammar_current_rule_action_append): Add
immediate argument.
(grammar_midrule_action): Use new interface for
code_props_rule_action_init.
(grammar_current_rule_action_append): Ditto.
(packgram): Transfer is_predicate value.
* src/reader.h (grammar_current_rule_action_append): New interface.
* doc/bison.texinfo: Document semantic predicates (%?).
* data/glr.c (yylhsNonterm, yyisDefaultedState,yyDefaultAction)
(yygetLRActions,yynewGLRStackItem,yyaddDeferredAction,yyinitStateSet)
(yyinitGLRStack,yyexpandGLRStack,yyupdateSplit,yymarkStackDeleted)
(yyundeleteLastStack,yyglrShift,yyglrShiftDefer,yydoAction,yyglrReduce)
(yyidenticalOptions,yymergeOptionSets,yyresolveStates,yyresolveAction)
(yyresolveLocations,yyresolveValue,yyreducePrint): Update parameter
names in comments and mention all parameters.
(struct yyGLRState): Fix description of yyposn field.
(yyresolveLocations): Correct comment so as not to imply action when
yyn1==0.
Reported by Tys Lefering at
<http://lists.gnu.org/archive/html/bug-bison/2010-03/msg00035.html>.
This fix is already implemented in glr.c and does not apply to
lalr1.java.
* data/lalr1.cc (yy::parser::parse): Increase size of
yyerror_range and adjust subscripting so you don't have to
subtract one from the beginning of the array.
* data/yacc.c (b4_declare_parser_state_variables,
yyparse, yypush_parse): Likewise.
(cherry picked from commit 48f4100a82)
Conflicts:
data/lalr1.cc
data/yacc.c
src/parse-gram.c
src/parse-gram.h
Reported by Tys Lefering at
<http://lists.gnu.org/archive/html/bug-bison/2010-03/msg00035.html>.
This fix is already implemented in glr.c and does not apply to
lalr1.java.
* data/lalr1.cc (yy::parser::parse): Increase size of
yyerror_range and adjust subscripting so you don't have to
subtract one from the beginning of the array.
* data/yacc.c (b4_declare_parser_state_variables,
yyparse, yypush_parse): Likewise.
(cherry picked from commit 48f4100a82)
Conflicts:
data/yacc.c
src/parse-gram.c
src/parse-gram.h
The problem was that yacc.c didn't always update the yychar
translation afterwards. However, other skeletons appear to be
fine. glr.c appears to already translate yychar before every
use. lalr1.cc does not define yychar and does not document its
replacement, yyla, for users. It does provide yyclearin, but
that does not manipulate yyla and thus requires no translation
update. In lalr1.java, yychar is out of scope during semantic
actions.
* NEWS (2.5): Document.
* data/yacc.c (YYBACKUP): Don't bother translating yychar into
yytoken here.
(yyparse, yypush_parse): Instead, translate before every use of
yytoken, and add comments explaining this approach.
* tests/actions.at (Destroying lookahead assigned by semantic
action): New test group checking that translation happens before
lookahead destructor calls at parser return. Previously,
incorrect destructors were called.
* tests/conflicts.at (%error-verbose and consistent
errors): New test group checking that translation happens at
syntax error detection before the associated verbose error
message and the associated lookahead destructor calls. While
the destructor call is fixed by this patch, the verbose error
message is currently incorrect due to another bug (see
comments in test group), so this is an expected failure for now.
(cherry picked from commit df222dfaf4)
Conflicts:
src/parse-gram.c
src/parse-gram.h
The problem was that yacc.c didn't always update the yychar
translation afterwards. However, other skeletons appear to be
fine. glr.c appears to already translate yychar before every
use. lalr1.cc does not define yychar and does not document its
replacement, yyla, for users. It does provide yyclearin, but
that does not manipulate yyla and thus requires no translation
update. In lalr1.java, yychar is out of scope during semantic
actions.
* NEWS (2.5): Document.
* data/yacc.c (YYBACKUP): Don't bother translating yychar into
yytoken here.
(yyparse, yypush_parse): Instead, translate before every use of
yytoken, and add comments explaining this approach.
* tests/actions.at (Destroying lookahead assigned by semantic
action): New test group checking that translation happens before
lookahead destructor calls at parser return. Previously,
incorrect destructors were called.
* tests/conflicts.at (parse.error=verbose and consistent
errors): New test group checking that translation happens at
syntax error detection before the associated verbose error
message and the associated lookahead destructor calls. While
the destructor call is fixed by this patch, the verbose error
message is currently incorrect due to another bug (see
comments in test group), so this is an expected failure for now.
