* doc/bison.texinfo (Mid-Rule Actions): Mention that periods and
dashes make symbol names less convenient for named references.
* src/scan-code.l:
(handle_action_dollar): New arg textlen. All callers changed.
(handle_action_at): Likewise. Also, args are pointers to const.
(ref_tail_fields): Remove; no longer used.
(letter): Now includes '-' and '.', since this is for Bison
identifiers.
(id): Now the simpler traditional defn, since letters now include
'-' and '.'.
(c_letter, c_id): New defns.
(ref): Use c_id for unbracketed IDs.
(<SC_RULE_ACTION>): Simplify, now that the distinction between
Bison and unbracketed IDs are now in the regular expressions.
(VARIANT_BAD_BRACKETING): Remove.
(VARIANT_NOT_VISIBLE_FROM_MIDRULE): Renumber.
(find_prefix_end): Remove, replacing with ....
(identifier_matches): New function.
(variant_add): Use it. Omit EXPLICIT_BRACKETING arg; no longer
needed. CP arg is pointer to constant. All callers changed.
(show_sub_messages): Remove args CP, EXPLICIT_BRACKETING, DOLLAR_OR_AT.
New arg TEXT. All callers changed. Do not worry about showing
trailing context.
(parse_ref): Args CP, RULE, TEXT are now pointers to const. New
arg TEXTLEN. Remove arg DOLLAR_OR_AT. All callers changed.
Simplify code now that the regular expressions capture the
restrictions.
* src/scan-gram.l (letter, id): Adjust to match scan-code.l.
* src/symlist.c (symbol_list_null): Arg is now pointer to const.
* src/symlist.h: Likewise.
* tests/named-refs.at (Misleading references): These are now caught
by the C compiler, not by Bison; that's good enough. Adjust test
to reflect this.
(Many kinds of errors, Unresolved references): Adjust expected
diagnostics to match new behavior. The same errors are caught,
though the diagnostics are not quite as fancy.
($ or @ followed by . or -): Likewise. Also, Make the grammar
unambiguous, so that diagnostics are not complicated by ambiguity
warnings.
* data/glr.c, data/yacc.c: Do not use apostrophes in '#' comments,
as they confuse xgettext, which tries to parse them as C character
constants in a preprocessor directive.
* data/yacc.c (yy_lac): Don't use printf %d format on *yyesp, as
that expression might not promote to int on some platforms.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
Previously, for this special case of an invalid reference, the
usual "symbol not found in production:" was printed. However,
because the symbol name was parsed as the empty string, that
message was followed immediately by a newline instead of a symbol
name. In reality, this is a syntax error, so the reference is
invalid regardless of the symbols actually appearing in the
production. Discussed at
<http://lists.gnu.org/archive/html/bison-patches/2011-01/msg00012.html>.
* src/scan-code.l (parse_ref): Report the above case as a syntax
error. Other than that, continue to handle this case like any
other invalid reference that Bison manages to parse because
"possibly meant" messages can still be helpful to the user.
* tests/named-refs.at ($ or @ followed by . or -): New test group.
Reported by Paul Hilfinger as a side effect of named references
support at
<http://lists.gnu.org/archive/html/bug-bison/2011-01/msg00000.html>.
Suggested by Paul Eggert.
* src/scan-code.l ({letter}, {id}): Adjust lexical definitions.
* src/scan-gram.l ({letter}, {id}): Likewise.
* data/yacc.c: Rename %define variable parse.lac.es-capacity to
parse.lac.es-capacity-initial. Accept parse.lac.memory-trace
with values of "failures" (default) or "full".
(b4_declare_parser_state_variables): Add yyesa, yyes, and
yyes_capacity variables.
(YYSTACK_USE_ALLOCA): Ignore it if LAC requested.
(YYSTACK_ALLOC, YYSTACK_FREE, YYSTACK_ALLOC_MAXIMUM): Define if
LAC requested.
(YYCOPY_NEEDED): New cpp macro.
(YYCOPY): Define if LAC requested.
(yy_lac_stack_realloc): New function implementing stack
reallocations. Use YYMAXDEPTH for maximum stack size given that
the stack should never need to grow larger than the main state
stack needs to grow without LAC.
(YY_LAC_ESTABLISH): Update yy_lac invocation.
(yy_lac): Add arguments for exploratory stack memory data
recorded in the main parser. Invoke yy_lac_stack_realloc when
reallocation is necessary.
(yysyntax_error): Add the same new arguments and pass them to
yy_lac.
(yypstate_delete): Free yyes if necessary.
(yyesa, yyes, yyes_capacity): #define these to yypstate members
in the case of push parsing.
(yyparse, yypush_parse): Initialize yyes and yyes_capacity.
Update yysyntax_error invocations. At yyreturn, free yyes if
necessary.
* src/parse-gram.y: %define parse.lac full.
* tests/input.at (LAC: errors for %define): Extend for
parse.lac-memory-trace.
* tests/regression.at (LAC: Exploratory stack): Extend to check
that stack reallocs happen when expected.
(LAC: Memory exhaustion): Update to use YYMAXDEPTH and
parse.lac.es-capacity-initial.
LAC = lookahead correction. See discussion at
<http://lists.gnu.org/archive/html/bison-patches/2009-09/msg00034.html>.
However, one point there must be corrected: because of %nonassoc,
LAC is *not* always redundant for lr.type=canonical-lr.
* data/yacc.c: Accept values of "none" (default) or "full" for
parse.lac. Accept %define parse.lac.es-capacity to specify
capacity of LAC's temporary exploratory stack. It defaults to 20
and, for now, will not grow dynamically.
(b4_lac_flag, b4_lac_if): New m4 macros. Evaluate as true for
parse.lac!=none.
(YYBACKUP): Invoke YY_LAC_DISCARD.
(YY_LAC_ESTABLISH, YY_LAC_DISCARD): New cpp macros that invoke
yy_lac and track when it needs to be invoked
(yy_lac): New function that, given the current stack, determines
whether a token can eventually be shifted. Return status mimics
yyparse return status.
(yysyntax_error): Change yystate argument to yyssp so stack top
can be passed to yy_lac. If LAC is requested, build expected
token list by invoking yy_lac for every token instead of just
checking the current state for lookaheads. Return 2 if yy_lac
exhausts memory.
(yyparse, yypush_parse): Use local variable yy_lac_established and
cpp macros YY_LAC_ESTABLISH and YY_LAC_DISCARD to implement LAC.
Update yysyntax_error invocation. Add yyexhaustedlab code if LAC
is requested.
* tests/conflicts.at (%nonassoc and eof): Extend to check the
effect of each of -Dlr.type=canonical-lr and -Dparse.lac=full.
(parse.error=verbose and consistent errors): Likewise.
(LAC: %nonassoc requires splitting canonical LR states): New test
group demonstrating how LAC can fix canonical LR.
* tests/input.at (LAC: Errors for %define): New test group.
* tests/regression.at (LAC: Exploratory stack): New test group.
(LAC: Memory exhaustion): New test group.
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it. With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage. Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
pass yyempty_ not yyla.type to yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (parse.error=verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general. Test
yacc.c, glr.c, lalr1.cc, and lalr1.java. No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
These are purely whitespace changes that result in ugly code
but that make the next couple of patches much easier to read.
* data/glr.c (yyreportSyntaxError): Reindent.
* data/lalr1.cc (yy::parser::yysyntax_error_): Reindent.
* data/lalr1.java (YYParser::yysyntax_error): Reindent.
* data/yacc.c (yysyntax_error): Reindent.
* bootstrap: Copy from gnulib's latest for the fix to
automatically anchor entries it constructs.
* gnulib: Update to latest just so it has the same bootstrap.
* .gitignore, build-aux/.gitignore, doc/.gitignore:
* lib/.gitignore, m4/.gitignore, po/.gitignore:
* runtime-po/.gitignore: Re-anchor all entries.
Jim Meyering reported this in
<http://lists.gnu.org/archive/html/bug-bison/2010-10/msg00007.html>.
* data/yacc.c: Use EXIT_SUCCESS as a witness for stdlib.h,
not _STDLIB_H. EXIT_SUCCESS has been defined by the standard
for quite some time.
* src/parse-gram.c, src/parse-gram.h: Regenerate.
* tests/regression.at: Tamper with the renamed witness.
* .cvsignore, build-aux/.cvsignore, doc/.cvsignore, etc/.cvsignore:
* examples/calc++/.cvsignore, lib/.cvsignore, m4/.cvsignore:
* po/.cvsignore, runtime-po/.cvsignore, src/.cvsignore:
* tests/.cvsignore: Remove; I don't use CVS to maintain Bison
anymore and don't know of anybody else who does. If someone needs
these files, they can resurrect them.
* .gitignore, build-aux/.gitignore, doc/.gitignore, lib/.gitignore:
* m4/.gitignore, po/.gitignore, runtime-po/.gitignore:
Omit leading '/', since bootstrap omits it.
Adjust file names to match current contents better.
* bootstrap: Sync from gnulib: this contains the new gnulib_mk_hook
installed just for us.
* bootstrap.conf (excluded_files): Don't exclude codeset.m4,
glibc21.m4, inttypes_h.m4, size_max.m4, xsize.m4, as they are now
needed somehow. Don't have time to look into why.
(gnulib_modules): Change malloc to malloc-gnu. Do we really assume
the GNU malloc behavior, where malloc (0) != NULL unless we're
out of storage? If not, we can omit malloc-gnu; but for now I left
it in to be safe.
(vc_ignore): Remove.
(gnulib_mk_hook): New function.
* README-hacking: Renamed from HACKING, since gnulib bootstrap now
uses that convention.
* NEWS (2.4.3): Document fix.
* src/complain.c (error_message): Extend to handle incomplete
error messages so warn and warn_at can be used in more cases.
* src/gram.c (grammar_rules_useless_report): Use warn_at so that
-Werror is always obeyed.
* src/reduce.c (reduce_print): Use warn so that the "warnings
being treated as errors" message is printed consistently before
the first warning message. This makes testing easier.
* tests/local.at (AT_BISON_WERROR_MSG): New macro.
(AT_BISON_CHECK_NO_XML): Extend to check -Werror and
--warnings=error when warnings appear in bison's stderr.
(cherry picked from commit 954474bfa1)
These changes allow users to prefix an action with %? to indicate that it
is a semantic predicate---an expression that is evaluated immediately (not
deferred, even in GLR nondeterministic mode) and causes a syntax error if
false. In GLR parsers, this has the effect of killing one of a set of
split-off parses, just as would an ordinary syntax error.
Changelog:
* NEWS: Describe new semantic-predicate feature.
* data/c.m4 (b4_predicate_case): New definition.
* data/java.m4 (b4_predicate_case): New definition.
* data/glr.c (yyimmediate): Add definition.
(yydoAction): Remove comment, now obsolete.
Do YY_REDUCE_PRINT here.
(yyglrReduce): Alter comment to indicate that semantic values
need not be deferred.
Remove YY_REDUCE_PRINT from here; done in yydoAction.
(yyprocessOneStack): Pass immediate flag.
Delete stacks rejected by predicates in newly split-off parsers.
Change handling of yyerr so that only current stack gets deleted
when semantic predicate fails.
(yyfillin): Don't crash if a semantic value is unresolved (as may
happen in predicate rules).
Copy lr state as well in debugging mode.
Update comment on setting of yysval to include yyloc as well.
(yy_reduce_print): Add yynormal argument. Perform fillin properly.
Report unresolved RHS values.
(yyimmediate): New table.
* src/gram.h (struct rule): Add is_predicate field.
* src/output.c (user_actions_output): Use b4_predicate_case for
predicates.
(prepare_symbols): Output yyimmediate.
* src/scan-gram.l: Add %? token, SC_PREDICATE state.
* src/scan-code.l (code_props_rule_action_init): Add is_predicate
argument.
* src/scan-code.h (struct code_props): Add is_predicate field.
(code_props_rule_action_init): New interface.
* src/parse-gram.y (%?{...}): New token.
(rhs): Add %?{...} rule.
* src/parse-gram.c: Regenerate.
* src/parse-gram.h: Regenerate.
* src/reader.c (grammar_current_rule_action_append): Add
immediate argument.
(grammar_midrule_action): Use new interface for
code_props_rule_action_init.
(grammar_current_rule_action_append): Ditto.
(packgram): Transfer is_predicate value.
* src/reader.h (grammar_current_rule_action_append): New interface.
* doc/bison.texinfo: Document semantic predicates (%?).
* data/glr.c (yylhsNonterm, yyisDefaultedState,yyDefaultAction)
(yygetLRActions,yynewGLRStackItem,yyaddDeferredAction,yyinitStateSet)
(yyinitGLRStack,yyexpandGLRStack,yyupdateSplit,yymarkStackDeleted)
(yyundeleteLastStack,yyglrShift,yyglrShiftDefer,yydoAction,yyglrReduce)
(yyidenticalOptions,yymergeOptionSets,yyresolveStates,yyresolveAction)
(yyresolveLocations,yyresolveValue,yyreducePrint): Update parameter
names in comments and mention all parameters.
(struct yyGLRState): Fix description of yyposn field.
(yyresolveLocations): Correct comment so as not to imply action when
yyn1==0.
* data/xslt/xml2dot.xsl (xsl:template match="bison-xml-report"):
Update output to include comments now produced by --graph.
(xsl:template match="automaton"): As for --graph, name the
digraph after the grammar file.
* src/print-xml.c (escape_bufs): Enlarge array.
(print_xml): Add bug-report and url attributes to
bison-xml-report element.
This handles the possibility that a "*/" might appear in
variable portions of those comments at some point in the future.
* src/graphviz.c (start_graph): Implement.
That is, keep them as warnings because that should be sufficient
to satisfy POSIX without creating backward compatibility issues.
Suggested by Richard Stallman at
<http://lists.gnu.org/archive/html/bison-patches/2010-03/msg00033.html>.
* NEWS (2.5): Remove mention of complaint.
* src/reader.c (grammar_rule_check): Convert warning back to
complaint.
* tests/input.at (%prec's token must be defined): Update.
Reported by Tys Lefering at
<http://lists.gnu.org/archive/html/bug-bison/2010-03/msg00035.html>.
This fix is already implemented in glr.c and does not apply to
lalr1.java.
* data/lalr1.cc (yy::parser::parse): Increase size of
yyerror_range and adjust subscripting so you don't have to
subtract one from the beginning of the array.
* data/yacc.c (b4_declare_parser_state_variables,
yyparse, yypush_parse): Likewise.
(cherry picked from commit 48f4100a82)
Conflicts:
data/lalr1.cc
data/yacc.c
src/parse-gram.c
src/parse-gram.h
M4's output pipe was not being drained upon fatal errors during
scan_skel. As a result, broken-pipe messages from M4 were seen
on at least AIX, HP-UX, Solaris, and RHEL4, and this caused a
failure in the test suite. The problem was that, on platforms
where the default disposition for SIGPIPE is ignore instead of
terminate, M4 sometimes saw fwrite fail with errno=EPIPE and
then reported it. However, there's some sort of race condition,
because the new test group occasionally succeeded.
Reported by Albert Chin at
<http://lists.gnu.org/archive/html/bug-bison/2010-02/msg00004.html>.
There were also problems with the test suite livelocking on
Tru64 5.1b. Reported by Didier Godefroy at
<http://lists.gnu.org/archive/html/bug-bison/2009-05/msg00005.html>.
Switching to create_pipe_bidi suggested by Akim Demaille.
To attempt to solve both of these problems, switch to gnulib's
create_pipe_bidi and register M4 process as a slave. Along the
way, clean up file name conflict handling, which was affected by
the broken-pipe problem before the switch.
* NEWS (2.4.2): Document.
* THANKS (Didier Godefroy): Add.
* bootstrap.conf (gnulib_modules): Add pipe.
* gnulib: Update to latest to make sure we have all the latest
fixes.
* lib/local.mk (lib_libbison_a_SOURCES): Remove subpipe.h and
subpipe.c.
* po/POTFILES.in (lib/subpipe.c): Remove.
* src/files.c (compute_output_file_names): Update invocations
of output_file_name_check.
(output_file_name_check): In the case that the grammar file
would be overwritten, use complain instead of fatal, but replace
the output file name with /dev/null. Use the /dev/null solution
for the case of two conflicting output files as well because it
seems safer in case Bison one day tries to open both files at
the same time.
* src/files.h (output_file_name_check): Update prototype.
* src/output.c (output_skeleton): Use create_pipe_bidi and
wait_subprocess. Assert that scan_skel completely drains the
pipe.
* src/scan-skel.l (at_directive_perform): Update
output_file_name_check invocation.
* tests/output.at (AT_CHECK_CONFLICTING_OUTPUT): Check that the
grammar file actually isn't overwritten.
(Conflicting output files: -o foo.y): Update expected output.
* tests/skeletons.at (Fatal errors but M4 continues producing
output): New test group.
(cherry picked from commit 22cc8d813e)
Conflicts:
NEWS
bootstrap.conf
lib/.cvsignore
lib/.gitignore
lib/Makefile.am
m4/.cvsignore
m4/.gitignore
src/output.c
Stop equating LR(0) with nondeterminism and LALR(1) with
determinism. That is, if all states are consistent, then LR(0)
tables are deterministic. On the other hand, LALR(1) tables
might be nondeterministic before conflict resolution, and GLR
permits LALR(1) tables to remain nondeterministic.
* src/LR0.c, src/LR0.h: Here.
* src/lalr.c, src/lalr.h: Here.
* src/main.c (main): Here.
* src/state.c, src/state.h: Here.
* src/ielr.h (ielr): In preconditions, expect LR(0) not LALR(1)
parser tables.
(cherry picked from commit 1c4ad777cb)
Reported by Florian Krohm at
<http://lists.gnu.org/archive/html/bug-bison/2009-12/msg00005.html>.
* NEWS (2.4.2): Document.
* src/reader.c (grammar_rule_check): Implement.
(grammar_current_rule_prec_set): Add comments explaining that we
here assume a %prec identifier is a token, but we still manage
to support POSIX.
* tests/input.at (%prec's token must be defined): New test
group.
(cherry picked from commit f1ceed0774)
Specifically, don't sort objects by their memory addresses when
they're not allocated in the same array or other object. Though
I haven't found a test case where that fails on my platform, C
says the behavior is undefined.
* src/AnnotationList.c (AnnotationList__insertInto): Remove
FIXME. Use new id field of InadequacyList nodes rather than
their memory addresses when sorting.
(AnnotationList__compute_from_inadequacies): Add
inadequacy_list_node_count argument to pass to
InadequacyList__new_conflict.
* src/AnnotationList.h
(AnnotationList__compute_from_inadequacies): Update prototype
and documentation for new argument.
* src/InadequacyList.c (InadequacyList__new_conflict): Add
node_count argument and use it to assign a unique ID.
* src/InadequacyList.h (InadequacyListNodeCount): New typedef.
(InadequacyList): Add id field.
(InadequacyList__new_conflict): Update prototype and
documentation for new argument.
* src/ielr.c (ielr_compute_annotation_lists): Update
AnnotationList__compute_from_inadequacies invocation.
(cherry picked from commit 2728ac7ecd)
The problem was that yacc.c didn't always update the yychar
translation afterwards. However, other skeletons appear to be
fine. glr.c appears to already translate yychar before every
use. lalr1.cc does not define yychar and does not document its
replacement, yyla, for users. It does provide yyclearin, but
that does not manipulate yyla and thus requires no translation
update. In lalr1.java, yychar is out of scope during semantic
actions.
* NEWS (2.5): Document.
* data/yacc.c (YYBACKUP): Don't bother translating yychar into
yytoken here.
(yyparse, yypush_parse): Instead, translate before every use of
yytoken, and add comments explaining this approach.
* tests/actions.at (Destroying lookahead assigned by semantic
action): New test group checking that translation happens before
lookahead destructor calls at parser return. Previously,
incorrect destructors were called.
* tests/conflicts.at (parse.error=verbose and consistent
errors): New test group checking that translation happens at
syntax error detection before the associated verbose error
message and the associated lookahead destructor calls. While
the destructor call is fixed by this patch, the verbose error
message is currently incorrect due to another bug (see
comments in test group), so this is an expected failure for now.
* NEWS (2.5): Document.
* data/lalr1.java (parser::YYStack::YYFAIL): Rename to YYERRLAB,
and make it private. Update all uses.
* src/scan-code.l (SC_RULE_ACTION): Implement warning.
* src/Sbitset.c (Sbitset__new_on_obstack): Use Sbitset instead
of char*.
(Sbitset__isEmpty): Use Sbitset instead of char*.
* src/Sbitset.h (Sbitset): Make it a pointer to unsigned char
instead of char. This helps to avoid casting errors.
(Sbitset__or): Use Sbitset instead of char*.
That is, use CHAR_BIT and UCHAR_MAX instead of 8 and 0xff.
* src/Sbitset.h (Sbitset__nbytes): Here.
(Sbitset__byteAddress): Here.
(Sbitset__bit_mask): Here.
(Sbitset__last_byte_mask): Here.
(Sbitset__ones): Here.
(SBITSET__FOR_EACH): Here.
* src/parse-gram.y: Clean up sorting of declarations.
Use types to simplify %printer declarations where possible.
Provide %printer for BRACKETED_ID and symbol.prec.
* src/symtab.c: Whitespace change.
Except when memory reallocation is required, this change
eliminates the need to invoke yysyntax_error twice and thus to
repeat the collection of lookaheads. It also prepares for
future extensions that will make those repetitions more
expensive and that will require additional memory management in
yysyntax_error. Finally, it fixes an obscure bug already
exercised in the test suite.
* data/yacc.c (yysyntax_error): Add arguments for message
buffer variables stored in the parser. Instead of size, return
status similar to yyparse status but indicating success of
message creation. Other than the actual reallocation of the
message buffer, import and clean up memory management code
from...
(yyparse, yypush_parse): ... here.
* tests/regression.at (parse.error=verbose overflow): No longer
an expected failure.
Reported by Jim Meyering.
* src/Sbitset.h (SBITSET__INDEX__CONVERSION_SPEC): New, "zu"
because Sbitset__Index is size_t.
* src/Sbitset.c (Sbitset__fprint): Use it instead of %d.
* src/location.h: (location_print): Add return value.
* src/location.c: (location_print): Return number of printed
characters.
* src/complain.h: Two new functions (complain_at_indent,
warn_at_indent).
* src/complain.cpp: Implement the alignment mechanism. Add new
static variable (indent_ptr). Use and update it (error_message,
complain_at_indent, warn_at_indent).
* src/scan-code.l: Fix strings for translations. Use new *_indent
functions (parse_ref, show_sub_messages).
* NEWS (2.5): Add an announcement about named references.
Add "symbol not found in production" error message when
an "invalid reference" is detected in named references
resolution.
* src/scan-code.l: Update "invalid reference" case.
* tests/named-ref.at: Adjust test-cases.
* data/yacc.c: Clean up M4 for readability, and make output
whitespace more consistent. For the main parse function
comment, instead of saying "yyparse or yypush_parse", say either
"yyparse" or "yypush_parse" depending on which it actually is.
