When reporting useless rules and nonterminals, say "useless in grammar"
instead of "useless", and say "useless in parser" instead of "never
reduced". Discussed starting at
<http://lists.gnu.org/archive/html/bison-patches/2007-10/msg00033.html>.
* NEWS (2.3a+): Mention this change.
* data/xslt/xml2text.xsl: Update output text and expected input XML
element names to match changes below.
* data/xslt/xml2xhtml.xsl: Likewise.
(xsl:template match="bison-xml-report"): Add missing entry in Table of
Contents: "Rules useless in parser due to conflicts".
* doc/bison.texinfo (Decl Summary): Reword a little.
(Understanding): Update example output for changes below.
* src/gram.c: (rule_useful_p): Rename to...
(rule_useful_in_grammar_p): ... this.
(rule_useless_p): Rename to...
(rule_useless_in_grammar_p): ... this.
(rule_never_reduced_p): Rename to...
(rule_useless_in_parser_p): ... this.
(grammar_rules_print): Update for renames.
(grammar_rules_print_xml): Update for renames.
(grammar_rules_never_reduced_report): Rename to...
(grammar_rules_useless_report): ... this since it is used for either
kind of useless rule.
* src/gram.h: Reword comments and update function names in prototypes.
* src/main.c (main): Say "rule useless in parser due to conflicts".
* src/print-xml.c (print_rules_never_reduced): Rename to...
(print_rules_useless_in_parser): ... this, and rename output XML
element "rules-never-reduced" to "rules-useless-in-parser".
(print_xml): Update for rename.
* src/print.c (print_results): Say "Rules useless in parser due to
conflicts".
* src/reduce.c (reduce_grammar_tables): Say "rule useless in grammar".
(nonterminals_reduce): Say "nonterminal useless in grammar".
(reduce_output): Say "Nonterminals useless in grammar".
Say "Rules useless in grammar".
(reduce_xml): Rename output XML element "useless" to
"useless-in-grammar".
(reduce_print): Don't report the count of grammatically useless rules
as "rules never reduced" just because %yacc is specified.
In the correct report of this count, say nonterminal(s) and rule(s)
"useless in grammar".
* tests/conflicts.at (S/R in initial): Update expected output.
(Defaulted Conflicted Reduction): Likewise.
(Unreachable States After Conflict Resolution): Likewise.
* tests/existing.at (GNU pic Grammar): Likewise.
* tests/reduce.at (Useless Nonterminals): Likewise.
(Useless Rules): Likewise.
(Reduced Automaton): Likewise.
(Underivable Rules): Likewise.
(Empty Language): Likewise.
(Option Cross Key): Likewise.
* src/print-xml.c (print_core): Don't print a reduction's lookahead set
next to an item whose dot is not at the end of the RHS even if it
happens to be associated with the same rule.
* src/print.c (print_core): Likewise.
* tests/conflicts.at (Unresolved SR Conflicts): Update output.
(Resolved SR Conflicts): Update output.
* tests/regression.at (Extra lookahead sets in report): New test case.
* data/xslt/xml2dot.xsl (xsl:template match="rule"): Use a ` '
instead of a `\n'. That is, don't add escapes yet or they'll be doubly
escaped later.
(xsl:template name="output-node"): Use the new escape template instead
of the string-replace template directly.
(xsl:template name="output-edge"): Likewise.
(xsl:template name="escape"): New, escapes backslashes and newlines in
addition to quotation marks.
* src/graphviz.c (start_graph, output_node, output_edge): Add
whitespace to output for legibility.
Make xml2text.xsl and --report produce the same output, and remove the
XML "conflicts" element since a conflict summary is easily extracted
from the automaton.
* data/xslt/bison.xsl: New.
(xsl:template match="state" mode="bison:count-conflicts): New.
* data/xslt/xml2text.xsl: Import bison.xsl.
(xsl:template match="bison-xml-report"): Instead of styling the
"conflicts" element, style the "automaton" element with mode
"conflicts". Unlike the former, the latter lists S/R and R/R
conflicts for a state on the same line.
(xsl:template match="conflicts"): Remove.
(xsl:template match="conflict"): Remove.
(xsl:template match="terminal"): Line-wrap the list of rules in which
the terminal is used.
(xsl:template match="nonterminal"): Likewise for nonterminals.
(xsl:template match="automaton" mode="conflicts"): New.
(xsl:template match="state" mode="conflicts"): New.
(xsl:template name="line-wrap"): New.
(xsl:template name="ws-search"): New.
* data/xslt/xml2xhtml.xsl: Import bison.xsl.
(xsl:template match="bison-xml-report"): Instead of styling the
"conflicts" element, style the "automaton" element with mode
"conflicts."
(xsl:template match="conflicts"): Remove.
(xsl:template match="conflict"): Remove.
(xsl:template match="automaton" mode="conflicts"): New.
(xsl:template match="state" mode="conflicts): New.
* src/conflicts.c (conflicts_output_xml): Remove.
* src/conflicts.h (conflicts_output_xml): Remove prototype.
* src/print-xml.c (print_xml): Don't invoke conflicts_output_xml.
* src/print.c (print_grammar): Consistently wrap at the 66th column so
the corresponding XSLT is easier. Also, never wrap between a word and
the comma that follows it.
conflicts involving multiple reductions.
* src/conflicts.c (resolve_sr_conflict): Don't assign the error action
set for a state here or Bison will abort if it is reassigned on a
later conflicted reduction in the same state.
Similarly, don't finalize and assign the solved conflicts report here
or it will be lost if it is reassigned on a later conflicted reduction
in the same state.
(set_conflicts): Instead, assign them both here after all S/R conflicts
in the state have been fully examined.
* src/print.c (shift_set): Rename to...
(no_reduce_set): ... this.
(print_reductions): Update for rename, and add %nonassoc error action
tokens to no_reduce_set so that, when printing the first remaining
reduction on an error action token, the reduction is enclosed in
brackets.
(print_results): Update for rename.
* tests/conflicts.at (Solved conflicts report for multiple reductions
in a state): New test case.
(%nonassoc error actions for multiple reductions in a state): New test
case.
* src/main.c (main): Don't depend on C99 features.
* src/LR0.c (new_itemsets): Use item_number_is_symbol_number.
(new_itemsets, save_reductions): Update for rename to nitemset.
* src/closure.c (nritemset): Rename to...
(nitemset): ... this since the "r" appears to meaningless and isn't
used in the comments.
(closure): Update for rename.
* src/closure.h (nritemset): Update extern to...
(nitemset): ... this.
* src/lalr.c (LA): Fix a typo in comments.
* src/print.c (print_core): Update for rename to nitemset.
* src/print_graph.c (print_graph): Likewise.
* src/state.h: Fix some typos in header comments.
(print_reductions): Replace state_default_rule invocation with
equivalent use of yydefact, which was computed in token_actions in
tables.c.
(print_results): Don't allocate lookahead_set.
`look_ahead'. Discussed starting at
<http://lists.gnu.org/archive/html/bison-patches/2006-01/msg00049.html>
and then at
<http://lists.gnu.org/archive/html/bison-patches/2006-06/msg00017.html>.
* NEWS: For the next release, note the change to `--report'.
* TODO, doc/bison.1: Update English.
* doc/bison.texinfo: Update English.
(Understanding Your Parser, Bison Options): Document as
`--report=lookahead' rather than `--report=look-ahead'.
* src/conflicts.c: Update English in comments.
(lookahead_set): Rename from look_ahead_set.
(flush_reduce): Rename argument look_ahead_tokens to lookahead_tokens.
(resolve_sr_conflict): Rename local look_ahead_tokens to
lookahead_tokens, and update other uses.
(flush_shift, set_conflicts, conflicts_solve, count_sr_conflicts,
count_rr_conflicts, conflicts_free): Update uses.
* src/getargs.c (report_args): Move "lookahead" before alternate
spellings.
(report_types): Update uses.
(usage): For `--report' usage description, state `lookahead' spelling
rather than `look-ahead'.
* src/getargs.h (report.report_lookahead_tokens): Rename from
report_look_ahead_tokens.
* src/lalr.c: Update English in comments.
(compute_lookahead_tokens): Rename from compute_look_ahead_tokens.
(state_lookahead_tokens_count): Rename from
state_look_ahead_tokens_count.
Rename local n_look_ahead_tokens to n_lookahead_tokens.
(lookahead_tokens_print): Rename from look_ahead_tokens_print.
Rename local n_look_ahead_tokens to n_lookahead_tokens.
Update other uses.
Update English in output.
(add_lookback_edge, initialize_LA, lalr, lalr_free): Update uses.
* src/print.c: Update English in comments.
(lookahead_set): Rename from look_ahead_set.
(print_reduction): Rename argument lookahead_token from
look_ahead_token.
(print_core, state_default_rule, print_reductions, print_results):
Update uses.
* src/print_graph.c: Update English in comments.
(print_core): Update uses.
* src/state.c: Update English in comments.
(reductions_new): Update uses.
(state_rule_lookahead_tokens_print): Rename from
state_rule_look_ahead_tokens_print, and update other uses.
* src/state.h: Update English in comments.
(reductions.lookahead_tokens): Rename from look_ahead_tokens.
(state_rule_lookahead_tokens_print): Rename from
state_rule_look_ahead_tokens_print.
* src/tables.c: Update English in comments.
(conflict_row, action_row): Update uses.
* tests/glr-regression.at
(Incorrect lookahead during deterministic GLR,
Incorrect lookahead during nondeterministic GLR): Rename
print_look_ahead to print_lookahead.
* tests/torture.at: Update English in comments.
(AT_DATA_LOOKAHEAD_TOKENS_GRAMMAR): Rename from
AT_DATA_LOOK_AHEAD_TOKENS_GRAMMAR.
(Many lookahead tokens): Update uses.
* data/glr.c: Update English in comments.
* lalr1.cc: Likewise.
* yacc.c: Likewise.
* src/conflicts.h: Likewise.
* src/lalr.h: Likewise.
* src/main.c: Likewise.
* src/output.c: Likewise.
* src/parse-gram.c: Likewise.
* src/tables.h: Likewise.
* tests/calc.at: Likewise.
reduced here, since anyway that computation doesn't work.
* src/gram.h, src/gram.h (rule_filter_t, rule_useful_p)
(rule_useless_p, rule_never_reduced_p): New.
(grammar_rules_partial_print): Use a filter instead of a range.
Display the title only if needed.
(grammar_rules_print): Adjust.
(grammar_rules_never_reduced_report): New.
* src/tables.c (action_row): Move the computation of rules never
reduced to...
(token_actions): here.
* src/main.c (main): Make the parser before making the report, so
that rules never reduced are computed.
Call grammar_rules_never_reduced_report.
* src/print.c (print_results): Report rules never reduced.
* tests/conflicts.at, tests/reduce.at: Adjust.
reduced by a state, attach the lookaheads to the reductions.
* src/state.h (state_t): Remove the `lookaheads',
`lookaheads_rule' member.
(reductions_t): Add a `lookaheads' member.
Use a regular array for the `rules'.
* src/state.c (reductions_new): Initialize the lookaheads member
to 0.
(state_rule_lookaheads_print): Adjust.
* src/state.h, src/state.c (state_reductions_find): New.
* src/conflicts.c (resolve_sr_conflict, set_conflicts)
(count_rr_conflicts): Adjust.
* src/lalr.c (LArule): Remove.
(add_lookback_edge): Adjust.
(state_lookaheads_count): New.
(states_lookaheads_initialize): Merge into...
(initialize_LA): this.
(lalr_free): Adjust.
* src/main.c (main): Don't free nullable and derives too early: it
is used by --verbose.
* src/print.c, src/print_graph.c, src/tables.c: Adjust.
by conflicts.
* src/LR0.c (save_reductions): Don't make the final state too
different: save its reduction (accept) instead of having a state
without any action (no shift or goto, no reduce).
Note: the final state is now a ``regular'' state, i.e., the
parsers now contain `reduce 0' as default reduction.
Nevertheless, since they decide to `accept' when yystate =
final_state, they still will not reduce rule 0.
* src/print.c (print_actions, print_reduction): Adjust.
* src/output.c (action_row): Track reduced rules.
(token_actions): Report rules never reduced.
* tests/conflicts.at, tests/regression.at: Adjust.
* src/LR0.c, src/closure.c, src/derives.c, src/gram.c, src/lalr.c
* src/nullable.c, src/output.c, src/print.c, src/reader.c
* src/reduce.c: Allocate and free from &rules[0], not &rules[1].
Iterate from 0 to nrules.
Use rule_number_as_item_number and item_number_as_rule_number.
Adjust to `derive' now containing possibly 0.
* src/gram.h (rule_number_as_item_number, item_number_as_rule_number):
Handle the `- 1' part in rule numbers from/to item numbers.
* src/conflicts.c (log_resolution): Fix the message which reversed
shift and reduce.
* src/output.c (action_row): Initialize default_rule to -1.
(token_actions): Adjust.
* tests/sets.at (Nullable, Firsts): Fix the previously bogus
expected output.
* tests/conflicts.at (Resolved SR Conflicts): Likewise.
* src/LR0.c (new_state): Merge into...
(state_list_append): this.
(new_states): Merge into...
(generate_states): here.
(set_states): Don't ensure a proper `errs' state member here, do it...
* src/conflicts.c (conflicts_solve): here.
* src/state.h, src/state.c: Comment changes.
(state_t): Rename member `shifts' as `transitions'.
Adjust all dependencies.
(errs_new): For consistency, also take the values as argument.
(errs_dup): Remove.
(state_errs_set): New.
(state_reductions_set, state_transitions_set): Assert that no
previous value was assigned.
(state_free): New.
(states_free): Use it.
* src/conflicts.c (resolve_sr_conflict): Don't use an `errs_t' as
temporary storage: use `errs' and `nerrs' as elsewhere.
(set_conflicts): Allocate and free this `errs'.
* src/gram.h, src/gram.c (grammar_rhs_print, grammar_rules_print):
New.
Don't print the rules' location, it is confusing and useless.
(rule_print): Use grammar_rhs_print.
* src/print.c (print_grammar): Use grammar_rules_print.
(input): Don't try to initialize yylloc here, do it in the
scanner.
* src/scan-gram.l (YY_USER_INIT): Initialize yylloc.
* src/gram.h (rule_t): Change line and action_line into location
and action_location, of location_t type.
Adjust all dependencies.
* src/location.h, src/location.c (empty_location): New.
* src/reader.h, src/reader.c (grammar_start_symbol_set)
(grammar_symbol_append, grammar_rule_begin, grammar_rule_end)
(grammar_current_rule_symbol_append)
(grammar_current_rule_action_append): Expect a location as argument.
* src/reader.c (grammar_midrule_action): Adjust to attach an
action's location as dummy symbol location.
* src/symtab.h, src/symtab.c (startsymbol_location): New.
* tests/regression.at (Web2c Report, Rule Line Numbers): Adjust
the line numbers.
(output_obstack): Remove it, and all its dependencies, as it is no
longer needed.
* src/reader.c (epilogue_set): Build the epilogue in the
muscle_obstack.
* src/output.h, src/output.c (muscle_obstack): Move to...
* src/muscle_tab.h, src/muscle_tab.h: here.
(muscle_init): Initialize muscle_obstack.
(muscle_free): New.
* src/main.c (main): Call it.
* src/LR0.c (new_state): Set it to 0.
* src/conflicts.h, src/conflicts.c (print_conflicts)
(free_conflicts, solve_conflicts): Rename as...
(conflicts_print, conflicts_free, conflicts_solve): these.
Adjust callers.
* src/conflicts.c (enum conflict_resolution_e)
(solved_conflicts_obstack): New, used by...
(log_resolution): this.
Adjust to attach the conflict resolution to each state.
Complete the description with the precedence/associativity
information.
(resolve_sr_conflict): Adjust.
* src/print.c (print_state): Output its solved_conflicts.
* tests/conflicts.at (Unresolved SR Conflicts)
(Solved SR Conflicts): Exercise --report=all.
(Tracing): this new section, its former contents, and...
(Understanding): this new section.
* src/getargs.h, src/getargs.c (verbose_flag): Remove, replaced
by...
(report_flag): this.
Adjust all dependencies.
(report_args, report_types, report_argmatch): New.
(usage, getargs): Report/support -r, --report.
* src/options.h
(struct option_table_struct): Rename as..,
(struct option_table_s): this.
Rename the `set_flag' member to `flag' to match with getopt_long's
struct.
* src/options.c (option_table): Split verbose into an entry for
%verbose, and another for --verbose.
Support --report/-r, so remove -r from the obsolete --raw.
* src/print.c: Attach full item sets and lookaheads reports to
report_flag instead of trace_flag.
* lib/argmatch.h, lib/argmatch.c: New, from Fileutils 4.1.
errtoken->number.
* src/reader.c (reader): Don't specify the user token number (2)
for $undefined, as it uselessly prevents using it.
* src/gram.h (token_number_t): Move to...
* src/symtab.h: here.
(state_t.number): Is a token_number_t.
* src/print.c, src/reader.c: Use undeftoken->number instead of
hard coded 2.
(Even though this 2 is not the same as above: the number of the
undeftoken remains being 2, it is its user token number which
might not be 2).
* src/output.c (prepare_tokens): Rename the `maxtok' muscle with
`user_token_number_max'.
Output `undef_token_number'.
* data/bison.simple, data/bison.c++: Use them.
Be sure to map invalid yylex return values to
`undef_token_number'. This saves us from gratuitous SEGV.
* tests/conflicts.at (Solved SR Conflicts)
(Unresolved SR Conflicts): Adjust.
* tests/regression.at (Web2c Actions): Adjust.
RITEM. Note that it must be able to code symbol numbers as
positive number, and the negation of rule numbers as negative
numbers.
Adjust all dependencies (pretty many).
* src/reduce.c (rule): Remove this `short *' pointer: use
item_number_t.
* src/system.h (MINSHORT, MAXSHORT): Remove.
Include `limits.h'.
Adjust dependencies to using SHRT_MAX and SHRT_MIN.
(shortcpy): Remove.
(MAXTABLE): Move to...
* src/output.c (MAXTABLE): here.
(prepare_rules): Use output_int_table to output rhs.
* data/bison.simple, data/bison.c++: Adjust.
* tests/torture.at (Big triangle): Move the limit from 254 to
500.
* tests/regression.at (Web2c Actions): Ajust.
Trying with bigger grammars shows various phenomena: at 3000 (28Mb
of grammar file) bison is killed by my system, at 2000 (12Mb) bison
passes, but produces negative #line number, once fixed, GCC is
killed while compiling 14Mb, at 1500 (6.7 Mb of grammar, 8.2Mb of
C), it passes.
* src/state.h (state_h): Code input lines on ints, not shorts.
