useful ones and add a "usefulness" attribute. Discussed starting at
<http://lists.gnu.org/archive/html/bison-patches/2007-09/msg00017.html>.
* src/gram.c (grammar_rules_partial_print_xml): Remove.
(grammar_rules_print_xml): Print all rules instead of just those
useful in the grammar, and add a "usefulness" attribute.
* src/gram.h (grammar_rules_partial_print_xml): Remove prototype.
* src/print-xml.c (print_rules_useless_in_parser): Remove.
(print_grammar): Print all nonterminals instead of just useful ones,
and add a "usefulness" attribute to nonterminals and terminals.
(print_xml): Don't print a separate "reductions" or
"rules-useless-in-parser" element.
* src/reduce.c (reduce_output): Use reduce_token_unused_in_grammar.
(reduce_xml): Remove.
(reduce_token_unused_in_grammar): New.
(reduce_nonterminal_useless_in_grammar): New.
* src/reduce.h (reduce_xml): Remove prototype.
(reduce_token_unused_in_grammar): Add prototype.
(reduce_nonterminal_useless_in_grammar): Add prototype.
* data/xslt/xml2text.xsl: Update for XML changes.
* data/xslt/xml2xhtml.xsl: Update for XML changes.
* tests/reduce.at (Useless Terminals): Update output.
(Useless Rules): Update output.
(Reduced Automaton): Update output.
Say "Terminals unused in grammar" instead of "Unused terminals".
* NEWS (2.3a+): Update.
* doc/bison.texinfo (Understanding): Update example output.
* src/reduce.c (reduce_output): Implement.
* data/xslt/xml2text.xsl: Implement.
* data/xslt/xml2xhtml.xsl: Implement.
When reporting useless rules and nonterminals, say "useless in grammar"
instead of "useless", and say "useless in parser" instead of "never
reduced". Discussed starting at
<http://lists.gnu.org/archive/html/bison-patches/2007-10/msg00033.html>.
* NEWS (2.3a+): Mention this change.
* data/xslt/xml2text.xsl: Update output text and expected input XML
element names to match changes below.
* data/xslt/xml2xhtml.xsl: Likewise.
(xsl:template match="bison-xml-report"): Add missing entry in Table of
Contents: "Rules useless in parser due to conflicts".
* doc/bison.texinfo (Decl Summary): Reword a little.
(Understanding): Update example output for changes below.
* src/gram.c: (rule_useful_p): Rename to...
(rule_useful_in_grammar_p): ... this.
(rule_useless_p): Rename to...
(rule_useless_in_grammar_p): ... this.
(rule_never_reduced_p): Rename to...
(rule_useless_in_parser_p): ... this.
(grammar_rules_print): Update for renames.
(grammar_rules_print_xml): Update for renames.
(grammar_rules_never_reduced_report): Rename to...
(grammar_rules_useless_report): ... this since it is used for either
kind of useless rule.
* src/gram.h: Reword comments and update function names in prototypes.
* src/main.c (main): Say "rule useless in parser due to conflicts".
* src/print-xml.c (print_rules_never_reduced): Rename to...
(print_rules_useless_in_parser): ... this, and rename output XML
element "rules-never-reduced" to "rules-useless-in-parser".
(print_xml): Update for rename.
* src/print.c (print_results): Say "Rules useless in parser due to
conflicts".
* src/reduce.c (reduce_grammar_tables): Say "rule useless in grammar".
(nonterminals_reduce): Say "nonterminal useless in grammar".
(reduce_output): Say "Nonterminals useless in grammar".
Say "Rules useless in grammar".
(reduce_xml): Rename output XML element "useless" to
"useless-in-grammar".
(reduce_print): Don't report the count of grammatically useless rules
as "rules never reduced" just because %yacc is specified.
In the correct report of this count, say nonterminal(s) and rule(s)
"useless in grammar".
* tests/conflicts.at (S/R in initial): Update expected output.
(Defaulted Conflicted Reduction): Likewise.
(Unreachable States After Conflict Resolution): Likewise.
* tests/existing.at (GNU pic Grammar): Likewise.
* tests/reduce.at (Useless Nonterminals): Likewise.
(Useless Rules): Likewise.
(Reduced Automaton): Likewise.
(Underivable Rules): Likewise.
(Empty Language): Likewise.
elsewhere.
* src/print-xml.c: Prefer "const" after types; that's more consistent.
(xml_printf): Indent just 1 space for level.
(e_char, xlate_char): Remove.
(xml_escape_string): Rewrite to avoid undefined behavior (used
storage that was freed from the stack).
(xml_escape_n): Don't bother checking for subscript error.
2007-09-21 Wojciech Polak <polak@gnu.org>
Add support for an -x option to generate an XML report.
It is not documented yet.
* src/print-xml.c: New file.
* src/print-xml.h: Likewise.
* lib/timevar.def (TV_XML): New var.
* src/Makefile.am (bison_SOURCES): Add print-xml.c, print-xml.h.
* src/conflicts.c: Include print-xml.h.
(solved_conflicts_xml_obstack): New var.
(log_resolution, conflicts_solve, conflicts_free):
Add support for XML report.
(conflicts_output_val): New function.
* src/conflicts.h (conflicts_output_val): New decl.
* src/files.c (spec_xml_file): New var.
(compute_output_file_names, output_file_names_free): Add XML support.
* src/files.h (spec_xml_file): New decl.
* src/getargs.c (xml_flag): New var.
(usage, short_options, long_options, getargs): Add XML support.
* src/getargs.h (xml_flag): New decl.
* src/gram.c: Include print-xml.h.
(rule_lhs_print_xml, rule_rhs_print_xml):
(grammar_rules_partial_print_xml, grammar_rules_print_xml):
New functions.
* src/gram.h: Declare external ones.
* src/main.c: Include print-xml.h.
(main): Add XML support.
* src/reduce.c: Include print-xml.h.
(reduce_xml): New function.
* src/reduce.h: Declare it.
* src/state.c: Include print-xml.h.
(state_new): Add XML support.
(state_rule_lookahead_tokens_print_xml): New function.
* src/state.h: Declare it.
(struct state): New member solved_conflicts_xml.
* src/symtab.c (symbol_class_get_string): New function.
* src/symtab.h: Declare it.
2007-09-21 Paul Eggert <eggert@cs.ucla.edu>
As a consequence, the relation between the grammar scanner and
parser is much simpler. We can also split "composite tokens" back
into simple tokens.
* src/gram.h (ITEM_NUMBER_MAX, RULE_NUMBER_MAX): New.
* src/scan-gram.l (add_column_width, adjust_location): Move to and
rename as...
* src/location.h, src/location.c (add_column_width)
(location_compute): these.
Fix the column count: the initial column is 0.
(location_print): Be robust to ending column being 0.
* src/location.h (boundary_set): New.
* src/main.c: Adjust to scanner_free being renamed as
gram_scanner_free.
* src/output.c: Include scan-code.h.
* src/parse-gram.y: Include scan-gram.h and scan-code.h.
Use boundary_set.
(PERCENT_DESTRUCTOR, PERCENT_PRINTER, PERCENT_INITIAL_ACTION)
(PERCENT_LEX_PARAM, PERCENT_PARSE_PARAM): Remove the {...} part,
which is now, again, a separate token.
Adjust all dependencies.
Whereever actions with $ and @ are used, use translate_code.
(action): Remove this nonterminal which is now useless.
* src/reader.c: Include assert.h, scan-gram.h and scan-code.h.
(grammar_current_rule_action_append): Use translate_code.
(packgram): Bound check ruleno, itemno, and rule_length.
* src/reader.h (gram_in, gram__flex_debug, scanner_cursor)
(last_string, last_braced_code_loc, max_left_semantic_context)
(scanner_initialize, scanner_free, scanner_last_string_free)
(gram_out, gram_lineno, YY_DECL_): Move to...
* src/scan-gram.h: this new file.
(YY_DECL): Rename as...
(GRAM_DECL): this.
* src/scan-code.h, src/scan-code.l, src/scan-code-c.c: New.
* src/scan-gram.l (gram_get_lineno, gram_get_in, gram_get_out):
(gram_get_leng, gram_get_text, gram_set_lineno, gram_set_in):
(gram_set_out, gram_get_debug, gram_set_debug, gram_lex_destroy):
Move these declarations, and...
(obstack_for_string, STRING_GROW, STRING_FINISH, STRING_FREE):
these to...
* src/flex-scanner.h: this new file.
* src/scan-gram.l (rule_length, rule_length_overflow)
(increment_rule_length): Remove.
(last_braced_code_loc): Rename as...
(gram_last_braced_code_loc): this.
Adjust to the changes of the parser.
Move all the handling of $ and @ into...
* src/scan-code.l: here.
* src/scan-gram.l (handle_dollar, handle_at): Remove.
(handle_action_dollar, handle_action_at): Move to...
* src/scan-code.l: here.
* src/Makefile.am (bison_SOURCES): Add flex-scanner.h,
scan-code.h, scan-code-c.c, scan-gram.h.
(EXTRA_bison_SOURCES): Add scan-code.l.
(BUILT_SOURCES): Add scan-code.c.
(yacc): Be robust to white spaces.
* tests/conflicts.at, tests/input.at, tests/reduce.at,
* tests/regression.at: Adjust the column numbers.
* tests/regression.at: Adjust the error message.
Reported by Alexandre Duret-Lutz.
* src/LR0.c (state_list_append): Remove the computation of
final_state.
(save_reductions): Do it here.
(get_state): Alpha conversion.
(generate_states): Use a for loop.
* src/gram.h (item_number_is_rule_number)
(item_number_is_symbol_number): New.
* src/state.c: Use assert.
* src/system.h: Include assert.h.
* tests/sets.at (Accept): New.
item_number_as_symbol_number, rule_number_as_item_number,
item_number_as_rule_number): Now inline functions rather
than macros, to avoid casts.
(int_of_rule_number): Remove; no longer used.
(rule_number): Renamed from rule_number_t.
(struct rule_s): Remove the "rule_s" part; not used.
(rule): Renamed from rule_t.
(rule_filter): Renamed from rule_filter_t.
(ITEM_NUMBER_MAX, ITEM_NUMBER_MIN, RULE_NUMBER_MAX):
Remove; unused.
reduced here, since anyway that computation doesn't work.
* src/gram.h, src/gram.h (rule_filter_t, rule_useful_p)
(rule_useless_p, rule_never_reduced_p): New.
(grammar_rules_partial_print): Use a filter instead of a range.
Display the title only if needed.
(grammar_rules_print): Adjust.
(grammar_rules_never_reduced_report): New.
* src/tables.c (action_row): Move the computation of rules never
reduced to...
(token_actions): here.
* src/main.c (main): Make the parser before making the report, so
that rules never reduced are computed.
Call grammar_rules_never_reduced_report.
* src/print.c (print_results): Report rules never reduced.
* tests/conflicts.at, tests/reduce.at: Adjust.
* src/symtab.h, src/symtab.c (eoftoken, axiom): Rename as...
(endtoken, accept): these.
* src/reader.c (reader): Set endtoken's default tag to "$end".
Set undeftoken's tag to "$undefined" instead of "$undefined.".
* doc/bison.texinfo (Table of Symbols): Mention $accept and $end.
Adjust.
* src/LR0.c, src/closure.c, src/derives.c, src/gram.c, src/lalr.c
* src/nullable.c, src/output.c, src/print.c, src/reader.c
* src/reduce.c: Allocate and free from &rules[0], not &rules[1].
Iterate from 0 to nrules.
Use rule_number_as_item_number and item_number_as_rule_number.
Adjust to `derive' now containing possibly 0.
* src/gram.h (rule_number_as_item_number, item_number_as_rule_number):
Handle the `- 1' part in rule numbers from/to item numbers.
* src/conflicts.c (log_resolution): Fix the message which reversed
shift and reduce.
* src/output.c (action_row): Initialize default_rule to -1.
(token_actions): Adjust.
* tests/sets.at (Nullable, Firsts): Fix the previously bogus
expected output.
* tests/conflicts.at (Resolved SR Conflicts): Likewise.
* src/gram.h, src/gram.c (grammar_rhs_print, grammar_rules_print):
New.
Don't print the rules' location, it is confusing and useless.
(rule_print): Use grammar_rhs_print.
* src/print.c (print_grammar): Use grammar_rules_print.
(input): Don't try to initialize yylloc here, do it in the
scanner.
* src/scan-gram.l (YY_USER_INIT): Initialize yylloc.
* src/gram.h (rule_t): Change line and action_line into location
and action_location, of location_t type.
Adjust all dependencies.
* src/location.h, src/location.c (empty_location): New.
* src/reader.h, src/reader.c (grammar_start_symbol_set)
(grammar_symbol_append, grammar_rule_begin, grammar_rule_end)
(grammar_current_rule_symbol_append)
(grammar_current_rule_action_append): Expect a location as argument.
* src/reader.c (grammar_midrule_action): Adjust to attach an
action's location as dummy symbol location.
* src/symtab.h, src/symtab.c (startsymbol_location): New.
* tests/regression.at (Web2c Report, Rule Line Numbers): Adjust
the line numbers.
* src/reduce.c (nonterminals_reduce): Don't loop over RITEM: loop
over the RHS of each rule.
* src/gram.h, src/gram.c (nritems): Is `unsigned int', not int.
* src/state.h (state_t): Member `nitems' is unsigned short.
* src/LR0.c (get_state): Adjust.
* src/reader.c (packgram): Likewise.
* src/output.c (GENERATE_MUSCLE_INSERT_TABLE): `max' is of type
`Type'.
(muscle_insert_int_table): Remove, unused.
(prepare_rules): Remove `max'.
errtoken->number.
* src/reader.c (reader): Don't specify the user token number (2)
for $undefined, as it uselessly prevents using it.
* src/gram.h (token_number_t): Move to...
* src/symtab.h: here.
(state_t.number): Is a token_number_t.
* src/print.c, src/reader.c: Use undeftoken->number instead of
hard coded 2.
(Even though this 2 is not the same as above: the number of the
undeftoken remains being 2, it is its user token number which
might not be 2).
* src/output.c (prepare_tokens): Rename the `maxtok' muscle with
`user_token_number_max'.
Output `undef_token_number'.
* data/bison.simple, data/bison.c++: Use them.
Be sure to map invalid yylex return values to
`undef_token_number'. This saves us from gratuitous SEGV.
* tests/conflicts.at (Solved SR Conflicts)
(Unresolved SR Conflicts): Adjust.
* tests/regression.at (Web2c Actions): Adjust.
RITEM. Note that it must be able to code symbol numbers as
positive number, and the negation of rule numbers as negative
numbers.
Adjust all dependencies (pretty many).
* src/reduce.c (rule): Remove this `short *' pointer: use
item_number_t.
* src/system.h (MINSHORT, MAXSHORT): Remove.
Include `limits.h'.
Adjust dependencies to using SHRT_MAX and SHRT_MIN.
(shortcpy): Remove.
(MAXTABLE): Move to...
* src/output.c (MAXTABLE): here.
(prepare_rules): Use output_int_table to output rhs.
* data/bison.simple, data/bison.c++: Adjust.
* tests/torture.at (Big triangle): Move the limit from 254 to
500.
* tests/regression.at (Web2c Actions): Ajust.
Trying with bigger grammars shows various phenomena: at 3000 (28Mb
of grammar file) bison is killed by my system, at 2000 (12Mb) bison
passes, but produces negative #line number, once fixed, GCC is
killed while compiling 14Mb, at 1500 (6.7 Mb of grammar, 8.2Mb of
C), it passes.
* src/state.h (state_h): Code input lines on ints, not shorts.
* src/output.c (output_table_data): Return the longest number.
(prepare_tokens): Output `token_number_max').
* data/bison.simple, data/bison.c++ (b4_sint_type, b4_uint_type):
New.
Use them to define yy_token_number_type/TokenNumberType.
Use this type for yytranslate.
* tests/torture.at (Big triangle): Push the limit from 124 to
253.
* tests/regression.at (Web2c Actions): Adjust.
bucket.
Adjust all dependencies.
* src/reduce.c (nonterminals_reduce): Don't forget to renumber the
`number' of the buckets too.
* src/gram.h: Include `symtab.h'.
(associativity): Move to...
* src/symtab.h: here.
No longer include `gram.h'.
(ritem_longest_rhs): Use it.
* src/gram.h (rule_t): `number' is a new member.
* src/reader.c (packgram): Set it.
* src/reduce.c (reduce_grammar_tables): Move the useless rules at
the end of `rules', and count them out of `nrules'.
(reduce_output, dump_grammar): Adjust.
* src/print.c (print_grammar): It is no longer needed to check for
the usefulness of a rule, as useless rules are beyond `nrules + 1'.
* tests/reduce.at (Reduced Automaton): New test.
* src/nullable.c, src/output.c, src/print.c, src/print_graph.c,
* src/reader.c, src/reduce.c: Let rule_t.rhs point directly to the
RHS, instead of being an index in RITEMS.
and rule_table.
* src/reader.c (packsymbols): Fill this table.
Drop sprec.
* src/conflicts.c (resolve_sr_conflict): Adjust.
* src/reduce.c (reduce_grammar): Adjust: just sort symbols, a
single table.
Use symbols[i]->tag instead of tags[i].
is 0 which was also used as a sentinel in ritem, (i) make sure >= 0
is used instead of > 0 where appropriate, (ii), depend upon nritems
instead of the 0-sentinel.
* src/gram.h, src/gram.c (nritems): New.
Expected to be duplication of nitems, but for the time being...
* src/reader.c (packgram): Assert nritems and nitems are equal.
* src/LR0.c (allocate_itemsets, new_itemsets): Adjust.
* src/closure.c (print_closure, print_fderives): Likewise.
* src/gram.c (ritem_print): Likewise.
* src/print.c (print_core, print_grammar): Likewise.
* src/print_graph.c: Likewise.
* src/reader.c (symbol_list): `guard' and `guard_line' are new
members.
(symbol_list_new): Adjust.
(copy_action): action_line is the first line, not the last.
(copy_guard): Just as for actions, store the `action' only, not
the switch/case/break flesh.
Don't parse the user action that might follow the guard, let...
(readgram): do it, i.e., now, there can be an action after a
guard.
In other words the guard is just explicitly optional.
(packgram): Adjust.
* src/output.c (guards_output): New.
(output_parser): Call it when needed.
(output): Also free the guard and attrs obstacks.
* src/files.c, src/files.h (obstack_save): Remove.
(output_files): Remove.
As a result, if one needs the former `.act' file, using an
appropriate skeleton which requires actions and guards is now
required.
* src/main.c (main): Adjust.
* tests/semantic.at: New.
* tests/regression.at: Use `input.y' as input file name.
Avoid 8+3 problems by requiring input.c when the test needs the
parser.
actions_obstack.
* src/gram.h (rule_t): action and action_line are new members.
* src/reader.c (symbol_list): Likewise.
(copy_action): Save the actions within the rule.
(packgram): Save them in rule_table.
* src/output.c (actions_output): New.
(output_parser): Use it on `%%actions'.
(output_rule_data): Don't free rule_table.
(output): Do it.
(prepare): Don't save the `action' muscle.
* src/bison.simple: s/%%action/%%actions/.
