These changes allow users to prefix an action with %? to indicate that it
is a semantic predicate---an expression that is evaluated immediately (not
deferred, even in GLR nondeterministic mode) and causes a syntax error if
false. In GLR parsers, this has the effect of killing one of a set of
split-off parses, just as would an ordinary syntax error.
Changelog:
* NEWS: Describe new semantic-predicate feature.
* data/c.m4 (b4_predicate_case): New definition.
* data/java.m4 (b4_predicate_case): New definition.
* data/glr.c (yyimmediate): Add definition.
(yydoAction): Remove comment, now obsolete.
Do YY_REDUCE_PRINT here.
(yyglrReduce): Alter comment to indicate that semantic values
need not be deferred.
Remove YY_REDUCE_PRINT from here; done in yydoAction.
(yyprocessOneStack): Pass immediate flag.
Delete stacks rejected by predicates in newly split-off parsers.
Change handling of yyerr so that only current stack gets deleted
when semantic predicate fails.
(yyfillin): Don't crash if a semantic value is unresolved (as may
happen in predicate rules).
Copy lr state as well in debugging mode.
Update comment on setting of yysval to include yyloc as well.
(yy_reduce_print): Add yynormal argument. Perform fillin properly.
Report unresolved RHS values.
(yyimmediate): New table.
* src/gram.h (struct rule): Add is_predicate field.
* src/output.c (user_actions_output): Use b4_predicate_case for
predicates.
(prepare_symbols): Output yyimmediate.
* src/scan-gram.l: Add %? token, SC_PREDICATE state.
* src/scan-code.l (code_props_rule_action_init): Add is_predicate
argument.
* src/scan-code.h (struct code_props): Add is_predicate field.
(code_props_rule_action_init): New interface.
* src/parse-gram.y (%?{...}): New token.
(rhs): Add %?{...} rule.
* src/parse-gram.c: Regenerate.
* src/parse-gram.h: Regenerate.
* src/reader.c (grammar_current_rule_action_append): Add
immediate argument.
(grammar_midrule_action): Use new interface for
code_props_rule_action_init.
(grammar_current_rule_action_append): Ditto.
(packgram): Transfer is_predicate value.
* src/reader.h (grammar_current_rule_action_append): New interface.
* doc/bison.texinfo: Document semantic predicates (%?).
* data/glr.c (yylhsNonterm, yyisDefaultedState,yyDefaultAction)
(yygetLRActions,yynewGLRStackItem,yyaddDeferredAction,yyinitStateSet)
(yyinitGLRStack,yyexpandGLRStack,yyupdateSplit,yymarkStackDeleted)
(yyundeleteLastStack,yyglrShift,yyglrShiftDefer,yydoAction,yyglrReduce)
(yyidenticalOptions,yymergeOptionSets,yyresolveStates,yyresolveAction)
(yyresolveLocations,yyresolveValue,yyreducePrint): Update parameter
names in comments and mention all parameters.
(struct yyGLRState): Fix description of yyposn field.
(yyresolveLocations): Correct comment so as not to imply action when
yyn1==0.
That is, keep them as warnings because that should be sufficient
to satisfy POSIX without creating backward compatibility issues.
Suggested by Richard Stallman at
<http://lists.gnu.org/archive/html/bison-patches/2010-03/msg00033.html>.
* NEWS (2.5): Remove mention of complaint.
* src/reader.c (grammar_rule_check): Convert warning back to
complaint.
* tests/input.at (%prec's token must be defined): Update.
Reported by Florian Krohm at
<http://lists.gnu.org/archive/html/bug-bison/2009-12/msg00005.html>.
* NEWS (2.4.2): Document.
* src/reader.c (grammar_rule_check): Implement.
(grammar_current_rule_prec_set): Add comments explaining that we
here assume a %prec identifier is a token, but we still manage
to support POSIX.
* tests/input.at (%prec's token must be defined): New test
group.
(cherry picked from commit f1ceed0774)
Continues Akim's work from his 2009-06-10 commits.
* src/reader.c (check_and_convert_grammar): Don't add any
symbols after the first symbols_do invocation.
* src/symtab.c (symbols_sorted): New static global.
(user_token_number_redeclaration): Update comments.
(symbol_from_uniqstr): If a new symbol is being created, assert
that symbols_sorted hasn't been allocated yet.
(symbols_free): Free symbols_sorted.
(symbols_cmp, symbols_cmp_qsort): New functions.
(symbols_do): Sort symbol_table into symbols_sorted on first
invocation.
* tests/input.at (Numbered tokens): Recombine tests now that the
output should be deterministic across multiple numbers.
* src/reader.c (reader): Move %define front-end variable
defaults and checking into...
(prepare_percent_define_front_end_variables): ... this new
function.
* src/scan-gram.l (INITIAL): For consistency with string
literals, don't store open quote on character literal. It's
discarded before returning anyway.
(SC_ESCAPED_CHARACTER): Similarly, don't store close quote.
Make length test more readable, and make the character stored
for an empty literal more obvious while consistent with the
previous behavior.
* src/symtab.c, src/symtab.h: Rename USER_NUMBER_ALIAS to
USER_NUMBER_HAS_STRING_ALIAS throughout.
* src/symtab.c (symbol_make_alias): Remove comment from symtab.c
that is repeated in symtab.h. Improve argument names to make it
clear which side of the symbol-string alias pair is which.
(symbol_check_alias_consistency): Improve local variable names
for the same purpose.
* src/symtab.h (struct symbol): Make comments about aliases
clearer.
(symbol_make_alias): Improve comments and argument name.
* src/output.c (token_definitions_output): Update for rename to
USER_NUMBER_HAS_STRING_ALIAS and improve comments about aliases.
* src/named-ref.h: Add comments.
* src/parse-gram.y: Readability and style changes.
* src/reader.c: Factoring: assign_named_ref function.
* src/scan-code.l: Factoring and style changes. Rename
parse_named_ref to parse_ref. Use "c-ctype.h" from gnulib.
Use "unsigned" type for variant index. Improve readablity.
* src/scan-gram.l: Change error messages and add comments.
* src/symlist.h: symbol_list_null: New function decl.
* src/symlist.c: symbol_list_null: Implement here.
* tests/named-refs.at: Adjust for new error messages.
Discussed in:
http://lists.gnu.org/archive/html/bison-patches/2009-01/msg00000.htmlhttp://lists.gnu.org/archive/html/bison-patches/2009-02/msg00002.htmlhttp://lists.gnu.org/archive/html/bison-patches/2009-03/msg00009.html
* src/parse-gram.y: Add new syntax (named_ref.opt).
* src/reader.c: Store named refs in symbol lists.
* src/reader.h: New argument for symbol_append and
action_append functions.
* src/scan-code.h: Add new field (named_ref) into
code_props data structure. Keeps named ref of midrule
actions.
* src/scan-code.l: Support for named refs in semantic
action code. New function 'parse_named_ref'.
* src/scan-gram.l: Support bracketed id.
* src/symlist.c: Store named refs in symbol lists.
* src/symlist.h: New field in symbol list: named_ref.
* src/named-ref.h: New file, a struct for named_ref.
* src/named-ref.cp: New file, named_ref_new function.
* src/local.mk: Add two new files.
* tests/testsuite.at: Include new test group:
* tests/named-refs.at: this new file.
For now, just api.push-pull and lr.keep-unreachable-states.
Maintain old names for backward compatibility.
* NEWS (2.5): Document.
* data/c.m4 (b4_identification): Update comment.
* data/yacc.c: Update access.
* doc/bison.texinfo: Update.
* etc/bench.pl.in (bench_push_parser): Update use.
* src/files.c (tr): Move to...
* src/getargs.c, src/getargs.h (tr): ... here because I can't
think of a better place to expose it. My logic is that, for all
uses of tr so far, command-line arguments can be involved, and
getargs.h is already included.
* src/main.c (main): Update access.
* src/muscle_tab.c (muscle_percent_define_insert): Convert old
variable names to new variable names before assigning value.
* src/reader.c (reader): Update setting default.
* tests/calc.at: Update uses.
* tests/conflicts.at (Unreachable States After Conflict
Resolution): Update use.
* tests/input.at (%define enum variables): Update use.
(%define backward compatibility): New test group.
* tests/push.at: Update uses.
* tests/reduce.at: Update uses.
* tests/torture.at: Update uses.
(cherry picked from commit 812775a039)
Conflicts:
data/c.m4
etc/bench.pl.in
src/parse-gram.c
src/parse-gram.h
tests/conflicts.at
Its value can be "LALR", "IELR", or "canonical LR".
* lib/timevar.def (TV_IELR_PHASE1): New var.
(TV_IELR_PHASE2): New var.
(TV_IELR_PHASE3): New var.
(TV_IELR_PHASE4): New var.
* src/local.mk (src_bison_SOURCES): Add AnnotationList.c,
AnnotationList.h, InadequacyList.c, InadequacyList.h, Sbitset.c,
Sbitset.h, ielr.c, and ielr.h.
* src/getargs.h, src/getargs.c (enum trace, trace_args,
trace_types): Add trace_ielr.
* src/lalr.h, src/lalr.c (ngotos): Export it.
(F): Rename to...
(goto_follows): ... this, update all uses, and export it.
(set_goto_map): Export it.
(map_goto): Export it.
(compute_lookahead_tokens): Don't free goto_follows yet. Now
handled in ielr.
(initialize_LA): Export it. Move lookback allocation to...
(lalr): ... here because, for canonical LR, initialize_LA must
be invoked but lookback and much of the rest of LALR isn't
needed.
* main.c (main): Instead of lalr, invoke ielr, which invokes
lalr.
* src/reader.c (reader): Default lr.type to "LALR".
Default lr.default_rules to "accepting" if lr.type is "canonical
LR". Leave the default as "all" otherwise.
Check for a valid lr.type value.
* src/state.h, src/state.c (struct state_list): Add state_list
member.
(state_new): Initialize state_list member to NULL.
(state_new_isocore): New function, exported.
* tests/existing.at (AT_TEST_EXISTING_GRAMMAR): New macro that
exercises all values of lr.type.
(GNU AWK Grammar): Rename test group to...
(GNU AWK 3.1.0 Grammar): ... this, and extend to use
AT_TEST_EXISTING_GRAMMAR.
(GNU Cim Grammar): Extend to use AT_TEST_EXISTING_GRAMMAR.
(GNU pic Grammar): Rename test group to...
(GNU pic (Groff 1.18.1) Grammar): ... this, and extend to use
AT_TEST_EXISTING_GRAMMAR.
* tests/reduce.at (AT_TEST_LR_TYPE): New macro that exercises
all values of lr.type.
(Single State Split): New test groups using AT_TEST_LR_TYPE.
(Lane Split): Likewise.
(Complex Lane Split): Likewise.
(Split During Added Lookahead Propagation): Likewise.
Its value describes the states that are permitted to contain
default rules: "all", "consistent", or "accepting".
* src/reader.c (reader): Default lr.default_rules to "all".
Check for a valid lr.default_rules value.
* src/lalr.c (state_lookahead_tokens_count): If lr.default_rules
is "accepting", then only mark the accepting state as
consistent.
(initialize_LA): Tell state_lookahead_tokens_count whether
lr.default_rules is "accepting".
* src/tables.c (action_row): If lr.default_rules is not "all",
then disable default rules in inconsistent states.
* src/print.c (print_reductions): Use this opportunity to
perform some assertions about whether lr.default_rules was
obeyed correctly.
* tests/local.at (AT_TEST_TABLES_AND_PARSE): New macro that
helps with checking the parser tables for a grammar.
* tests/input.at (%define lr.default_rules invalid values): New
test group.
* tests/reduce.at (AT_TEST_LR_DEFAULT_RULES): New macro using
AT_TEST_TABLES_AND_PARSE.
(`no %define lr.default_rules'): New test group generated by
AT_TEST_LR_DEFAULT_RULES.
(`%define lr.default_rules "all"'): Likewise.
(`%define lr.default_rules "consistent"'): Likewise.
(`%define lr.default_rules "accepting"'): Likewise.
* src/conflicts.c (conflicts_update_state_numbers): Fix for-loop.
* src/lalr.c (lalr_update_state_numbers): Fix for-loop.
* src/reader.c (check_and_convert_grammar): Fix for-loop.
* src/state.c (state_mark_reachable_states): Fix for-loop.
(state_remove_unreachable_states): Fix for-loop.
Don't widen struct state with member reachable just to temporarily
record reachability. Instead, use a local bitset.
* src/state.h (struct state): Remove member.
* src/state.c (state_new): Don't initialize it.
(state_mark_reachable_states): Rename to...
(state_record_reachable_states): ... this, and use bitset.
(state_remove_unreachable_states): Use bitset.
* data/push.c (yypull_parse): Report memory exhaustion and return 2 if
yypstate_new returns NULL.
(yypstate_new): Return NULL if malloc does.
* src/reader.c (packgram): Move translation of rule actions from the
beginning of packgram to...
(check_and_convert_grammar): ... here right before packgram is invoked
so it's easier to write more complete comments, and remove redundant
code.
params with digraphs.
* data/bison.m4 (b4_pre_prologue, b4_post_prologue): Set their default
values to the empty string since these are no longer guaranteed
initialized by the front-end.
* data/glr.c, data/glr.cc, data/lalr1.cc, data/push.c, data/yacc.c: Add
braces around b4_user_stype since this is no longer done by the
front-end.
* src/files.c, src/files.h (pre_prologue_obstack,
post_prologue_obstack): Remove.
* src/muscle_tab.c (muscle_pair_list_grow): Don't duplicate header
comments here. Use MUSCLE_OBSTACK_SGROW so that values are escaped
with digraphs. This fixes lex params and parse params.
* src/muscle_tab.h (muscle_pair_list_grow): Update comments.
* src/output.c (prepare): Remove muscle insertion of the prologues.
(output): Remove freeing of pre_prologue_obstack and
post_prologue_obstack.
* src/parse-gram.y (prologue_declaration): Use muscle_code_grow rather
than prologue_augment for prologue parsing so you don't need prologue
obstacks.
(grammar_declaration): Use `braceless' instead of "{...}" so that
braces are already stripped and code is escaped with digraphs.
* src/reader.c (prologue_augment): Remove.
(reader): Remove initialization of pre_prologue_obstack and
post_prologue_obstack.
* src/reader.h (prologue_augment): Remove.
* data/c.m4: Remove stray parenthesis.
* src/symtab.h (symbol, semantic_type): Remove destructor_location and
printer_location members, and change the type of the destructor and
printer members to code_props.
(symbol_destructor_set, symbol_destructor_get, symbol_printer_set,
symbol_printer_get, semantic_type_destructor_set,
semantic_type_printer_set, default_tagged_destructor_set,
default_tagless_destructor_set, default_tagged_printer_set,
default_tagless_printer_set): Use code_props in arguments and return
types in place of char const * and location.
(symbol_destructor_location_get, symbol_printer_location_get): Remove
since the locations are now contained in the return of
symbol_destructor_get and symbol_printer_get.
* src/output.c (symbol_destructors_output, symbol_printers_output):
Replace with...
(symbol_code_props_output): ... this to eliminate duplicate code.
(output_skeleton): Update to use symbol_code_props_output.
* src/reader.c (symbol_should_be_used): Update use of
symbol_destructor_get.
* src/symlist.c (symbol_list_destructor_set, symbol_list_printer_set):
Update uses of the various _destructor_set and _printer_set functions.
* src/symtab.c: (default_tagged_destructor_location,
default_tagless_destructor_location, default_tagged_printer_location,
default_tagless_printer_location): Remove since we...
(default_tagged_destructor, default_tagless_destructor,
default_tagged_printer, default_tagless_printer): ... change the type
of these to code_props.
(symbol_new, semantic_type_new, symbol_destructor_set,
semantic_type_destructor_set, symbol_destructor_get,
symbol_printer_set, semantic_type_printer_set, symbol_printer_get,
symbol_check_alias_consistency, default_tagged_destructor_set,
default_tagless_destructor_set, default_tagged_printer_set,
default_tagless_printer_set): Update.
(symbol_destructor_location_get, symbol_printer_location_get): Remove.
(SYMBOL_CODE_PRINT): New similar to SYMBOL_ATTR_PRINT but for
code_props members.
(symbol_print): Use SYMBOL_CODE_PRINT.
* src/reader.c (grammar_current_rule_prec_set): Make the %prec symbol a
token.
* tests/input.at (%prec takes a token): New test checking that %prec
won't take a nonterminal.
destructors, printers, and actions into a code_props structure and
interface.
* src/location.h (EMPTY_LOCATION_INIT): Define so that it's easier to
consistently initialize const structs that have an empty location field.
* src/location.c (empty_location): Initialize with EMPTY_LOCATION_INIT
to ensure consistency.
* src/output.c (symbol_destructors_output, symbol_printers_output):
Replace with...
(symbol_code_props_output): ... this to eliminate duplicate code.
(output_skeleton): Update to use symbol_code_props_output.
* src/parse-gram.y (prologue_declaration, braceless, epilogue.opt):
Update all uses of translate_* functions to use the new code_props
interface and to use gram_scanner_last_string_free and
code_scanner_last_string_free where possible.
(grammar_declaration): symbol_list_destructor_set and
symbol_list_printer_set now perform the translation, so don't do it
here. Use gram_scanner_last_string_free where possible.
* src/reader.c: Update to use code_props interface for destructors and
rule actions.
* src/scan-code.h (code_props): New structure.
(code_props_none_init, CODE_PROPS_NONE_INIT, code_props_none): New
function, macro, and const global variable for initializing a
code_props with no code.
(code_props_plain_init, code_props_symbol_action_init,
code_props_rule_action_init, code_props_translate_code,
code_props_code_get, code_props_location_get,
code_props_is_value_used): The rest of the new code_props interface.
(translate_rule_action, translate_symbol_action, translate_code):
Remove as these are now just special cases within
code_props_translate_code, which is switched on the code_props kind.
(code_scanner_last_string_free): New function similar to
gram_scanner_last_string_free.
* src/scan-code.l: Implement the new interface.
(last_string): New static global similar to the one in scan-gram.l.
(SC_SYMBOL_ACTION): For $$, set the is_value_used member of the
code_props since Bison will one day use this information for
destructors and printers.
(<*><<EOF>>): Use STRING_FINISH so that last_string is set.
(handle_action_dollar): Update to use the code_props interface of rule
actions. Use symbol_list_n_get and set is_value_used directly since
symbol_list_n_used_set is removed.
* src/symlist.h, src/symlist.c (symbol_list): Replace action,
action_location, and used members with a code_props member, and update
all uses.
(symbol_list_n_used_set): Remove since it would need to break the
encapsulation of code_props.
(symbol_list_destructor_set, symbol_list_printer_set): Perform code
translation here rather than depending on the caller to do so.
* src/symtab.h (symbol, semantic_type): Remove destructor_location and
printer_location members and change the type of the destructor and
printer members to code_props.
(symbol_destructor_location_get, symbol_printer_location_get): Remove
unneeded.
(symbol_destructor_set, symbol_destructor_get, symbol_printer_set,
symbol_printer_get, semantic_type_destructor_set,
semantic_type_printer_set, default_tagged_destructor_set,
default_tagless_destructor_set, default_tagged_printer_set,
default_tagless_printer_set): Use code_props in arguments and return
types in place of const char * and location.
* src/symtab.c: Update implementation for interface and struct changes.
(default_tagged_destructor_location,
default_tagless_destructor_location, default_tagged_printer_location,
default_tagless_printer_location): Remove since we...
(default_tagged_destructor, default_tagless_destructor,
default_tagged_printer, default_tagless_printer): ... change the type
of these to code_props.
(SYMBOL_CODE_PRINT): New similar to SYMBOL_ATTR_PRINT but for
code_props members.
(symbol_print): Use SYMBOL_CODE_PRINT.
* src/scan-gram.h (gram_last_string): Remove declaration.
* src/scan-gram.l (last_string): Declare it static.
--warnings=midrule-values to enable them. Discussed starting at
<http://lists.gnu.org/archive/html/help-bison/2006-10/msg00030.html>.
* NEWS (2.3a+): Mention.
* src/getargs.c, src/getargs.h (warnings_args, warnings_types, enum
warnings): Add entry for midrule-values subargument.
* src/reader.c (symbol_should_be_used): Don't return true just because
the value is a set/used mid-rule value unless
--warnings=midrule-values was specified.
* tests/input.at (Unused values, Unused values before symbol
declarations): Run tests with and without --warnings=midrule-values.
* src/reader.c (check_and_convert_grammar): Use symbol_list_free rather
than LIST_FREE directly.
value. Mentioned at
<http://lists.gnu.org/archive/html/bison-patches/2006-09/msg00104.html>.
* src/symtab.c (dummy_symbol_get): Name all dummy symbols initially
like $@n instead of just @n so that the default %destructor/%printer
logic doesn't see them as user-defined symbols.
(symbol_is_dummy): Check for both forms of the name.
* src/reader.c (packgram): Remove the `$' from each midrule symbol
name for which the midrule value is referenced in any action.
* tests/actions.at (Default %printer and %destructor for mid-rule
values): New test.
* tests/regression.at (Rule Line Numbers, Web2c Report): Update output
for change to dummy symbol names.
* src/reader.c (symbol_should_be_used): Check midrule parent rule for
$n usage.
(packgram): Before invoking grammar_rule_check on any rule, make sure
all actions have already been scanned in order to set `used' flags.
Otherwise, checking that a midrule's $$ is set will not always work
properly because the midrule check must forward-reference the midrule's
parent rule.
* tests/input.at (AT_CHECK_UNUSED_VALUES): Extend to check the new
warning.
alternative to the pre-prologue. Mentioned at
<http://lists.gnu.org/archive/html/bison-patches/2006-10/msg00063.html>.
Also, let the prologue alternatives appear in the grammar section.
* src/parse-gram.y (PERCENT_CODE_TOP): New token.
(prologue_declaration): Move the existing prologue alternatives to...
(grammar_declaration): ... here and add %code-top.
* src/scan-gram.l (PERCENT_CODE_TOP): New token.
Clean up and extend documentation for the prologue alternatives.
* NEWS (2.3a+): Describe prologue alternatives.
* doc/bison.texinfo (Prologue): Move discussion of prologue
alternatives to...
(Prologue Alternatives): ... this new section, and extend it to discuss
all 4 directives in detail.
(Bison Symbols): Clean up discussion of prologue alternatives and add
%code-top.
<http://lists.gnu.org/archive/html/bug-bison/2006-09/msg00018.html>.
* src/parse-gram.y (symbol_declaration): Don't put statements
before declarations; it's not portable to C89.
* src/scan-code.l (handle_action_at): Likewise.
* src/scan-code.l: Always initialize braces_level; the old code
left it uninitialized and therefore had undefined behavior.
Don't attempt to redefine 'assert', since it runs afoul of
systems where standard headers (mistakenly) include <assert.h>.
Instead, define and use our own alternative, called 'aver'.
* src/reader.c: Don't include assert.h, since we no longer
use assert.
* src/scan-code.l: Likewise.
* src/system.h (assert): Remove, replacing with....
(aver): New function, taking a bool arg. All uses changed.
* src/tables.c (pack_vector): Ensure that aver arg is bool,
not merely an integer.
%symbol-default instead of an empty symbol list, and start working on
new per-type %destructor/%printer. Discussed at
<http://lists.gnu.org/archive/html/bison-patches/2006-09/msg00007.html>.
* NEWS (2.3+): Add %symbol-default to example.
* bison.texinfo (Freeing Discarded Symbols): Likewise.
(Bison Symbols): Add entry for %symbol-default.
* src/parse-gram.y (PERCENT_SYMBOL_DEFAULT): New token.
(generic_symlist, generic_symlist_item): New nonterminals for creating
a list in which each item is a symbol, semantic type, or
%symbol-default.
(grammar_declaration): Use generic_symlist in %destructor and %printer
declarations instead of symbols.1 or an empty list.
(symbol_declaration, precedence_declaration, symbols.1): Update actions
for changes to symbol_list.
* src/reader.c: Update for changes to symbol_list.
* src/scan-code.l: Likewise.
* src/scan-gram.l: Scan new PERCENT_SYMBOL_DEFAULT token.
* src/symlist.c, src/symlist.h: Extend such that a list node may
represent a semantic type or a %symbol-default in addition to just an
ordinary symbol. Add switched functions for setting %destructor's and
%printer's.
* tests/actions.at, tests/input.at: Add %symbol-default to all default
%destructor/%printer declarations.
* src/reader.c (grammar_current_rule_begin): Don't set the start symbol
when parsing the first rule.
(check_and_convert_grammar): Search for it here after all grammar
declarations have been parsed. Skip midrules, which have dummy LHS
nonterminals.
* src/symtab.c (symbol_is_dummy): New function.
* src/symtab.h (symbol_is_dummy): Declare it.
* tests/input.at (%start after first rule): New test.
non-aliased/undeclared string literals since it might be useful to
those declaring %token-table.
* src/reader.c (check_and_convert_grammar): Undo changes in previous
commit: don't worry about complaints from symbols_pack.
* src/symtab.c (symbol_new, symbol_class_set,
symbol_check_alias_consistency): Undo changes in previous commit: count each string literal as a new symbol and token, assign it a symbol
number, and don't complain about non-aliased string literals.
(symbols_pack): Since symbol_make_alias still does not decrement symbol
and token counts but does still set aliased tokens to the same number,
symbol_pack_processor now leaves empty slots in the symbols array.
Remove those slots.
* tests/regression.at (Undeclared string literal): Remove test case
added in previous commit since non-aliased string literals are allowed
again.
(Characters Escapes, Web2c Actions): Undo changes in previous commit:
remove unnecessary string literal declarations.
* tests/sets.at (Firsts): Likewise.
be used before its declaration.
* src/reader.c (check_and_convert_grammar): Don't invoke packgram if
symbols_pack complained.
* src/symtab.c (symbol_new): Don't count a string literal as a new
symbol.
(symbol_class_set): Don't count a string literal as a new token, and
don't assign it a symbol number since symbol_make_alias does that.
(symbol_make_alias): It's not necessary to decrement the symbol and
token counts anymore. Don't assume that an alias declaration occurs
before any uses of the identifier or string, and thus don't assert that
one of them has the highest symbol number so far.
(symbol_check_alias_consistency): Complain if there's a string literal
that wasn't declared as an alias.
(symbols_pack): Bail if symbol_check_alias_consistency failed since
symbol_pack asserts that every token has been assigned a symbol number
although undeclared string literals have not.
* tests/regression.at (String alias declared after use, Undeclared
string literal): New test case.
(Characters Escapes, Web2c Actions): Declare string literals as
aliases.
* tests/sets.at (Firsts): Likewise.
use these for all user-declared grammar symbols for which the user does
not declare a specific %printer/%destructor. Thus, the parser uses it
for token 0 if the user declares it but not if Bison generates it as
$end. Discussed starting at
<http://lists.gnu.org/archive/html/bison-patches/2006-02/msg00064.html>,
<http://lists.gnu.org/archive/html/bison-patches/2006-06/msg00091.html>,
and
<http://lists.gnu.org/archive/html/bison-patches/2006-07/msg00019.html>.
* NEWS (2.3+): Mention.
* doc/bison.texinfo (Actions in Mid-Rule): It's no longer impossible to
declare a %destructor for a mid-rule's semantic value. It's just
impossible to declare one specific to it.
(Freeing Discarded Symbols): Mention that @$ can be used in %destructor
code. Describe default %destructor form.
* src/parse-gram.y (grammar_declaration): Parse default
%printer/%destructor declarations.
* src/output.c (symbol_destructors_output): Use symbol_destructor_get
and symbol_destructor_location_get rather than accessing the destructor
and destructor_location members of struct symbol.
(symbol_printers_output): Likewise but for %printer's.
* src/reader.c (symbol_should_be_used): Likewise but for %destructor's
again.
* src/symtab.c (default_destructor, default_destructor_location,
default_printer, default_printer_location): New static global
variables to record the default %destructor and %printer.
(symbol_destructor_get, symbol_destructor_location_get,
symbol_printer_get, symbol_printer_location_get): New functions to
compute the appropriate %destructor and %printer for a symbol.
(default_destructor_set, default_printer_set): New functions to set the
default %destructor and %printer.
* src/symtab.h: Prototype all those new functions.
* tests/actions.at (Default %printer and %destructor): New test to
check that the right %printer and %destructor are called, that they're
not called for $end, and that $$ and @$ work correctly.
(Default %printer and %destructor for user-declared end token): New
test to check that the default %printer and %destructor are called for
a user-declared end token.
* tests/input.at (Default %printer and %destructor redeclared, Unused
values with default %destructor): New tests to check related grammar
warnings and errors.
Discussed starting at
<http://lists.gnu.org/archive/html/bison-patches/2006-07/msg00019.html>
and
<http://lists.gnu.org/archive/html/help-bison/2006-07/msg00013.html>.
Make the skeletons consistent in how they pop the end token and invoke
its %destructor.
* data/glr.c (yyrecoverSyntaxError, yyparse): Don't pop the start
state, which has token number 0, since this would invoke the
%destructor for the end token.
* data/lalr1.cc (yy::parser::parse): Don't check for the final state
until after shifting the end token, or else it won't be popped.
* data/yacc.c (yyparse): Likewise.
* data/glr.c (yyparse): Clear the lookahead after shifting it even when
it's the end token. Upon termination, destroy an unshifted lookahead
even when it's the end token.
* data/lalr1.cc (yy::parser::parse): Likewise.
* data/yacc.c (yyparse): Likewise.
* src/reader.c (packgram): Don't check rule 0. This suppresses unused
value warnings for the end token when the user gives the end token a
%destructor.
* tests/actions.at (Printers and Destructors): Test all the above.
YYSTYPE if your grammar contains at least one <type> tag.
Your YYSTYPE need not be a macro; it can be a typedef.
* doc/bison.texinfo (Value Type, Multiple Types, Location Type):
(Union Decl, Decl Summary): Document this.
* data/glr.c (YYSTYPE): Implement this.
* data/glr.cc (YYSTYPE): Likewise.
* data/lalr1.cc (YYSTYPE): Likewise.
* data/yacc.c (YYSTYPE): Likewise.
* src/output.c (prepare): Output tag_seen_flag.
* src/parse-gram.y (declaration, grammar_declaration):
Use 'union_seen' rather than 'typed' to determine whether
%union has been seen, since grammars can now be typed without
%union.
(symbol_declaration, type.opt, symbol_def):
Keep track of whether a tag has been seen.
* src/reader.c (union_seen, tag_seen): New vars.
(typed): remove.
* src/reader.h (union_seen, tag_seen, typed): Likewise.
* src/scan-code.l (untyped_var_seen): New variable.
(handle_action_dollar): Adjust to above changes.
(handle_action_dollar, handle_action_at):
Improve overflow checking for outlandish numbers.
* tests/input.at (AT_CHECK_UNUSED_VALUES): Redo test to
avoid new diagnostics generated by above changes.
* tests/regression.at (YYSTYPE typedef): Add test to check
for type tags without %union.
declaration".
* src/symtab.c (redeclaration): Do that here.
* src/reader.c (record_merge_function_type): In the case of a result
type clash, report the previous declaration rather than the very first
one in the grammar file.
* tests/glr-regression.at (Missed %merge type warnings when LHS type is
declared later): Add a third declaration to check this behavior.
* tests/input.at (Incompatible Aliases): Update output.
declared after the %merge.
* src/reader.c (get_merge_function): Don't set the merger type yet.
(record_merge_function_type): New function for setting the merger type
and checking for clashes.
(grammar_current_rule_merge_set): Set the location of the %merge for
the current rule.
(packgram): Invoke record_merge_function_type for each rule now that
all symbol type declarations have been parsed.
* src/reader.h (merger_list.type_declaration_location): New member
storing the location of the first %merge from which the type for this
merging function was derived.
* src/symlist.h (symbol_list.merger_declaration_location): New member
storing the location of a rule's %merge, if any.
* tests/glr-regression.at (Missed %merge type warnings when LHS type is
declared later): New test to catch the error fixed by the above patch.
declarations appear after the rules. Don't mistake the type of $$ in
a midrule to be that of its parent rule's $$.
* src/reader.c (grammar_current_rule_end): Don't invoke
grammar_rule_check yet since not all symbol declarations may have been
parsed yet.
(grammar_midrule_action): Likewise.
Don't record whether the midrule's $$ has been used yet since actions
haven't been translated yet.
Record the midrule's parent rule and its RHS index within the parent
rule.
(grammar_current_rule_action_append): Don't translate the action yet
since not all symbol declarations may have been parsed yet and, thus,
warnings about types for $$, $n, @$, and @n can't be reported yet.
(packgram): Translate the action and invoke grammar_rule_check now that
all symbol declarations have been parsed.
* src/scan-code.l (handle_action_dollar): Now that this is invoked
after parsing the entire grammar file, the symbol list here in the case
of a midrule is actually the midrule's empty RHS, so reference its
parent rule's RHS where necessary.
On the other hand, now that you can already know it's a midrule, you
aren't forced to think $$ has the same type as its parent rule's $$.
(handle_action_at): In the case of a midrule, reference the parent rule
where necessary.
* src/symlist.c (symbol_list_new): Initialize new midrule-related
members.
(symbol_list_length): Now that this is invoked after all rules have
been parsed, a NULL symbol (rather than a NULL symbol list node)
terminates a rule. symbol_list_print already does this correctly.
* src/symlist.h (symbol_list.midrule_parent_rule,
symbol_list.midrule_parent_rhs_index): New members so that midrules can
remember their relationships with their parents.
* tests/input.at (Type Clashes): Extend to catch the midrule $$ error
fixed by the above patch.
(_AT_UNUSED_VALUES_DECLARATIONS, AT_CHECK_UNUSED_VALUES): New m4 macros
implementing...
(Unused values): ... this old test case and...
(Unused values before symbol declarations): ... this new test case.
This one is the same as `Unused values' except that all symbol
declarations appear after the rules in order to catch the rest of the
errors fixed by the above patch.
* parse-gram.y (rhs): Move grammar_midrule_action invocation from here
to...
* src/reader.c (grammar_current_rule_action_append): ... here for
consistency with grammar_current_rule_symbol_append.
* tests/regression.at (Braced code in declaration in rules section):
Make yyerror and yylex static as usual.
%end-header. Don't use these declarations to separate pre-prologue
blocks from post-prologue blocks. Add new order-independent
declarations %before-header and %after-header as alternatives to the
traditional Yacc pre-prologue and post-prologue blocks. Discussed at
<http://lists.gnu.org/archive/html/bison-patches/2006-06/msg00110.html>.
* NEWS (2.3+): Update for these changes.
* data/glr.c (b4_before_definitions): Update to...
(b4_start_header): ... this.
(b4_after_definitions): Update to...
(b4_end_header): ... this.
* data/glr.cc: Likewise.
* data/lalr1.cc: Likewise.
* data/yacc.c: Likewise.
* doc/bison.texinfo (The prologue): Update names, and replace remaining
prologue blocks with %*-header declarations.
(Calc++ Parser): Likewise.
(Bison Declaration Summary): Update names.
(Bison Symbols): Update description.
* src/parse-gram.y (PERCENT_AFTER_DEFINITIONS): Update to...
(PERCENT_END_HEADER): ... this.
(PERCENT_BEFORE_DEFINITIONS): Update to...
(PERCENT_START_HEADER): ... this.
(PERCENT_AFTER_HEADER, PERCENT_BEFORE_HEADER): New tokens.
(declaration): Update token names and m4 macro names.
When parsing %end-header and %start-header, invoke translate_code
before muscle_code_grow, and no longer set global booleans to remember
whether these declarations have been seen.
Parse new %after-header and %before-header.
* src/reader.c (before_definitions, after_definitions): Remove.
(prologue_augment): Accept a new bool argument to specify whether to
augment the pre-prologue or post-prologue.
* src/reader.h (before_definitions, after_definitions): Remove these
extern's.
(prologue_augment): Add new bool argument.
* src/scan-gram.l (PERCENT_AFTER_DEFINITIONS): Update to...
(PERCENT_END_HEADER): ... this.
(PERCENT_BEFORE_DEFINITIONS): Update to...
(PERCENT_START_HEADER): ... this.
(PERCENT_AFTER_HEADER, PERCENT_BEFORE_HEADER): New tokens.
* tests/actions.at (Printers and Destructors): Update names.
file and the glr.c header and code files, move the pre-prologue before
the token definitions. Add new %before-definitions and
%after-definitions to declare code that will go in both the header file
and code file. Discussed at
<http://lists.gnu.org/archive/html/bison-patches/2005-12/msg00000.html>,
<http://lists.gnu.org/archive/html/bison-patches/2006-01/msg00016.html>,
and
<http://lists.gnu.org/archive/html/bison-patches/2006-06/msg00055.html>.
* NEWS (2.3+): Describe these changes.
* data/glr.c (b4_pre_prologue): Move from within to before...
(b4_shared_declarations): ... this.
Add new b4_before_definitions before b4_token_enums.
Add new b4_after_definitions at the end.
* data/glr.cc (b4_pre_prologue): Replace with...
(b4_before_definitions): ... this in the header file.
(b4_after_definitions): New near the end of the header file.
* data/lalr1.cc (b4_pre_prologue): Move from the header file to the
code file right before including the header file.
(b4_before_definitions): New in the previous position of
b4_pre_prologue in the header file.
(b4_after_definitions): New near the end of the header file.
* data/yacc.c: Clean up some m4 quoting especially in the header file.
(b4_token_enums_defines): In the code file, move to right before
YYSTYPE for consistency with the header file.
(b4_before_definitions): New right before b4_token_enums_defines in
both the header and code file.
(b4_after_definitions): New right after YYLTYPE and yylloc in both the
header and code file.
* doc/bison.texinfo (Prologue): Show use of %before-definitions instead
of prologues for %union dependencies.
(Bison Declaration Summary): In %defines description, mention the
effect of %before-definitions and %after-definitions on the header
file.
(Calc++ Parser): Forward declare driver in a %before-definitions rather
than in the pre-prologue so that make check succeeds.
(Bison Symbols): Add entries for %before-definitions and
%after-definitions.
* src/parse-gram.y (PERCENT_BEFORE_DEFINITIONS): New token for
%before-definitions.
(PERCENT_AFTER_DEFINITIONS): New token for %after-definitions.
(declaration): Parse those declarations and append to
b4_before_definitions and b4_after_definitions, respectively.
* src/reader.c (before_definitions, after_definitions): New bools to
track whether those declarations have been seen.
(prologue_augment): Add to the post-prologue if %union,
%before-definitions, or %after-definitions has been seen.
* src/reader.h (before_definitions, after_definitions): New extern's.
* src/scan-gram.l: Scan the new declarations.
* tests/actions.at (_AT_CHECK_PRINTER_AND_DESTRUCTOR): Place the second
prologue block in a %before-definitions or a %after-definitions based
on whether the %union is declared.
* tests/regression.at (Early token definitions with --yacc, Early token
definitions without --yacc): Move tests for token definitions into the
post-prologue since token names are no longer defined in the
pre-prologue.
As a consequence, the relation between the grammar scanner and
parser is much simpler. We can also split "composite tokens" back
into simple tokens.
* src/gram.h (ITEM_NUMBER_MAX, RULE_NUMBER_MAX): New.
* src/scan-gram.l (add_column_width, adjust_location): Move to and
rename as...
* src/location.h, src/location.c (add_column_width)
(location_compute): these.
Fix the column count: the initial column is 0.
(location_print): Be robust to ending column being 0.
* src/location.h (boundary_set): New.
* src/main.c: Adjust to scanner_free being renamed as
gram_scanner_free.
* src/output.c: Include scan-code.h.
* src/parse-gram.y: Include scan-gram.h and scan-code.h.
Use boundary_set.
(PERCENT_DESTRUCTOR, PERCENT_PRINTER, PERCENT_INITIAL_ACTION)
(PERCENT_LEX_PARAM, PERCENT_PARSE_PARAM): Remove the {...} part,
which is now, again, a separate token.
Adjust all dependencies.
Whereever actions with $ and @ are used, use translate_code.
(action): Remove this nonterminal which is now useless.
* src/reader.c: Include assert.h, scan-gram.h and scan-code.h.
(grammar_current_rule_action_append): Use translate_code.
(packgram): Bound check ruleno, itemno, and rule_length.
* src/reader.h (gram_in, gram__flex_debug, scanner_cursor)
(last_string, last_braced_code_loc, max_left_semantic_context)
(scanner_initialize, scanner_free, scanner_last_string_free)
(gram_out, gram_lineno, YY_DECL_): Move to...
* src/scan-gram.h: this new file.
(YY_DECL): Rename as...
(GRAM_DECL): this.
* src/scan-code.h, src/scan-code.l, src/scan-code-c.c: New.
* src/scan-gram.l (gram_get_lineno, gram_get_in, gram_get_out):
(gram_get_leng, gram_get_text, gram_set_lineno, gram_set_in):
(gram_set_out, gram_get_debug, gram_set_debug, gram_lex_destroy):
Move these declarations, and...
(obstack_for_string, STRING_GROW, STRING_FINISH, STRING_FREE):
these to...
* src/flex-scanner.h: this new file.
* src/scan-gram.l (rule_length, rule_length_overflow)
(increment_rule_length): Remove.
(last_braced_code_loc): Rename as...
(gram_last_braced_code_loc): this.
Adjust to the changes of the parser.
Move all the handling of $ and @ into...
* src/scan-code.l: here.
* src/scan-gram.l (handle_dollar, handle_at): Remove.
(handle_action_dollar, handle_action_at): Move to...
* src/scan-code.l: here.
* src/Makefile.am (bison_SOURCES): Add flex-scanner.h,
scan-code.h, scan-code-c.c, scan-gram.h.
(EXTRA_bison_SOURCES): Add scan-code.l.
(BUILT_SOURCES): Add scan-code.c.
(yacc): Be robust to white spaces.
* tests/conflicts.at, tests/input.at, tests/reduce.at,
* tests/regression.at: Adjust the column numbers.
* tests/regression.at: Adjust the error message.
