them to the parser in order to improve error messages.
* src/scan-gram.l (SC_ESCAPED_STRING, SC_ESCAPED_CHARACTER,
SC_BRACED_CODE, SC_PROLOGUE): Implement.
* tests/input.at (Unclosed constructs): New test case.
* tests/regression.at (Invalid inputs): Update now that unclosed %{ is
seen.
* src/scan-gram.h, src/scan-gram.l (gram_last_braced_code_loc): Remove
unused global.
* src/flex-scanner.h (FLEX_NO_OBSTACK): New macro that blocks obstack
definitions so gcc won't warn when obstack_for_string is unused.
* src/scan-code.l: config.h and system.h are already #include'd by
scan-code-c.c, so get rid of them here.
* src/scan-gram.l: Likewise.
* src/scan-skel.l: Likewise, and use flex-scanner.h without obstack
definitions rather than duplicating the rest of it.
* src/scan-gram-c.c, scan-skel-c.c: #include "system.h".
* src/scan-code.h, src/scan-code.l (translate_action)
(translate_rule_action, translate_symbol_action, translate_code):
Return char *, not const char *.
* src/parse-gram.y (declaration): Rename as...
(prologue_declaration): this.
(string_content): Remove this nonterminal, use STRING.
(braceless, content, content.opt): New nonterminal.
Use them.
(%define): Now accept content.opt, i.e., accept also BRACED_CODE
as value.
* src/scan-gram.l (getargs.h): Don't include it.
on every other line.
* src/parse-gram.y (%initial-action): Initialize @$ correctly.
* tests/input.at (Torturing the Scanner): Update output.
* src/scan-gram.l (scanner_cursor): Declare it static.
to be a rule action. Mentioned at
<http://lists.gnu.org/archive/html/bison-patches/2006-06/msg00105.html>.
* src/scan-gram.l: Move midrule action detection from the start of the
scanning of any braced code to...
* src/parse-gram.y (rhs): ... the parsing of braced code as a rule
action. For readability, use $2 and @2 rather than the equivalent
global variables.
* tests/regression.at (Braced code in declaration in rules section):
New test to catch the error fixed by the above patch.
Work on code readability some.
* src/scan-code.l (current_rule): Get rid of this misleading and
redundant declaration: it's actually extern'ed in reader.h.
(YY_DECL, code_lex, handle_action_dollar, handle_action_at,
translate_action): Add a rule argument and use it instead of the global
current_rule.
(translate_rule_action): This already receives current_rule through an
argument, so pass it on to translate_action instead of assigning
current_rule to current_rule.
(translate_symbol_action, translate_code): Pass rule = NULL to
translate_action.
%end-header. Don't use these declarations to separate pre-prologue
blocks from post-prologue blocks. Add new order-independent
declarations %before-header and %after-header as alternatives to the
traditional Yacc pre-prologue and post-prologue blocks. Discussed at
<http://lists.gnu.org/archive/html/bison-patches/2006-06/msg00110.html>.
* NEWS (2.3+): Update for these changes.
* data/glr.c (b4_before_definitions): Update to...
(b4_start_header): ... this.
(b4_after_definitions): Update to...
(b4_end_header): ... this.
* data/glr.cc: Likewise.
* data/lalr1.cc: Likewise.
* data/yacc.c: Likewise.
* doc/bison.texinfo (The prologue): Update names, and replace remaining
prologue blocks with %*-header declarations.
(Calc++ Parser): Likewise.
(Bison Declaration Summary): Update names.
(Bison Symbols): Update description.
* src/parse-gram.y (PERCENT_AFTER_DEFINITIONS): Update to...
(PERCENT_END_HEADER): ... this.
(PERCENT_BEFORE_DEFINITIONS): Update to...
(PERCENT_START_HEADER): ... this.
(PERCENT_AFTER_HEADER, PERCENT_BEFORE_HEADER): New tokens.
(declaration): Update token names and m4 macro names.
When parsing %end-header and %start-header, invoke translate_code
before muscle_code_grow, and no longer set global booleans to remember
whether these declarations have been seen.
Parse new %after-header and %before-header.
* src/reader.c (before_definitions, after_definitions): Remove.
(prologue_augment): Accept a new bool argument to specify whether to
augment the pre-prologue or post-prologue.
* src/reader.h (before_definitions, after_definitions): Remove these
extern's.
(prologue_augment): Add new bool argument.
* src/scan-gram.l (PERCENT_AFTER_DEFINITIONS): Update to...
(PERCENT_END_HEADER): ... this.
(PERCENT_BEFORE_DEFINITIONS): Update to...
(PERCENT_START_HEADER): ... this.
(PERCENT_AFTER_HEADER, PERCENT_BEFORE_HEADER): New tokens.
* tests/actions.at (Printers and Destructors): Update names.
file and the glr.c header and code files, move the pre-prologue before
the token definitions. Add new %before-definitions and
%after-definitions to declare code that will go in both the header file
and code file. Discussed at
<http://lists.gnu.org/archive/html/bison-patches/2005-12/msg00000.html>,
<http://lists.gnu.org/archive/html/bison-patches/2006-01/msg00016.html>,
and
<http://lists.gnu.org/archive/html/bison-patches/2006-06/msg00055.html>.
* NEWS (2.3+): Describe these changes.
* data/glr.c (b4_pre_prologue): Move from within to before...
(b4_shared_declarations): ... this.
Add new b4_before_definitions before b4_token_enums.
Add new b4_after_definitions at the end.
* data/glr.cc (b4_pre_prologue): Replace with...
(b4_before_definitions): ... this in the header file.
(b4_after_definitions): New near the end of the header file.
* data/lalr1.cc (b4_pre_prologue): Move from the header file to the
code file right before including the header file.
(b4_before_definitions): New in the previous position of
b4_pre_prologue in the header file.
(b4_after_definitions): New near the end of the header file.
* data/yacc.c: Clean up some m4 quoting especially in the header file.
(b4_token_enums_defines): In the code file, move to right before
YYSTYPE for consistency with the header file.
(b4_before_definitions): New right before b4_token_enums_defines in
both the header and code file.
(b4_after_definitions): New right after YYLTYPE and yylloc in both the
header and code file.
* doc/bison.texinfo (Prologue): Show use of %before-definitions instead
of prologues for %union dependencies.
(Bison Declaration Summary): In %defines description, mention the
effect of %before-definitions and %after-definitions on the header
file.
(Calc++ Parser): Forward declare driver in a %before-definitions rather
than in the pre-prologue so that make check succeeds.
(Bison Symbols): Add entries for %before-definitions and
%after-definitions.
* src/parse-gram.y (PERCENT_BEFORE_DEFINITIONS): New token for
%before-definitions.
(PERCENT_AFTER_DEFINITIONS): New token for %after-definitions.
(declaration): Parse those declarations and append to
b4_before_definitions and b4_after_definitions, respectively.
* src/reader.c (before_definitions, after_definitions): New bools to
track whether those declarations have been seen.
(prologue_augment): Add to the post-prologue if %union,
%before-definitions, or %after-definitions has been seen.
* src/reader.h (before_definitions, after_definitions): New extern's.
* src/scan-gram.l: Scan the new declarations.
* tests/actions.at (_AT_CHECK_PRINTER_AND_DESTRUCTOR): Place the second
prologue block in a %before-definitions or a %after-definitions based
on whether the %union is declared.
* tests/regression.at (Early token definitions with --yacc, Early token
definitions without --yacc): Move tests for token definitions into the
post-prologue since token names are no longer defined in the
pre-prologue.
`{'. Rather, return a single "%union" token.
No longer make symbols: return strings, and leave the conversion
to symbols to the parser.
(SC_PRE_CODE, token_type): Remove.
* src/parse-gram.y (%union): New field `character'.
Sort tokens.
(CHAR): New token.
(ID, ID_COLON): Now that the scanner no longer makes them
identifiers, adjust all uses to invoke symbol_get.
(id_colon): New, wraps the conversion from string to symbol.
(%union): Accept a possible union_name.
(symbol): Now can be a char.
* data/c.m4 (b4_union_name): Leave a default value.
* data/glr.c, data/yacc.c: Use it.
As a consequence, the relation between the grammar scanner and
parser is much simpler. We can also split "composite tokens" back
into simple tokens.
* src/gram.h (ITEM_NUMBER_MAX, RULE_NUMBER_MAX): New.
* src/scan-gram.l (add_column_width, adjust_location): Move to and
rename as...
* src/location.h, src/location.c (add_column_width)
(location_compute): these.
Fix the column count: the initial column is 0.
(location_print): Be robust to ending column being 0.
* src/location.h (boundary_set): New.
* src/main.c: Adjust to scanner_free being renamed as
gram_scanner_free.
* src/output.c: Include scan-code.h.
* src/parse-gram.y: Include scan-gram.h and scan-code.h.
Use boundary_set.
(PERCENT_DESTRUCTOR, PERCENT_PRINTER, PERCENT_INITIAL_ACTION)
(PERCENT_LEX_PARAM, PERCENT_PARSE_PARAM): Remove the {...} part,
which is now, again, a separate token.
Adjust all dependencies.
Whereever actions with $ and @ are used, use translate_code.
(action): Remove this nonterminal which is now useless.
* src/reader.c: Include assert.h, scan-gram.h and scan-code.h.
(grammar_current_rule_action_append): Use translate_code.
(packgram): Bound check ruleno, itemno, and rule_length.
* src/reader.h (gram_in, gram__flex_debug, scanner_cursor)
(last_string, last_braced_code_loc, max_left_semantic_context)
(scanner_initialize, scanner_free, scanner_last_string_free)
(gram_out, gram_lineno, YY_DECL_): Move to...
* src/scan-gram.h: this new file.
(YY_DECL): Rename as...
(GRAM_DECL): this.
* src/scan-code.h, src/scan-code.l, src/scan-code-c.c: New.
* src/scan-gram.l (gram_get_lineno, gram_get_in, gram_get_out):
(gram_get_leng, gram_get_text, gram_set_lineno, gram_set_in):
(gram_set_out, gram_get_debug, gram_set_debug, gram_lex_destroy):
Move these declarations, and...
(obstack_for_string, STRING_GROW, STRING_FINISH, STRING_FREE):
these to...
* src/flex-scanner.h: this new file.
* src/scan-gram.l (rule_length, rule_length_overflow)
(increment_rule_length): Remove.
(last_braced_code_loc): Rename as...
(gram_last_braced_code_loc): this.
Adjust to the changes of the parser.
Move all the handling of $ and @ into...
* src/scan-code.l: here.
* src/scan-gram.l (handle_dollar, handle_at): Remove.
(handle_action_dollar, handle_action_at): Move to...
* src/scan-code.l: here.
* src/Makefile.am (bison_SOURCES): Add flex-scanner.h,
scan-code.h, scan-code-c.c, scan-gram.h.
(EXTRA_bison_SOURCES): Add scan-code.l.
(BUILT_SOURCES): Add scan-code.c.
(yacc): Be robust to white spaces.
* tests/conflicts.at, tests/input.at, tests/reduce.at,
* tests/regression.at: Adjust the column numbers.
* tests/regression.at: Adjust the error message.
got from <http://bro-ids.org/download.html>. The bug is that
when two actions appeared in succession, the second one was
scanned before the first one was added to the grammar rule
as a midrule action. Bison then output the incorrect warning
"parse.y:905.17-906.36: warning: unused value: $3".
* src/parse-gram.y (BRACED_CODE, action): These are no longer
associated with a value.
(rhs): Don't invoke grammar_current_rule_action_append.
(action): Invoke it here instead.
* src/reader.c (grammar_midrule_action): Now extern.
(grammar_current_rule_action_append): Don't invoke
grammar_midrule_action; that is now the scanner's job.
* src/reader.h (last_string, last_braced_code_loc):
(grammar_midrule_action): New decls.
* src/scan-gram.l (last_string): Now extern, sigh.
(last_braced_code_loc): New extern variable.
(<INITIAL>"{"): Invoke grammar_midrule_action if the current
rule already has an action.
(<SC_BRACED_CODE>"}"): Set last_braced_code_loc before returning.
* tests/input.at (AT_CHECK_UNUSED_VALUES):
Add some tests to check that the above changes fixed the bug.
to pacify gcc -Wswitch-default.
* src/scan-gram.l (adjust_location): Use a default: break; case
to pacify gcc -Wswitch-default.
* src/scan-skel.h (skel_get_lineno, skel_get_in, skel_get_out):
(skel_get_leng, skel_get_text, skel_set_lineno, skel_set_in):
(skel_set_out, skel_get_debug, skel_set_debug, skel_lex_destroy):
Move these decls to scan-skel.l, since they don't need to be
visible elsewhere.
* src/scan-skel.l: Accept the above decls.
(skel_lex): New decl, to pacify GCC when unpatched flex 2.5.31
is used.
* src/symtab.h (struct symbol.declared): New member.
* src/symtab.c (symbol_new): Initialize it to false.
(symbol_class_set): New arg DECLARING, specifying whether
this is a declaration that we want to warn about, if there
is more than one of them. All uses changed.
to INT_MAX indicates an overflow.
* src/scan-gram.l: Include verify.h. Don't include get-errno.h.
(rule_length_overflow, increment_rule_length, add_column_width):
New functions.
(<INITIAL>{id}, <SC_ESCAPED_STRING>"\"", <SC_ESCAPED_CHARACTER>"'"):
(<SC_BRACED_CODE>"}"):
Use increment_rule_length rather than incrementing it by hand.
(adjust_location, handle_syncline): Diagnose overflow.
(handle_action_dollar, handle_action_at):
Fix bug with monstrosities like $-2147483648.
Remove now-unnecessary checks.
(scan_integer): Verify assumptions and remove now-unnecessary checks.
(convert_ucn_to_byte): Verify assumptions.
(handle_syncline): New arg LOC. All callers changed.
Don't store through a value derived from char const * pointer.
is used.
Take the rule to check as argument, hence rename as...
(grammar_rule_check): this.
* src/reader.h, src/reader.c (grammar_rule_begin, grammar_rule_end):
Rename as...
(grammar_rule_begin, grammar_rule_end): these, for consistency.
(grammar_midrule_action, grammar_symbol_append): Now static.
* tests/torture.at (input): Don't rely on the default action
being always performed.
* tests/calc.at: "Set" $$ even when the action is "cut" with
YYERROR or other.
* tests/actions.at (Exotic Dollars): Instead of using unused
values, check that the warning is issued.
(xlc and xlC_r) reported by John P. Hartmann.
* data/location.cc (initial_column, initial_line): Remove.
All uses replaced by 0 and 1.
* src/scan-gram.l (gram_wrap): Redefine to avoid bug in flex 2.5.31
that xlc complains about.
* src/scan-skel.l (skel_wrap): Likewise.
(string_as_id): Quote $1 before using it as a key, since the
lexer no longer quotes it for us.
(string_content): Don't strip quotes, since lexer no longer
quotes it for us.
* src/scan-gram.l: Include quotearg.h.
("\""): Omit quote.
("'"<SC_ESCAPED_CHARACTER>): Quote symbol before using it as
a key, since the rest of the lexer doesn't quote it.
* src/symtab.c (symbol_get): Don't quote symbol; caller does it now.
* tests/regression.at (Token definitions): Check for backslashes
in token strings.
escaped_file_name_output, since we now use it for symbol tags as
well. All uses changed.
(symbol_destructors_output, symbol_printers_output):
Escape symbol tags too.
Problem reported by Matyas Forstner in
<http://lists.gnu.org/archive/html/bug-bison/2005-03/msg00009.html>.
* src/muscle_tab.c (muscle_code_grow): Don't quote numbers; it's
not needed.
* src/output.c (user_actions_output, token_definitions_output,
symbol_destructors_output, symbol_printers_output): Likewise.
* src/reader.c (prologue_augment): Likewise.
* src/scan-gram.l (handle_action_dollar, handle_action_at): Likewise.
(declaration): Replace expected_conflicts with expected_sr_conflicts.
Add %expect-rr rule.
* src/scan-gram.l: Recognize %expect-rr.
* src/conflicts.h (expected_sr_conflicts): Rename from
expected_conflicts.
(expected_rr_conflicts): Declare.
* src/conflicts.c (expected_sr_conflicts): Rename from
expected_conflicts.
(expected_rr_conflicts): Define.
(conflicts_print): Check r/r conflicts against expected_rr_conflicts
for GLR parsers.
Use expected_sr_conflicts in place of expected_conflicts.
Warn if expected_rr_conflicts used in non-GLR parser.
* doc/bison.texinfo: Add documentation for %expect-rr.
location that is not defined. This results in garbage, and that
affects Bison's own parser. Therefore we need (i) to extend Bison
to support a means to initialize this location, and (ii) to use
this CVS Bison to fix CVS Bison's parser.
* src/reader.h, reader.c (epilogue_augment): Remove, replace
with...
* src/muscle_tab.h, src/muscle_tab.c (muscle_code_grow): this.
* src/parse-gram.y: Adjust.
(%initial-action): New.
(%error-verbose): Since we require CVS Bison, there is no reason
not to use it.
* src/scan-gram.l: Adjust.
* src/Makefile.am (YACC): New, to make sure we use our own parser.
* data/yacc.c (yyparse): Use b4_initial_action.
(yyrecoverSyntaxError): Correct yyerrState logic. Correct comment.
Allow states with only a default reduction.
Fixes to avoid problem that $-N rules in GLR parsers can cause
buffer overruns, corrupting state.
* src/output.c (prepare_rules): Output max_left_semantic_context.
* src/reader.h (max_left_semantic_context): New
* src/scan-gram.l (max_left_semantic_context): Define.
(handle_action_dollar): Update max_left_semantic_context.
* data/glr.c (YYMAXLEFT): New.
(yydoAction): Increase yyrhsVals size.
(yyresolveAction): Ditto.
Fixes to problems with location handling in GLR parsers reported by
Frank Heckenbach (2003/06/05).
* data/glr.c (YYLTYPE): Make trivial if locations not used.
(YYRHSLOC): Add parentheses, make depend on whether locations used.
(YYLLOC_DEFAULT): Ditto.
(yyuserAction): Use YYLLOC_DEFAULT.
(yydoAction): Remove redundant code.
* tests/cxx-type.at: Exercise location information.
(yylex): Track locations.
(stmtMerge): Return value rather than printing.
(unexpected_eof): Renamed from unexpected_end_of_file, for brevity.
Do not insert the expected token via unput, as this runs afoul
of a POSIX-compatibility bug in flex 2.5.31.
All uses changed to BEGIN the parent state,
since we no longer insert the expected token via unput.
not loc->end, since loc->end might contain garbage and this leads
to undefined behavior on some platforms.
(id_loc, token_start): Use (IF_LINTed) initial values that do not
depend on *loc, so that the reader doesn't give the the false
impression that *loc is initialized.
(<INITIAL>"%%"): Do not bother setting code_start, since its value
does not survive the return.
into yylex, as SC_EPILOGUE is activated *before* the corresponding
yylex invocation. An alternative would be making it static, but
then it starts with the second %%'s beginning, instead of its end.
