Future/bison
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/bison.git synced 2026-03-09 20:33:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix handling of yychar manipulation in user semantic actions.

Browse Source 
The problem was that yacc.c didn't always update the yychar
translation afterwards.  However, other skeletons appear to be
fine.  glr.c appears to already translate yychar before every
use.  lalr1.cc does not define yychar and does not document its
replacement, yyla, for users.  It does provide yyclearin, but
that does not manipulate yyla and thus requires no translation
update.  In lalr1.java, yychar is out of scope during semantic
actions.
* NEWS (2.5): Document.
* data/yacc.c (YYBACKUP): Don't bother translating yychar into
yytoken here.
(yyparse, yypush_parse): Instead, translate before every use of
yytoken, and add comments explaining this approach.
* tests/actions.at (Destroying lookahead assigned by semantic
action): New test group checking that translation happens before
lookahead destructor calls at parser return.  Previously,
incorrect destructors were called.
* tests/conflicts.at (parse.error=verbose and consistent
errors): New test group checking that translation happens at
syntax error detection before the associated verbose error
message and the associated lookahead destructor calls.  While
the destructor call is fixed by this patch, the verbose error
message is currently incorrect due to another bug (see
comments in test group), so this is an expected failure for now.
This commit is contained in:
Joel E. Denny
2009-12-20 02:22:53 -05:00
parent 4395a9ff4f
commit df222dfaf4
7 changed files with 612 additions and 344 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
ChangeLog
View File

@@ -1,3 +1,31 @@
2009-12-20 Joel E. Denny <jdenny@clemson.edu>
Fix handling of yychar manipulation in user semantic actions.
The problem was that yacc.c didn't always update the yychar
translation afterwards. However, other skeletons appear to be
fine. glr.c appears to already translate yychar before every
use. lalr1.cc does not define yychar and does not document its
replacement, yyla, for users. It does provide yyclearin, but
that does not manipulate yyla and thus requires no translation
update. In lalr1.java, yychar is out of scope during semantic
actions.
* NEWS (2.5): Document.
* data/yacc.c (YYBACKUP): Don't bother translating yychar into
yytoken here.
(yyparse, yypush_parse): Instead, translate before every use of
yytoken, and add comments explaining this approach.
* tests/actions.at (Destroying lookahead assigned by semantic
action): New test group checking that translation happens before
lookahead destructor calls at parser return. Previously,
incorrect destructors were called.
* tests/conflicts.at (parse.error=verbose and consistent
errors): New test group checking that translation happens at
syntax error detection before the associated verbose error
message and the associated lookahead destructor calls. While
the destructor call is fixed by this patch, the verbose error
message is currently incorrect due to another bug (see
comments in test group), so this is an expected failure for now.
2009-12-21 Joel E. Denny <jdenny@clemson.edu>
YYFAIL: warn about uses and remove from lalr1.java.

7
NEWS
View File

@@ -223,6 +223,13 @@ Bison News
were resolved with %nonassoc. Such tokens are now properly omitted
from the list.
** Destructor calls fixed for lookaheads altered in semantic actions.
Previously for deterministic parsers in C, if a user semantic action
altered yychar, the parser in some cases used the old yychar value to
determine which destructor to call for the lookahead upon a syntax
error or upon parser return. This bug has been fixed.
* Changes in version 2.4.2 (????-??-??):
** Detection of GNU M4 1.4.6 or newer during configure is improved.

25
data/yacc.c
View File

@@ -630,7 +630,6 @@ do \
{ \
yychar = (Token); \
yylval = (Value); \
yytoken = YYTRANSLATE (yychar); \
YYPOPSTACK (1); \
goto yybackup; \
} \
@@ -1429,6 +1428,17 @@ yyreduce:
]b4_user_actions[
default: break;
}
/* User semantic actions sometimes alter yychar, and that requires
that yytoken be updated with the new translation. We take the
approach of translating immediately before every use of yytoken.
One alternative is translating here after every semantic action,
but that translation would be missed if the semantic action invokes
YYABORT, YYACCEPT, or YYERROR immediately after altering yychar or
if it invokes YYBACKUP. In the case of YYABORT or YYACCEPT, an
incorrect destructor might then be invoked immediately. In the
case of YYERROR or YYBACKUP, subsequent parser actions might lead
to an incorrect destructor call or verbose syntax error message
before the lookahead is translated. */
YY_SYMBOL_PRINT ("-> $$ =", yyr1[yyn], &yyval, &yyloc);
YYPOPSTACK (yylen);
@@ -1457,6 +1467,10 @@ yyreduce:
| yyerrlab -- here on detecting error |
`------------------------------------*/
yyerrlab:
/* Make sure we have latest lookahead translation. See comments at
user semantic actions for why this is necessary. */
yytoken = YYTRANSLATE (yychar);
/* If not already recovering from an error, report this error. */
if (!yyerrstatus)
{
@@ -1609,8 +1623,13 @@ yyexhaustedlab:
yyreturn:
if (yychar != YYEMPTY)
yydestruct ("Cleanup: discarding lookahead",
yytoken, &yylval]b4_locations_if([, &yylloc])[]b4_user_args[);
{
/* Make sure we have latest lookahead translation. See comments at
user semantic actions for why this is necessary. */
yytoken = YYTRANSLATE (yychar);
yydestruct ("Cleanup: discarding lookahead",
yytoken, &yylval]b4_locations_if([, &yylloc])[]b4_user_args[);
}
/* Do not reclaim the symbols of the rule which action triggered
this YYABORT or YYACCEPT. */
YYPOPSTACK (yylen);

689
src/parse-gram.c
View File

File diff suppressed because it is too large Load Diff

12
src/parse-gram.h
View File

@@ -1,4 +1,4 @@
/* A Bison parser, made by GNU Bison 2.4.456-ea2e. */
/* A Bison parser, made by GNU Bison 2.4.457-4395a. */
/* Interface for Bison's Yacc-like parsers in C
@@ -31,7 +31,7 @@
This special exception was added by the Free Software Foundation in
version 2.2 of Bison. */
/* "%code requires" blocks. */
/* Line 1640 of yacc.c */
/* Line 1659 of yacc.c */
#line 202 "src/parse-gram.y"
# ifndef PARAM_TYPE
@@ -46,7 +46,7 @@
# endif
/* Line 1640 of yacc.c */
/* Line 1659 of yacc.c */
#line 51 "src/parse-gram.h"
/* Tokens. */
@@ -173,7 +173,7 @@
#if ! defined YYSTYPE && ! defined YYSTYPE_IS_DECLARED
typedef union YYSTYPE
{
/* Line 1640 of yacc.c */
/* Line 1659 of yacc.c */
#line 88 "src/parse-gram.y"
assoc assoc;
@@ -186,13 +186,13 @@ typedef union YYSTYPE
uniqstr uniqstr;
unsigned char character;
/* Line 1640 of yacc.c */
/* Line 1659 of yacc.c */
#line 226 "src/parse-gram.y"
param_type param;
/* Line 1640 of yacc.c */
/* Line 1659 of yacc.c */
#line 197 "src/parse-gram.h"
} YYSTYPE;
# define YYSTYPE_IS_TRIVIAL 1

71
tests/actions.at
View File

@@ -1408,3 +1408,74 @@ AT_MATCHES_CHECK([input.c], [[// TEST:Y:1 [;{}]*\n;\}$]], [[12]])
AT_MATCHES_CHECK([input.c], [[#define TEST_MACRO_N \\\n\[\]"broken\\" \$ \@ \$\$ \@\$ \[\];\\\nstring;"\}]], [[2]])
AT_CLEANUP
## -------------------------------------------------- ##
## Destroying lookahead assigned by semantic action. ##
## -------------------------------------------------- ##
AT_SETUP([[Destroying lookahead assigned by semantic action]])
AT_DATA_GRAMMAR([input.y],
[[
%code {
#include <assert.h>
#include <stdio.h>
static void yyerror (char const *);
static int yylex (void);
#define USE(Var)
}
%destructor { fprintf (stderr, "'a' destructor\n"); } 'a'
%destructor { fprintf (stderr, "'b' destructor\n"); } 'b'
%%
// In a previous version of Bison, yychar assigned by the semantic
// action below was not translated into yytoken before the lookahead was
// discarded and thus before its destructor (selected according to
// yytoken) was called in order to return from yyparse. This would
// happen even if YYACCEPT was performed in a later semantic action as
// long as only consistent states with default reductions were visited
// in between. However, we leave YYACCEPT in the same semantic action
// for this test in order to show that skeletons cannot simply translate
// immediately after every semantic action because a semantic action
// that has set yychar might not always return normally. Instead,
// skeletons must translate before every use of yytoken.
start: 'a' accept { USE($1); } ;
accept: /*empty*/ {
assert (yychar == YYEMPTY);
yychar = 'b';
YYACCEPT;
} ;
%%
static void
yyerror (char const *msg)
{
fprintf (stderr, "%s\n", msg);
}
static int
yylex (void)
{
static char const *input = "a";
return *input++;
}
int
main (void)
{
return yyparse ();
}
]])
AT_BISON_CHECK([[-o input.c input.y]])
AT_COMPILE([[input]])
AT_PARSER_CHECK([[./input]], [[0]], [],
[['b' destructor
'a' destructor
]])
AT_CLEANUP

124
tests/conflicts.at
View File

@@ -139,6 +139,130 @@ AT_CLEANUP
## ------------------------------------------- ##
## parse.error=verbose and consistent errors. ##
## ------------------------------------------- ##
AT_SETUP([[parse.error=verbose and consistent errors]])
m4_pushdef([AT_CONSISTENT_ERRORS_CHECK], [
AT_BISON_CHECK([$1[ -o input.c input.y]])
AT_COMPILE([[input]])
m4_pushdef([AT_EXPECTING], [m4_if($3, [ab], [[, expecting 'a' or 'b']],
$3, [a], [[, expecting 'a']],
$3, [b], [[, expecting 'b']])])
AT_PARSER_CHECK([[./input]], [[1]], [],
[[syntax error, unexpected ]$2[]AT_EXPECTING[
]])
m4_popdef([AT_EXPECTING])
])
AT_DATA_GRAMMAR([input.y],
[[%code {
#include <assert.h>
#include <stdio.h>
int yylex (void);
void yyerror (char const *);
#define USE(Var)
}
%define parse.error verbose
// The point isn't to test IELR here, but state merging happens to
// complicate the example.
%define lr.type ielr
%nonassoc 'a'
// If yylval=0 here, then we know that the 'a' destructor is being
// invoked incorrectly for the 'b' set in the semantic action below.
// All 'a' tokens are returned by yylex, which sets yylval=1.
%destructor {
if (!$$)
fprintf (stderr, "Wrong destructor.\n");
} 'a'
%%
// The lookahead assigned by the semantic action isn't needed before
// either error action is encountered. In a previous version of Bison,
// this was a problem as it meant yychar was not translated into yytoken
// before either error action. The second error action thus invoked a
// destructor that it selected according to the incorrect yytoken. The
// first error action would have reported an incorrect unexpected token
// except that, due to another bug, the unexpected token is not reported
// at all because the error action is the default action in a consistent
// state. That bug still needs to be fixed.
start: error-reduce consistent-error 'a' { USE ($3); } ;
error-reduce:
'a' 'a' consistent-reduction consistent-error 'a'
{ USE (($1, $2, $5)); }
| 'a' error
{ USE ($1); }
;
consistent-reduction: /*empty*/ {
assert (yychar == YYEMPTY);
yylval = 0;
yychar = 'b';
} ;
consistent-error:
'a' { USE ($1); }
| /*empty*/ %prec 'a'
;
// Provide another context in which all rules are useful so that this
// test case looks a little more realistic.
start: 'b' consistent-error 'b' ;
%%
int
yylex (void)
{
static char const *input = "aa";
yylval = 1;
return *input++;
}
void
yyerror (char const *msg)
{
fprintf (stderr, "%s\n", msg);
}
int
main (void)
{
return yyparse ();
}
]])
# See comments in grammar for why this test doesn't succeed.
AT_XFAIL_IF([[:]])
AT_CONSISTENT_ERRORS_CHECK([], [['b']], [[none]])
AT_CONSISTENT_ERRORS_CHECK([[-Dlr.default-reductions=consistent]],
[['b']], [[none]])
# Canonical LR doesn't foresee the error for 'a'!
AT_CONSISTENT_ERRORS_CHECK([[-Dlr.default-reductions=accepting]],
[[$end]], [[a]])
AT_CONSISTENT_ERRORS_CHECK([[-Flr.type=canonical-lr]], [[$end]], [[a]])
m4_popdef([AT_CONSISTENT_ERRORS_CHECK])
AT_CLEANUP
## ------------------------- ##
## Unresolved SR Conflicts. ##
## ------------------------- ##