Future/bison
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/bison.git synced 2026-03-09 04:13:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

yysyntax_error: fix for consistent error with lookahead.

Browse Source 
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it.  With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage.  Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
pass yyempty_ not yyla.type to yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (parse.error=verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general.  Test
yacc.c, glr.c, lalr1.cc, and lalr1.java.  No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
This commit is contained in:
Joel E. Denny
2010-11-07 16:01:56 -05:00
parent 25a648d8a6
commit d2060f0634
11 changed files with 802 additions and 466 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
ChangeLog
View File

@@ -1,3 +1,39 @@
2010-11-07 Joel E. Denny <jdenny@clemson.edu>
yysyntax_error: fix for consistent error with lookahead.
* NEWS (2.5): Document.
* data/yacc.c (yysyntax_error): In a verbose syntax error
message while in a consistent state with a default action (which
must be an error action given that yysyntax_error is being
invoked), continue to drop the expected token list, but don't
drop the unexpected token unless there actually is no lookahead.
Moreover, handle that internally instead of returning 1 to tell
the caller to do it. With that meaning of 1 gone, renumber
return codes more usefully.
(yyparse, yypush_parse): Update yysyntax_error usage. Most
importantly, set yytoken to YYEMPTY when there's no lookahead.
* data/glr.c (yyreportSyntaxError): As in yacc.c, don't drop the
unexpected token unless there actually is no lookahead.
* data/lalr1.cc (yy::parser::parse): If there's no lookahead,
pass yyempty_ not yyla.type to yysyntax_error_.
(yy::parser::yysyntax_error_): Again, don't drop the unexpected
token unless there actually is no lookahead.
* data/lalr1.java (YYParser::parse): If there's no lookahead,
set yytoken to yyempty_ before invoking yysyntax_error.
(YYParser::yysyntax_error): Again, don't drop the unexpected
token unless there actually is no lookahead.
* tests/conflicts.at (parse.error=verbose and consistent
errors): Extend test group to further reveal how the previous
use of the simple "syntax error" message was too general. Test
yacc.c, glr.c, lalr1.cc, and lalr1.java. No longer an expected
failure.
* tests/java.at (AT_JAVA_COMPILE, AT_JAVA_PARSER_CHECK): Move
to...
* tests/local.at: ... here.
(_AT_BISON_OPTION_PUSHDEFS): Push AT_SKEL_JAVA_IF definition.
(AT_BISON_OPTION_POPDEFS): Pop it.
(AT_FULL_COMPILE): Extend to handle Java.
2010-11-07 Joel E. Denny <jdenny@clemson.edu>
yysyntax_error: more preparation for readability of next patch.

41
NEWS
View File

@@ -223,14 +223,41 @@ Bison News
Bison now warns when a character literal is not of length one. In
some future release, Bison will report an error instead.
** Verbose error messages fixed for nonassociative tokens.
** Verbose syntax error message fixes:
When %error-verbose is specified, syntax error messages produced by
the generated parser include the unexpected token as well as a list of
expected tokens. Previously, this list erroneously included tokens
that would actually induce a syntax error because conflicts for them
were resolved with %nonassoc. Such tokens are now properly omitted
from the list.
When %error-verbose or `#define YYERROR_VERBOSE' is specified, syntax
error messages produced by the generated parser include the unexpected
token as well as a list of expected tokens. The effect of %nonassoc
on these verbose messages has been corrected in two ways, but
additional fixes are still being implemented:
*** When %nonassoc is used, there can exist parser states that accept no
tokens, and so the parser does not always require a lookahead token
in order to detect a syntax error. Because no unexpected token or
expected tokens can then be reported, the verbose syntax error
message described above is suppressed, and the parser instead
reports the simpler message, "syntax error". Previously, this
suppression was sometimes erroneously triggered by %nonassoc when a
lookahead was actually required. Now verbose messages are
suppressed only when all previous lookaheads have already been
shifted or discarded.
*** Previously, the list of expected tokens erroneously included tokens
that would actually induce a syntax error because conflicts for them
were resolved with %nonassoc in the current parser state. Such
tokens are now properly omitted from the list.
*** Expected token lists are still often wrong due to state merging
(from LALR or IELR) and default reductions, which can both add and
subtract valid tokens. Canonical LR almost completely fixes this
problem by eliminating state merging and default reductions.
However, there is one minor problem left even when using canonical
LR and even after the fixes above. That is, if the resolution of a
conflict with %nonassoc appears in a later parser state than the one
at which some syntax error is discovered, the conflicted token is
still erroneously included in the expected token list. We are
currently working on a fix to eliminate this problem and to
eliminate the need for canonical LR.
** Destructor calls fixed for lookaheads altered in semantic actions.

52
data/glr.c
View File

@@ -2081,11 +2081,7 @@ yyreportSyntaxError (yyGLRStack* yystackp]b4_user_formals[)
#if ! YYERROR_VERBOSE
yyerror (]b4_lyyerror_args[YY_("syntax error"));
#else
int yyn;
yyn = yypact[yystackp->yytops.yystates[0]->yylrState];
if (YYPACT_NINF < yyn && yyn <= YYLAST)
{
yySymbol yytoken = YYTRANSLATE (yychar);
yySymbol yytoken = yychar == YYEMPTY ? YYEMPTY : YYTRANSLATE (yychar);
size_t yysize0 = yytnamerr (NULL, yytokenName (yytoken));
size_t yysize = yysize0;
size_t yysize1;
@@ -2096,23 +2092,47 @@ if (YYPACT_NINF < yyn && yyn <= YYLAST)
const char *yyformat = 0;
/* Arguments of yyformat. */
char const *yyarg[YYERROR_VERBOSE_ARGS_MAXIMUM];
/* Number of reported tokens (one for the "unexpected", one per
"expected"). */
int yycount = 0;
/* There are many possibilities here to consider:
- If this state is a consistent state with a default action, then
the only way this function was invoked is if the default action
is an error action. In that case, don't check for expected
tokens because there are none.
- The only way there can be no lookahead present (in yychar) is if
this state is a consistent state with a default action. Thus,
detecting the absence of a lookahead is sufficient to determine
that there is no unexpected or expected token to report. In that
case, just report a simple "syntax error".
- Don't assume there isn't a lookahead just because this state is a
consistent state with a default action. There might have been a
previous inconsistent state, consistent state with a non-default
action, or user semantic action that manipulated yychar.
- Of course, the expected token list depends on states to have
correct lookahead information, and it depends on the parser not
to perform extra reductions after fetching a lookahead from the
scanner and before detecting a syntax error. Thus, state merging
(from LALR or IELR) and default reductions corrupt the expected
token list. However, the list is correct for canonical LR with
one exception: it will still contain any token that will not be
accepted due to an error action in a later state.
*/
if (yytoken != YYEMPTY)
{
int yyn = yypact[yystackp->yytops.yystates[0]->yylrState];
yyarg[yycount++] = yytokenName (yytoken);
if (!yypact_value_is_default (yyn))
{
/* Start YYX at -YYN if negative to avoid negative indexes in
YYCHECK. In other words, skip the first -YYN actions for this
state because they are default actions. */
int yyxbegin = yyn < 0 ? -yyn : 0;
/* Stay within bounds of both yycheck and yytname. */
int yychecklim = YYLAST - yyn + 1;
int yyxend = yychecklim < YYNTOKENS ? yychecklim : YYNTOKENS;
/* Number of reported tokens (one for the "unexpected", one per
"expected"). */
int yycount = 0;
int yyx;
yyarg[yycount++] = yytokenName (yytoken);
for (yyx = yyxbegin; yyx < yyxend; ++yyx)
if (yycheck[yyx + yyn] == yyx && yyx != YYTERROR
&& !yytable_value_is_error (yytable[yyx + yyn]))
@@ -2128,6 +2148,8 @@ if (YYPACT_NINF < yyn && yyn <= YYLAST)
yysize_overflow |= yysize1 < yysize;
yysize = yysize1;
}
}
}
switch (yycount)
{
@@ -2135,6 +2157,7 @@ if (YYPACT_NINF < yyn && yyn <= YYLAST)
case N: \
yyformat = S; \
break
YYCASE_(0, YY_("syntax error"));
YYCASE_(1, YY_("syntax error, unexpected %s"));
YYCASE_(2, YY_("syntax error, unexpected %s, expecting %s"));
YYCASE_(3, YY_("syntax error, unexpected %s, expecting %s or %s"));
@@ -2175,9 +2198,6 @@ if (YYPACT_NINF < yyn && yyn <= YYLAST)
yyerror (]b4_lyyerror_args[YY_("syntax error"));
yyMemoryExhausted (yystackp);
}
}
else
yyerror (]b4_lyyerror_args[YY_("syntax error"));
#endif /* YYERROR_VERBOSE */
yynerrs += 1;
}

60
data/lalr1.cc
View File

@@ -862,7 +862,8 @@ m4_ifdef([b4_lex_param], [, ]b4_lex_param)));])[
{
++yynerrs_;
error (]b4_args(b4_locations_if([yyla.location]),
[[yysyntax_error_ (yystack_[0].state, yyla.type)]])[);
[[yysyntax_error_ (yystack_[0].state,
yyempty ? yyempty_ : yyla.type)]])[);
}
]b4_locations_if([[
@@ -979,26 +980,52 @@ b4_error_verbose_if([state_type yystate, int yytoken],
[int, int])[)
{]b4_error_verbose_if([[
std::string yyres;
int yyn = yypact_[yystate];
if (yypact_ninf_ < yyn && yyn <= yylast_)
// Number of reported tokens (one for the "unexpected", one per
// "expected").
size_t yycount = 0;
// Its maximum.
enum { YYERROR_VERBOSE_ARGS_MAXIMUM = 5 };
// Arguments of yyformat.
char const *yyarg[YYERROR_VERBOSE_ARGS_MAXIMUM];
/* There are many possibilities here to consider:
- If this state is a consistent state with a default action, then
the only way this function was invoked is if the default action
is an error action. In that case, don't check for expected
tokens because there are none.
- The only way there can be no lookahead present (in yytoken) is
if this state is a consistent state with a default action.
Thus, detecting the absence of a lookahead is sufficient to
determine that there is no unexpected or expected token to
report. In that case, just report a simple "syntax error".
- Don't assume there isn't a lookahead just because this state is
a consistent state with a default action. There might have
been a previous inconsistent state, consistent state with a
non-default action, or user semantic action that manipulated
yyla. (However, yyla is currently not documented for users.)
- Of course, the expected token list depends on states to have
correct lookahead information, and it depends on the parser not
to perform extra reductions after fetching a lookahead from the
scanner and before detecting a syntax error. Thus, state
merging (from LALR or IELR) and default reductions corrupt the
expected token list. However, the list is correct for
canonical LR with one exception: it will still contain any
token that will not be accepted due to an error action in a
later state.
*/
if (yytoken != yyempty_)
{
yyarg[yycount++] = yytname_[yytoken];
int yyn = yypact_[yystate];
if (!yy_pact_value_is_default_ (yyn))
{
/* Start YYX at -YYN if negative to avoid negative indexes in
YYCHECK. In other words, skip the first -YYN actions for
this state because they are default actions. */
int yyxbegin = yyn < 0 ? -yyn : 0;
/* Stay within bounds of both yycheck and yytname. */
int yychecklim = yylast_ - yyn + 1;
int yyxend = yychecklim < yyntokens_ ? yychecklim : yyntokens_;
// Number of reported tokens (one for the "unexpected", one per
// "expected").
size_t yycount = 0;
// Its maximum.
enum { YYERROR_VERBOSE_ARGS_MAXIMUM = 5 };
// Arguments of yyformat.
char const *yyarg[YYERROR_VERBOSE_ARGS_MAXIMUM];
yyarg[yycount++] = yytname_[yytoken];
for (int yyx = yyxbegin; yyx < yyxend; ++yyx)
if (yycheck_[yyx + yyn] == yyx && yyx != yyterror_
&& !yy_table_value_is_error_ (yytable_[yyx + yyn]))
@@ -1011,6 +1038,8 @@ b4_error_verbose_if([state_type yystate, int yytoken],
else
yyarg[yycount++] = yytname_[yyx];
}
}
}
char const* yyformat = 0;
switch (yycount)
@@ -1019,6 +1048,7 @@ b4_error_verbose_if([state_type yystate, int yytoken],
case N: \
yyformat = S; \
break
YYCASE_(0, YY_("syntax error"));
YYCASE_(1, YY_("syntax error, unexpected %s"));
YYCASE_(2, YY_("syntax error, unexpected %s, expecting %s"));
YYCASE_(3, YY_("syntax error, unexpected %s, expecting %s or %s"));
@@ -1026,6 +1056,7 @@ b4_error_verbose_if([state_type yystate, int yytoken],
YYCASE_(5, YY_("syntax error, unexpected %s, expecting %s or %s or %s or %s"));
#undef YYCASE_
}
// Argument number.
size_t yyi = 0;
for (char const* yyp = yyformat; *yyp; ++yyp)
@@ -1036,9 +1067,6 @@ b4_error_verbose_if([state_type yystate, int yytoken],
}
else
yyres += *yyp;
}
else
yyres = YY_("syntax error");
return yyres;]], [[
return YY_("syntax error");]])[
}

53
data/lalr1.java
View File

@@ -627,6 +627,8 @@ m4_popdef([b4_at_dollar])])dnl
if (yyerrstatus_ == 0)
{
++yynerrs_;
if (yychar == yyempty_)
yytoken = yyempty_;
yyerror (]b4_locations_if([yylloc, ])[yysyntax_error (yystate, yytoken));
}
@@ -727,17 +729,52 @@ m4_popdef([b4_at_dollar])])dnl
{]b4_error_verbose_if([[
if (yyErrorVerbose)
{
int yyn = yypact_[yystate];
if (yypact_ninf_ < yyn && yyn <= yylast_)
/* There are many possibilities here to consider:
- Assume YYFAIL is not used. It's too flawed to consider.
See
<http://lists.gnu.org/archive/html/bison-patches/2009-12/msg00024.html>
for details. YYERROR is fine as it does not invoke this
function.
- If this state is a consistent state with a default action,
then the only way this function was invoked is if the
default action is an error action. In that case, don't
check for expected tokens because there are none.
- The only way there can be no lookahead present (in tok) is
if this state is a consistent state with a default action.
Thus, detecting the absence of a lookahead is sufficient to
determine that there is no unexpected or expected token to
report. In that case, just report a simple "syntax error".
- Don't assume there isn't a lookahead just because this
state is a consistent state with a default action. There
might have been a previous inconsistent state, consistent
state with a non-default action, or user semantic action
that manipulated yychar. (However, yychar is currently out
of scope during semantic actions.)
- Of course, the expected token list depends on states to
have correct lookahead information, and it depends on the
parser not to perform extra reductions after fetching a
lookahead from the scanner and before detecting a syntax
error. Thus, state merging (from LALR or IELR) and default
reductions corrupt the expected token list. However, the
list is correct for canonical LR with one exception: it
will still contain any token that will not be accepted due
to an error action in a later state.
*/
if (tok != yyempty_)
{
StringBuffer res;
// FIXME: This method of building the message is not compatible
// with internationalization.
StringBuffer res =
new StringBuffer ("syntax error, unexpected ");
res.append (yytnamerr_ (yytname_[tok]));
int yyn = yypact_[yystate];
if (!yy_pact_value_is_default_ (yyn))
{
/* Start YYX at -YYN if negative to avoid negative
indexes in YYCHECK. In other words, skip the first
-YYN actions for this state because they are default
actions. */
int yyxbegin = yyn < 0 ? -yyn : 0;
/* Stay within bounds of both yycheck and yytname. */
int yychecklim = yylast_ - yyn + 1;
int yyxend = yychecklim < yyntokens_ ? yychecklim : yyntokens_;
@@ -746,11 +783,6 @@ m4_popdef([b4_at_dollar])])dnl
if (yycheck_[x + yyn] == x && x != yyterror_
&& !yy_table_value_is_error_ (yytable_[x + yyn]))
++count;
// FIXME: This method of building the message is not compatible
// with internationalization.
res = new StringBuffer ("syntax error, unexpected ");
res.append (yytnamerr_ (yytname_[tok]));
if (count < 5)
{
count = 0;
@@ -762,6 +794,7 @@ m4_popdef([b4_at_dollar])])dnl
res.append (yytnamerr_ (yytname_[x]));
}
}
}
return res.toString ();
}
}

84
data/yacc.c
View File

@@ -906,20 +906,13 @@ yytnamerr (char *yyres, const char *yystr)
/* Copy into *YYMSG, which is of size *YYMSG_ALLOC, an error message
about the unexpected token YYTOKEN while in state YYSTATE.
Return 0 if *YYMSG was successfully written. Return 1 if an ordinary
"syntax error" message will suffice instead. Return 2 if *YYMSG is
not large enough to hold the message. In the last case, also set
*YYMSG_ALLOC to either (a) the required number of bytes or (b) zero
if the required number of bytes is too large to store. */
Return 0 if *YYMSG was successfully written. Return 1 if *YYMSG is
not large enough to hold the message. In that case, also set
*YYMSG_ALLOC to the required number of bytes. Return 2 if the
required number of bytes is too large to store. */
static int
yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg,
int yystate, int yytoken)
{
int yyn = yypact[yystate];
if (! (YYPACT_NINF < yyn && yyn <= YYLAST))
return 1;
else
{
YYSIZE_T yysize0 = yytnamerr (0, yytname[yytoken]);
YYSIZE_T yysize = yysize0;
@@ -929,22 +922,51 @@ yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg,
const char *yyformat = 0;
/* Arguments of yyformat. */
char const *yyarg[YYERROR_VERBOSE_ARGS_MAXIMUM];
/* Number of reported tokens (one for the "unexpected", one per
"expected"). */
int yycount = 0;
/* There are many possibilities here to consider:
- Assume YYFAIL is not used. It's too flawed to consider. See
<http://lists.gnu.org/archive/html/bison-patches/2009-12/msg00024.html>
for details. YYERROR is fine as it does not invoke this
function.
- If this state is a consistent state with a default action, then
the only way this function was invoked is if the default action
is an error action. In that case, don't check for expected
tokens because there are none.
- The only way there can be no lookahead present (in yychar) is if
this state is a consistent state with a default action. Thus,
detecting the absence of a lookahead is sufficient to determine
that there is no unexpected or expected token to report. In that
case, just report a simple "syntax error".
- Don't assume there isn't a lookahead just because this state is a
consistent state with a default action. There might have been a
previous inconsistent state, consistent state with a non-default
action, or user semantic action that manipulated yychar.
- Of course, the expected token list depends on states to have
correct lookahead information, and it depends on the parser not
to perform extra reductions after fetching a lookahead from the
scanner and before detecting a syntax error. Thus, state merging
(from LALR or IELR) and default reductions corrupt the expected
token list. However, the list is correct for canonical LR with
one exception: it will still contain any token that will not be
accepted due to an error action in a later state.
*/
if (yytoken != YYEMPTY)
{
int yyn = yypact[yystate];
yyarg[yycount++] = yytname[yytoken];
if (!yypact_value_is_default (yyn))
{
/* Start YYX at -YYN if negative to avoid negative indexes in
YYCHECK. In other words, skip the first -YYN actions for
this state because they are default actions. */
int yyxbegin = yyn < 0 ? -yyn : 0;
/* Stay within bounds of both yycheck and yytname. */
int yychecklim = YYLAST - yyn + 1;
int yyxend = yychecklim < YYNTOKENS ? yychecklim : YYNTOKENS;
/* Number of reported tokens (one for the "unexpected", one per
"expected"). */
int yycount = 0;
int yyx;
yyarg[yycount++] = yytname[yytoken];
for (yyx = yyxbegin; yyx < yyxend; ++yyx)
if (yycheck[yyx + yyn] == yyx && yyx != YYTERROR
&& !yytable_value_is_error (yytable[yyx + yyn]))
@@ -957,14 +979,13 @@ yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg,
}
yyarg[yycount++] = yytname[yyx];
yysize1 = yysize + yytnamerr (0, yytname[yyx]);
if (! (yysize <= yysize1 && yysize1 <= YYSTACK_ALLOC_MAXIMUM))
{
/* Overflow. */
*yymsg_alloc = 0;
return 2;
}
if (! (yysize <= yysize1
&& yysize1 <= YYSTACK_ALLOC_MAXIMUM))
return 2;
yysize = yysize1;
}
}
}
switch (yycount)
{
@@ -972,6 +993,7 @@ yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg,
case N: \
yyformat = S; \
break
YYCASE_(0, YY_("syntax error"));
YYCASE_(1, YY_("syntax error, unexpected %s"));
YYCASE_(2, YY_("syntax error, unexpected %s, expecting %s"));
YYCASE_(3, YY_("syntax error, unexpected %s, expecting %s or %s"));
@@ -982,11 +1004,7 @@ yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg,
yysize1 = yysize + yystrlen (yyformat);
if (! (yysize <= yysize1 && yysize1 <= YYSTACK_ALLOC_MAXIMUM))
{
/* Overflow. */
*yymsg_alloc = 0;
return 2;
}
return 2;
yysize = yysize1;
if (*yymsg_alloc < yysize)
@@ -995,7 +1013,7 @@ yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg,
if (! (yysize <= *yymsg_alloc
&& *yymsg_alloc <= YYSTACK_ALLOC_MAXIMUM))
*yymsg_alloc = YYSTACK_ALLOC_MAXIMUM;
return 2;
return 1;
}
/* Avoid sprintf, as that infringes on the user's name space.
@@ -1018,7 +1036,6 @@ yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg,
}
return 0;
}
}
#endif /* YYERROR_VERBOSE */
@@ -1467,7 +1484,7 @@ yyreduce:
yyerrlab:
/* Make sure we have latest lookahead translation. See comments at
user semantic actions for why this is necessary. */
yytoken = YYTRANSLATE (yychar);
yytoken = yychar == YYEMPTY ? YYEMPTY : YYTRANSLATE (yychar);
/* If not already recovering from an error, report this error. */
if (!yyerrstatus)
@@ -1483,7 +1500,7 @@ yyerrlab:
int yysyntax_error_status = YYSYNTAX_ERROR;
if (yysyntax_error_status == 0)
yymsgp = yymsg;
else if (yysyntax_error_status == 2 && 0 < yymsg_alloc)
else if (yysyntax_error_status == 1)
{
if (yymsg != yymsgbuf)
YYSTACK_FREE (yymsg);
@@ -1492,6 +1509,7 @@ yyerrlab:
{
yymsg = yymsgbuf;
yymsg_alloc = sizeof yymsgbuf;
yysyntax_error_status = 2;
}
else
{

666
src/parse-gram.c
View File

File diff suppressed because it is too large Load Diff

10
src/parse-gram.h
View File

@@ -31,7 +31,7 @@
This special exception was added by the Free Software Foundation in
version 2.2 of Bison. */
/* "%code requires" blocks. */
/* Line 1663 of yacc.c */
/* Line 1681 of yacc.c */
#line 202 "src/parse-gram.y"
# ifndef PARAM_TYPE
@@ -46,7 +46,7 @@
# endif
/* Line 1663 of yacc.c */
/* Line 1681 of yacc.c */
#line 51 "src/parse-gram.h"
/* Tokens. */
@@ -175,7 +175,7 @@
#if ! defined YYSTYPE && ! defined YYSTYPE_IS_DECLARED
typedef union YYSTYPE
{
/* Line 1663 of yacc.c */
/* Line 1681 of yacc.c */
#line 88 "src/parse-gram.y"
assoc assoc;
@@ -188,13 +188,13 @@ typedef union YYSTYPE
uniqstr uniqstr;
unsigned char character;
/* Line 1663 of yacc.c */
/* Line 1681 of yacc.c */
#line 226 "src/parse-gram.y"
param_type param;
/* Line 1663 of yacc.c */
/* Line 1681 of yacc.c */
#line 199 "src/parse-gram.h"
} YYSTYPE;
# define YYSTYPE_IS_TRIVIAL 1

200
tests/conflicts.at
View File

@@ -147,15 +147,30 @@ AT_SETUP([[parse.error=verbose and consistent errors]])
m4_pushdef([AT_CONSISTENT_ERRORS_CHECK], [
AT_DATA_GRAMMAR([input.y],
[[%code {
AT_BISON_OPTION_PUSHDEFS([$1])
m4_pushdef([AT_YYLEX_PROTOTYPE],
[AT_SKEL_CC_IF([[int yylex (yy::parser::semantic_type *lvalp)]],
[[int yylex (YYSTYPE *lvalp)]])])
AT_SKEL_JAVA_IF([AT_DATA], [AT_DATA_GRAMMAR])([input.y],
[AT_SKEL_JAVA_IF([[
%code imports {
import java.io.IOException;
}]], [[
%code {]AT_SKEL_CC_IF([[
#include <string>]], [[
#include <assert.h>
#include <stdio.h>
int yylex (void);
void yyerror (char const *);
void yyerror (char const *msg);]])[
]AT_YYLEX_PROTOTYPE[;
#define USE(Var)
}
]AT_SKEL_CC_IF([[%defines]], [[%define api.pure]])])[
]$1[
%define parse.error verbose
@@ -164,63 +179,193 @@ AT_DATA_GRAMMAR([input.y],
]$2[
%%
]AT_SKEL_JAVA_IF([[%code lexer {]], [[%%]])[
int
yylex (void)
/*--------.
| yylex. |
`--------*/]AT_SKEL_JAVA_IF([[
public String input = "]$3[";
public int index = 0;
public int yylex ()
{
if (index < input.length ())
return input.charAt (index++);
else
return 0;
}
public Object getLVal ()
{
return new Integer(1);
}]], [[
]AT_YYLEX_PROTOTYPE[
{
static char const *input = "]$3[";
yylval = 1;
*lvalp = 1;
return *input++;
}]])[
/*----------.
| yyerror. |
`----------*/]AT_SKEL_JAVA_IF([[
public void yyerror (String msg)
{
System.err.println (msg);
}
};
%%]], [AT_SKEL_CC_IF([[
void
yy::parser::error (std::string const &msg)
{
std::cerr << msg << std::endl;
}]], [[
void
yyerror (char const *msg)
{
fprintf (stderr, "%s\n", msg);
}
}]])])[
/*-------.
| main. |
`-------*/]AT_SKEL_JAVA_IF([[
class input
{
public static void main (String args[]) throws IOException
{
YYParser p = new YYParser ();
p.parse ();
}
}]], [AT_SKEL_CC_IF([[
int
main (void)
{
yy::parser parser;
return parser.parse ();
}]], [[
int
main (void)
{
return yyparse ();
}
}]])])[
]])
AT_BISON_CHECK([[-o input.c input.y]])
AT_COMPILE([[input]])
AT_FULL_COMPILE([[input]])
m4_pushdef([AT_EXPECTING], [m4_if($5, [ab], [[, expecting 'a' or 'b']],
$5, [a], [[, expecting 'a']],
$5, [b], [[, expecting 'b']])])
AT_PARSER_CHECK([[./input]], [[1]], [[]],
AT_SKEL_JAVA_IF([AT_JAVA_PARSER_CHECK([[input]], [[0]]],
[AT_PARSER_CHECK([[./input]], [[1]]]),
[[]],
[[syntax error, unexpected ]$4[]AT_EXPECTING[
]])
m4_popdef([AT_EXPECTING])
m4_popdef([AT_YYLEX_PROTOTYPE])
AT_BISON_OPTION_POPDEFS
])
m4_pushdef([AT_PREVIOUS_STATE_GRAMMAR],
[[%nonassoc 'a';
start: consistent-error-on-a-a 'a' ;
consistent-error-on-a-a:
'a' default-reduction
| 'a' default-reduction 'a'
| 'a' shift
;
default-reduction: /*empty*/ ;
shift: 'b' ;
// Provide another context in which all rules are useful so that this
// test case looks a little more realistic.
start: 'b' consistent-error-on-a-a 'c' ;
]])
m4_pushdef([AT_PREVIOUS_STATE_INPUT], [[a]])
# Unfortunately, no expected tokens are reported even though 'b' can be
# accepted. Nevertheless, the main point of this test is to make sure
# that at least the unexpected token is reported. In a previous version
# of Bison, it wasn't reported because the error is detected in a
# consistent state with an error action, and that case always triggered
# the simple "syntax error" message.
#
# The point isn't to test IELR here, but state merging happens to
# complicate this example.
AT_CONSISTENT_ERRORS_CHECK([[%define lr.type ielr]],
[AT_PREVIOUS_STATE_GRAMMAR],
[AT_PREVIOUS_STATE_INPUT],
[[$end]], [[none]])
AT_CONSISTENT_ERRORS_CHECK([[%define lr.type ielr
%glr-parser]],
[AT_PREVIOUS_STATE_GRAMMAR],
[AT_PREVIOUS_STATE_INPUT],
[[$end]], [[none]])
AT_CONSISTENT_ERRORS_CHECK([[%define lr.type ielr
%language "c++"]],
[AT_PREVIOUS_STATE_GRAMMAR],
[AT_PREVIOUS_STATE_INPUT],
[[$end]], [[none]])
AT_CONSISTENT_ERRORS_CHECK([[%define lr.type ielr
%language "java"]],
[AT_PREVIOUS_STATE_GRAMMAR],
[AT_PREVIOUS_STATE_INPUT],
[[end of input]], [[none]])
# Even canonical LR doesn't foresee the error for 'a'!
AT_CONSISTENT_ERRORS_CHECK([[%define lr.type ielr
%define lr.default-reductions consistent]],
[AT_PREVIOUS_STATE_GRAMMAR],
[AT_PREVIOUS_STATE_INPUT],
[[$end]], [[ab]])
AT_CONSISTENT_ERRORS_CHECK([[%define lr.type ielr
%define lr.default-reductions accepting]],
[AT_PREVIOUS_STATE_GRAMMAR],
[AT_PREVIOUS_STATE_INPUT],
[[$end]], [[ab]])
AT_CONSISTENT_ERRORS_CHECK([[%define lr.type canonical-lr]],
[AT_PREVIOUS_STATE_GRAMMAR],
[AT_PREVIOUS_STATE_INPUT],
[[$end]], [[ab]])
m4_popdef([AT_PREVIOUS_STATE_GRAMMAR])
m4_popdef([AT_PREVIOUS_STATE_INPUT])
m4_pushdef([AT_USER_ACTION_GRAMMAR],
[[%nonassoc 'a';
// If yylval=0 here, then we know that the 'a' destructor is being
// invoked incorrectly for the 'b' set in the semantic action below.
// All 'a' tokens are returned by yylex, which sets yylval=1.
// If $$ = 0 here, then we know that the 'a' destructor is being invoked
// incorrectly for the 'b' set in the semantic action below. All 'a'
// tokens are returned by yylex, which sets $$ = 1.
%destructor {
if (!$$)
fprintf (stderr, "Wrong destructor.\n");
} 'a';
// The lookahead assigned by the semantic action isn't needed before
// either error action is encountered. In a previous version of Bison,
// this was a problem as it meant yychar was not translated into yytoken
// before either error action. The second error action thus invoked a
// Rather than depend on an inconsistent state to induce reading a
// lookahead as in the previous grammar, just assign the lookahead in a
// semantic action. That lookahead isn't needed before either error
// action is encountered. In a previous version of Bison, this was a
// problem as it meant yychar was not translated into yytoken before
// either error action. The second error action thus invoked a
// destructor that it selected according to the incorrect yytoken. The
// first error action would have reported an incorrect unexpected token
// except that, due to another bug, the unexpected token is not reported
// at all because the error action is the default action in a consistent
// state. That bug still needs to be fixed.
// except that, due to the bug described in the previous grammar, the
// unexpected token was not reported at all.
start: error-reduce consistent-error 'a' { USE ($][3); } ;
error-reduce:
@@ -247,13 +392,16 @@ start: 'b' consistent-error 'b' ;
]])
m4_pushdef([AT_USER_ACTION_INPUT], [[aa]])
# See comments in grammar for why this test doesn't succeed.
AT_XFAIL_IF([[:]])
AT_CONSISTENT_ERRORS_CHECK([[]],
[AT_USER_ACTION_GRAMMAR],
[AT_USER_ACTION_INPUT],
[['b']], [[none]])
AT_CONSISTENT_ERRORS_CHECK([[%glr-parser]],
[AT_USER_ACTION_GRAMMAR],
[AT_USER_ACTION_INPUT],
[['b']], [[none]])
# No C++ or Java test because yychar cannot be manipulated by users.
AT_CONSISTENT_ERRORS_CHECK([[%define lr.default-reductions consistent]],
[AT_USER_ACTION_GRAMMAR],
[AT_USER_ACTION_INPUT],

19
tests/java.at
View File

@@ -219,25 +219,6 @@ m4_define([AT_DATA_JAVA_CALC_Y],
])
# AT_JAVA_COMPILE(SOURCE)
# -----------------------
# Compile SOURCES into Java class files. Skip the test if java or javac is
# not installed.
m4_define([AT_JAVA_COMPILE],
[AT_KEYWORDS(java)
AT_CHECK([test -n "$CONF_JAVA" || exit 77
test -n "$CONF_JAVAC" || exit 77])
AT_CHECK([$SHELL ../../../javacomp.sh $1],
0, [ignore], [ignore])])
# AT_JAVA_PARSER_CHECK(COMMAND, EXIT-STATUS, EXPOUT, EXPERR, [PRE])
# -----------------------------------------------------------------
m4_define([AT_JAVA_PARSER_CHECK],
[AT_CHECK([$5 $SHELL ../../../javaexec.sh $1], [$2], [$3], [$4])])
# _AT_CHECK_JAVA_CALC_ERROR(BISON-OPTIONS, INPUT,
# [VERBOSE-AND-LOCATED-ERROR-MESSAGE])
# ---------------------------------------------------------

47
tests/local.at
View File

@@ -80,6 +80,8 @@ m4_pushdef([AT_DEFINES_IF],
[m4_bmatch([$3], [%defines], [$1], [$2])])
m4_pushdef([AT_SKEL_CC_IF],
[m4_bmatch([$3], [%language "[Cc]\+\+"\|%skeleton "[a-z0-9]+\.cc"], [$1], [$2])])
m4_pushdef([AT_SKEL_JAVA_IF],
[m4_bmatch([$3], [%language "[Jj][Aa][Vv][Aa]"\|%skeleton "[a-z0-9]+\.java"], [$1], [$2])])
m4_pushdef([AT_GLR_IF],
[m4_bmatch([$3], [%glr-parser\|%skeleton "glr\.], [$1], [$2])])
m4_pushdef([AT_LALR1_CC_IF],
@@ -189,6 +191,7 @@ m4_popdef([AT_LEXPARAM_IF])
m4_popdef([AT_YACC_IF])
m4_popdef([AT_GLR_IF])
m4_popdef([AT_SKEL_CC_IF])
m4_popdef([AT_SKEL_JAVA_IF])
m4_popdef([AT_GLR_CC_IF])
m4_popdef([AT_LALR1_CC_IF])
m4_popdef([AT_DEFINES_IF])
@@ -404,19 +407,38 @@ AT_CHECK([$BISON_CXX_WORKS], 0, ignore, ignore)
AT_CHECK([$CXX $CXXFLAGS $CPPFLAGS m4_bmatch([$1], [[.]], [], [$LDFLAGS ])-o $1 m4_default([$2], [$1.cc])[]m4_bmatch([$1], [[.]], [], [ $LIBS])],
0, [ignore], [ignore])])
# AT_JAVA_COMPILE(SOURCES)
# ------------------------
# Compile SOURCES into Java class files. Skip the test if java or javac
# is not installed.
m4_define([AT_JAVA_COMPILE],
[AT_KEYWORDS(java)
AT_CHECK([[test -n "$CONF_JAVA" || exit 77
test -n "$CONF_JAVAC" || exit 77]])
AT_CHECK([[$SHELL ../../../javacomp.sh ]$1],
[[0]], [ignore], [ignore])])
# AT_FULL_COMPILE(OUTPUT, [OTHER])
# --------------------------------
# Compile OUTPUT.y to OUTPUT.c or OUTPUT.cc, and compile it to OUTPUT.
# If OTHER is specified, compile OUTPUT-OTHER.c or OUTPUT-OTHER.cc to OUTPUT
# along with it.
# Relies on AT_SKEL_CC_IF.
m4_define([AT_FULL_COMPILE],
[AT_SKEL_CC_IF(
[AT_BISON_CHECK([-o $1.cc $1.y])
AT_COMPILE_CXX([$1]m4_ifval($2, [, [$1.cc $1-$2.cc]]))],
[AT_BISON_CHECK([-o $1.c $1.y])
AT_COMPILE([$1]m4_ifval($2, [, [$1.c $1-$2.c]]))])
# Compile OUTPUT.y to OUTPUT.c, OUTPUT.cc, or OUTPUT.java, and then
# compile it to OUTPUT or OUTPUT.class. If OTHER is specified, compile
# OUTPUT-OTHER.c, OUTPUT-OTHER.cc, or OUTPUT-OTHER.java to OUTPUT or
# OUTPUT.java along with it. Relies on AT_SKEL_CC_IF and
# AT_SKEL_JAVA_IF.
m4_define([AT_FULL_COMPILE], [
AT_SKEL_JAVA_IF([
AT_BISON_CHECK([[-o ]$1[.java ]$1[.y]])
AT_JAVA_COMPILE([$1[.java]]m4_ifval($2,
[[$1[.java ]$1[-]$2[.java]]]))
], [
AT_SKEL_CC_IF([
AT_BISON_CHECK([[-o ]$1[.cc ]$1[.y]])
AT_COMPILE_CXX([$1]m4_ifval($2, [, [$1[.cc ]$1[-]$2[.cc]]]))
], [
AT_BISON_CHECK([[-o ]$1[.c ]$1[.y]])
AT_COMPILE([$1]m4_ifval($2, [, [$1[.c ]$1[-]$2[.c]]]))
])
])
])
@@ -430,6 +452,11 @@ m4_define([AT_FULL_COMPILE],
m4_define([AT_PARSER_CHECK],
[AT_CHECK([$5 $PREPARSER $1], [$2], [$3], [$4])])
# AT_JAVA_PARSER_CHECK(COMMAND, EXIT-STATUS, EXPOUT, EXPERR, [PRE])
# -----------------------------------------------------------------
m4_define([AT_JAVA_PARSER_CHECK],
[AT_CHECK([$5[ $SHELL ../../../javaexec.sh ]$1], [$2], [$3], [$4])])
# AT_TEST_TABLES_AND_PARSE(TITLE, COND-VALUE, TEST-SPEC,
# DECLS, GRAMMAR, INPUT,
# BISON-STDERR, TABLES-OR-LAST-STATE,