Check in m4's output if there are sequences such as m4_foo or b4_foo,
which are probably resulting from incorrect m4 processing.
It actually already is useful:
- it caught a leaking b4_lac_if leaking from glr.c, where LAC is not
supported, hence b4_lac_if is not defined.
- it also caught references to location.hh in position.hh when
location.hh does not exist.
- while making "Code injection" robust to these new warnings (it is
its very purpose to let b4_canary pass unevaluated), I saw that it
did not check lalr1.d, and when adding lalr1.d, it revealed it did
underquote ocurrences of token value types.
* src/scan-skel.l (macro): New abbreviation.
Use it.
* data/skeletons/glr.c: Don't use b4_lac_if, we don't have it.
* data/skeletons/location.cc: Don't generate position.hh when we don't
generate location.hh.
* data/skeletons/d.m4 (b4_basic_symbol_constructor_define): Fix
underquotation.
* data/skeletons/bison.m4 (b4_canary): New.
* tests/input.at (Code injection): Use it, and check lalr1.d too.
It is very helpful to be able to look at the locations in the debug
traces, let's provide the users with (i) a means to do that for their
location types, and (ii) a public macro to print locations when debug
traces are enabled.
* data/skeletons/c.m4 (b4_yy_location_print_define): Rename as...
(b4_yylocation_print_define): this.
Define YYLOCATION_PRINT instead of YY_LOCATION_PRINT.
Ensure backward compatibility for those who might have defined/used
YY_LOCATION_PRINT in spite the warnings.
Adjust dependencies.
* data/skeletons/glr2.cc: We don't use YYLOCATION_PRINT here.
Suggested by Joe Nelson <joe@begriffs.com>.
https://lists.gnu.org/r/help-bison/2020-12/msg00020.html
* data/skeletons/glr.c, data/skeletons/yacc.c (YYNOMEM): New.
Use it.
(yyexhaustedlab): Rename as...
(yynomemlab): this.
* tests/calc.at: Check it.
* doc/bison.texi: Document it.
Fix incorrect statements about non-existing constants for YYERROR etc.
This macro is not exposed to users, make start it with 'YY_'.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/glr2.cc, data/skeletons/lalr1.cc,
* src/parse-gram.c, tests/actions.at, tests/c++.at, tests/headers.at,
* tests/local.at (YYUSE): Rename as...
(YY_USE): this.
Currently deferred reductions are not "verbose" at all: only immediate
reductions are displayed in the YYDEBUG traces. I don't understand
why. Besides it seems actually simpler the install the reduction
traces right around the user action inside yyuserAction rather that
around calls to yyuserAction.
This only trouble is that yyuserAction does not know the stack number
it works on, so we have to pass it. And pass -1 when we are actually
running on a temporary stack.
The glr example, on "T(x) + y;" as input, adds these logs, which
allow to see when the `<cast>` is built:
Stack 0 Entering state 26
Reduced stack 0 by rule 7 (line 108); action deferred. Now in state 7.
Stack 0 Entering state 7
Reading a token
Next token is token '+' (1.6: )
Stack 1 Entering state 27
Reduced stack 1 by rule 13 (line 123); action deferred. Now in state 12.
Stack 1 Entering state 12
Next token is token '+' (1.6: )
Stack 1 dies.
Removing dead stacks.
On stack 0, shifting token '+' (1.6: )
Stack 0 now in state #14
+Reducing stack -1 by rule 6 (line 107):
+ $1 = token identifier (1.3: x)
+-> $$ = nterm expr (1.3: x)
+Reducing stack -1 by rule 7 (line 108):
+ $1 = token typename (1.0: T)
+ $2 = token '(' (1.2: )
+ $3 = nterm expr (1.3: x)
+ $4 = token ')' (1.4: )
+-> $$ = nterm expr (1.0-3: <cast>(x,T))
Returning to deterministic operation.
* data/skeletons/glr.c (yyuserAction): Take yyk as a new argument.
Rename argument yyn as yyrule for clarity.
Log before and after the user action.
Adjust callers to not call YY_REDUCE_PRINT and YY_SYMBOL_PRINT.
The next commit wants to use YY_REDUCE_PRINT above its current
definition. Move it higher.
* data/skeletons/glr.c (yylhsNonterm, YY_REDUCE_PRINT): Make available
earlier.
Reported by Jot Dot.
https://lists.gnu.org/r/help-bison/2020-12/msg00014.html
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): Use
the symbol's slot, not its type.
* examples/c/glr/c++-types.y: Use explicit per-symbol typing together
with api.value.type=union.
(yylex): Use yytoken_kind_t.
Don't generate C code from bison, leave that to the skeletons.
* src/output.c (merger_output): Emit invocations to b4_call_merger.
* data/skeletons/glr.c, data/skeletons/glr2.cc (b4_call_merger): New.
* data/skeletons/glr.c: A bit more doc.
(yypstates): Rename yyst (only occurrence) to yys (commonly used for
yyGLRState).
* data/skeletons/glr2.cc: Ditto.
Prefer '\n' to "\n".
When using glr.cc, the C function yyparse is an internal detail that
should not be exposed. Users might call it by accident (I did).
* data/skeletons/glr.c (yyparse): When used for glr.cc, rename as yy_parse_impl.
* data/skeletons/glr.cc: Adjust.
This should have been part of commit "symbols: stop dealing with YYEMPTY
as b4_symbol(-2, ...)" (cd40ec9526).
Give names to all the special symbols: "eof", "error" and "undef".
* data/skeletons/bison.m4 (b4_symbol): Let `b4_symbol(eof, ...)` mean
`b4_symbol(0, ...)`, `b4_symbol(error, ...)` mean `b4_symbol(1, ...)`,
and , `b4_symbol(undef, ...)` mean `b4_symbol(2, ...)`..
* data/skeletons/c.m4, data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/glr2.cc, data/skeletons/lalr1.cc,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/yacc.c:
Prefer symbols to numbers.
The name "defines" is incorrect, the generated file contains far more
than just #defines.
* src/getargs.h, src/getargs.c (-H, --header): New option.
With optional argument, just like --defines, --xml, etc.
(defines_flag): Rename as...
(header_flag): this.
Adjust dependencies.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/glr2.cc, data/skeletons/lalr1.cc,
* data/skeletons/yacc.c:
Adjust.
* examples, doc/bison.texi: Adjust.
* tests/headers.at, tests/local.at, tests/output.at: Convert most
tests from using --defines to using --header.
* data/skeletons/c++.m4, data/skeletons/glr.c, data/skeletons/lalr1.d,
* data/skeletons/lalr1.java, data/skeletons/yacc.c:
Be more accurate about yychar and yytoken.
Don't name local variables as if they were members.
* upstream/maint:
maint: post-release administrivia
version 3.6.4
glr.cc: don't leak glr.c/glr.cc scaffolding to the user
Some fixes were needed to adjust to recent changes in glr.cc and
glr.c.
* data/skeletons/glr.cc: Stop messing with the user's epilogue to
insert glr.cc code. We need that code to be inserted _before_ the
user's epilogue, not after. So define b4_glr_cc_pre_epilogue.
* data/skeletons/glr.c: Use it.
Until we have a decent reimplementation of glr.cc, we have to use
tricks to shoehorn C++ symbols to the C engine of glr.c. Some of them
are done via #define. Unfortunately in Bison 3.6 some of these we
done in the header file, which broke valid user code.
Reported by Egor Pugin.
https://lists.gnu.org/r/bug-bison/2020-06/msg00003.html
* data/skeletons/glr.cc: Stop playing tricks with b4_pre_epilogue.
(b4_glr_cc_setup, b4_glr_cc_cleanup): New.
Much cleaner way to instal glr.cc's scaffolding around glr.c.
* data/skeletons/glr.c: Adjust to use them.
Teaches bison about a new command line option, --file-prefix-map OLD=NEW
(based on the -ffile-prefix-map option from GCC) which causes it to
replace and file path of OLD in the text of the output file with NEW,
mainly for header guards and comments. The primary use of this is to
make builds reproducible with different input paths, and in particular
the debugging information produced when the source code is compiled. For
example, a distro may know that the bison source code will be located at
"/usr/src/bison" and thus can generate bison files that are reproducible
with the following command:
bison --output=/build/bison/parse.c -d --file-prefix-map=/build/bison/=/usr/src/bison/ parse.y
Importantly, this will change the header guards and #line directives
from:
#ifndef YY_BUILD_BISON_PARSE_H
#line 100 "/build/bison/parse.h"
to
#ifndef YY_USR_SRC_BISON_PARSE_H
#line 100 "/usr/src/bison/parse.h"
which is reproducible.
See https://lists.gnu.org/r/bison-patches/2020-05/msg00016.html
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
* src/files.h, src/files.c (spec_mapped_header_file)
(mapped_dir_prefix, map_file_name, add_prefix_map): New.
* src/getargs.c (-M, --file-prefix-map): New option.
* src/output.c (prepare): Define b4_mapped_dir_prefix and
b4_spec_header_file.
* src/scan-skel.l (@ofile@): Output the mapped file name.
* data/skeletons/glr.c, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, data/skeletons/location.cc,
* data/skeletons/yacc.c:
Adjust.
* doc/bison.texi: Document.
* tests/input.at, tests/output.at: Check.
This should have been done in 3.6, but I wanted to avoid introducing
conflicts into Vincent's work on counterexamples. It turns out it's
completely orthogonal.
* data/README.md, data/skeletons/bison.m4, data/skeletons/c++.m4,
* data/skeletons/c.m4, data/skeletons/glr.c, data/skeletons/java.m4,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/variant.hh, data/skeletons/yacc.c, src/conflicts.c,
* src/derives.c, src/gram.c, src/gram.h, src/output.c,
* src/parse-gram.c, src/parse-gram.y, src/print-xml.c, src/print.c,
* src/reader.c, src/symtab.c, src/symtab.h, tests/input.at,
* tests/types.at:
s/user_token_number/code/g.
Plus minor changes.
The user gives yyexpected_tokens a limit: the max number of tokens she
wants to hear about. That's because an error message that reports a
bazillion of possible tokens is useless.
In that case yyexpected_tokens returned 0, so the user would not know
if there are too many expected tokens or none (yes, that's possible).
There are several ways to tell the user in which situation she's in:
- return some E2MANY, a negative value. Then it makes the pattern
int argsize = yypcontext_expected_tokens (ctx, arg, ARGS_MAX);
if (argsize < 0)
return argsize;
no longer valid, as for E2MANY (i) the user must generate the error
message anyway, and (ii) she should not return E2MANY
- return ARGS_MAX + 1. Then it makes it dangerous for the user, as
she has to iterate update `min (ARGS_MAX, argsize)`.
Returning 0 is definitely simpler and safer for the user, as it tells
her "this is not an error, just generate your message without a list
of expecting tokens". So let's still return 0, but set arg[0] to the
empty token when the list is really empty.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/lalr1.java
* data/skeletons/yacc.c (yyexpected_tokens): Put the empty symbol
first if there are no possible tokens at all.
* examples/c/bistromathic/parse.y: Demonstrate how to use that.
For instance test 386, "glr.cc api.value.type={double}":
types.at:366: $CXX $CXXFLAGS $CPPFLAGS $LDFLAGS -o test test.cc $LIBS
stderr:
test.cc: In function 'ptrdiff_t yysplitStack(yyGLRStack*, ptrdiff_t)':
test.cc:490:4: error: 'PTRDIFF_MAX' was not declared in this scope
(PTRDIFF_MAX < SIZE_MAX ? PTRDIFF_MAX : YY_CAST (ptrdiff_t, SIZE_MAX))
^
test.cc:1805:37: note: in expansion of macro 'YYSIZEMAX'
ptrdiff_t half_max_capacity = YYSIZEMAX / 2 / state_size;
^~~~~~~~~
test.cc:490:4: note: suggested alternative: '__PTRDIFF_MAX__'
(PTRDIFF_MAX < SIZE_MAX ? PTRDIFF_MAX : YY_CAST (ptrdiff_t, SIZE_MAX))
^
test.cc:1805:37: note: in expansion of macro 'YYSIZEMAX'
ptrdiff_t half_max_capacity = YYSIZEMAX / 2 / state_size;
^~~~~~~~~
The failing tests are using glr.cc only, which I don't understand, the
problem is rather in glr.c, so I would expect glr.c tests to also fail.
Reported by Bruno Haible.
https://lists.gnu.org/archive/html/bug-bison/2020-05/msg00053.html
* data/skeletons/yacc.c: Move the block that defines
YYPTRDIFF_T/YYPTRDIFF_MAXIMUM, YYSIZE_T/YYSIZE_MAXIMUM, and
YYSIZEOF to...
* data/skeletons/c.m4 (b4_sizes_types_define): Here.
(b4_c99_int_type): Also take care of the #undefinition of short.
* data/skeletons/yacc.c, data/skeletons/glr.c: Use
b4_sizes_types_define.
* data/skeletons/glr.c: Adjust to use YYPTRDIFF_T/YYPTRDIFF_MAXIMUM,
YYSIZE_T/YYSIZE_MAXIMUM.
I'm quite pleased to see that the tricky case of glr.c was already
prepared by the changes to support syntax_error exceptions. Better
yet, it is actually syntax_error that becomes a special case of the
general pattern: make yytoken be YYERRCODE.
* data/skeletons/glr.c (YYFAULTYTOK): Remove the now useless (Basil)
Faulty token.
Instead, use the error token.
* data/skeletons/lalr1.d, data/skeletons/lalr1.java: When computing
the action, first check the case of the error token.
* tests/calc.at: Check cases for the error token symbols before and
after it.
We will not keep YYERRCODE anyway, it causes backward compatibility
issues. So as a first step, let all the skeletons use that name,
until we have a better one.
* data/skeletons/bison.m4, data/skeletons/glr.c,
* data/skeletons/glr.cc, data/skeletons/lalr1.cc,
* data/skeletons/lalr1.d, data/skeletons/lalr1.java,
* data/skeletons/yacc.c, doc/bison.texi, tests/headers.at,
* tests/input.at:
here.
In C/C++, N_ is a no-op. Define it if the user didn't.
Suggested by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2020-04/msg00010.html
* src/output.c (prepare_symbol_names): Rename has_translations as
has_translations_flag.
* data/skeletons/bison.m4 (b4_has_translations_if): New.
* data/skeletons/java.m4 (b4_trans): Use it.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/yacc.c
(N_): Provide a default definition.
Not all the symbols have a fixed symbol code. UNDEF's one is fixed:
-2.
* data/skeletons/glr.c, data/skeletons/lalr1.cc, data/skeletons/lalr1.d,
* data/skeletons/yacc.c: here.
I have been hesitating a lot before doing it ---after all the user
must not use this kind, so what's the point of showing it in
yytoken_kind_t. And eventually I chose to play it safe with the
typing system and make it possible to use yytoken_kind_t for all the
tokens, even the "empty token".
* data/skeletons/c.m4: Give an id and a tag to YYEMPTY.
(b4_token_enums): Define YYEMPTY.
* data/skeletons/c++.m4 (b4_token_enums): Define YYEMPTY.
* data/skeletons/glr.c, data/skeletons/glr.cc, data/skeletons/yacc.c:
(YYEMPTY): Remove.
Use b4_symbol(-2, id) instead.
The user should think of yypcontext fields as accessible only via
yypcontext_* functions. So let's rename yyexpected_tokens to reflect
that.
Let's _not_ rename yyreport_syntax_error, as the user may define this
function, and is not allowed to access directly the fields of
yypcontext_t: she *must* use the "accessors". This is comparable to
the case of C++/Java where the user defines
parser::report_syntax_error, not parser::context::report_syntax_error.
* data/skeletons/glr.c, data/skeletons/yacc.c (yyexpected_tokens):
Rename as...
(yypcontext_expected_tokens): this.
Adjust dependencies.
Currently EOF is handled in an adhoc way, with a #define YYEOF 0 in
the implementation file. As a result, the user has to define her own
EOF token if she wants to use it, which is a pity.
Give the $end token a visible kind name, YYEOF. Except that in C,
where enums are not scoped, we would have collisions between all the
definitions of YYEOFs in the header files, so in C, make it
<api.PREFIX>EOF.
* data/skeletons/c.m4 (YYEOF): Override its name to avoid collisions.
Unless the user already gave it a different name.
* data/skeletons/glr.c (YYEOF): Remove.
Use ]b4_symbol(0, [id])[ instead.
Add support for "pre_epilogue", for glr.cc.
* data/skeletons/glr.cc: Remove dead code (never emitted #undefs).
* data/skeletons/yacc.c
* src/parse-gram.c
* src/reader.c
* src/symtab.c
* tests/actions.at
* tests/input.at
