It is more consistent with --color=html, --color=test, etc.
* src/getargs.h, src/getargs.c (style_debug): Rename as...
(color_debug): this.
(getargs_colors): Rename --style=debug as --color=debug.
Adjust dependencies.
An experimental commit introduced a fix-it hint that changes comments
such as "/* empty */" into %empty. But in some case, because
diagnostics are not necessarily emitted in order, the fixits also come
in disorder, which must never happen, as the fixes are installed in
one pass.
* src/fixits.c (fixits_register): Insert them in order.
Currently we remove the rhs to install %empty instead.
* src/reader.c (grammar_rule_check_and_complete): Insert the missing
%empty in front of the rhs, not in replacement thereof.
* tests/actions.at (Add missing %empty): Check that.
Some members are called foo_location, others are foo_loc. Stick to
the latter.
* src/gram.h, src/location.h, src/location.c, src/output.c,
* src/parse-gram.y, src/reader.h, src/reader.c, src/reduce.c,
* src/scan-gram.l, src/symlist.h, src/symlist.c, src/symtab.h,
* src/symtab.c:
Use _loc consistently, not _location.
symbol_list features a 'location' and a 'sym_loc' member. The former
is expected to be set only for symbol_lists that denote a symbol (not
a type name), and the latter should only denote the location of the
symbol/type name. Yet both are set, and the name "location" is too
unprecise.
* src/symlist.h, src/symlist.c (symbol_list::location): Rename as
rhs_loc for clarity. Move it to the "section" of data valid only
for rules.
* src/reader.c, src/scan-code.l: Adjust.
Revert "build: also generate the graph
reports" (4ec413da32). The problem is
Automake's ylwrap which does not rename y.dot with the appropriate
name. We should completely stop using Automake's support for Yacc,
which is not something I will do right now. So step back.
* Makefile.am (AM_YFLAGS_WITH_LINES): Don't pass --graph.
Because some of our examples use
%C%_reccalc_SOURCES = %D%/parse.y
Automake ships parse.y and parse.c, and possibly parse.h when it
"understands" that there is one. This is not what we want: ship only
parser.y. Yet we still want to use Automake to compile the sources
from parser.y. The easiest seems to use
nodist_%C%_reccalc_SOURCES = %D%/parse.y
together with
dist_reccalc_DATA = %D%/parse.y %D%/scan.l %D%/Makefile %D%/README.md
which guarantees that parse.y is indeed shipped.
* examples/c/calc/local.mk, examples/c/lexcalc/local.mk,
* examples/c/reccalc/local.mk: Always use nodist_*SOURCES for parsers,
let the dist_*_DATA rules do their job.
* cfg.mk: Disable checks where needed (e.g., we do want to check the
behavior with tabs).
(sc_at_parser_check): Remove. Unfortunately since
a11c144609 we no longer use the './'
prefix to run programs in the current directory. That was so that we
could run Java programs like the other, although they are no run with
the `./` prefix (see 967a59d2c0).
As a consequence this sc check no longer makes sense.
However, since now AT_PARSER_CHECK passes the `./` prefix itself, this
sc-check was superfluous.
* examples/c/reccalc/scan.l: Use memcpy, not strncpy.
* src/ielr.c, src/reader.c: Obfuscate "lr(0)" so that the sc-check for
"space before paren" does not fire.
* tests/diagnostics.at: Avoid space-tab, use tab-tab.
This makes reading the trace slightly easier. It would be very nice
to highlight the "big steps", especially reductions. But this is a
private experiment: do not use it.
* data/diagnostics.css (value): New.
* src/parse-gram.y: Use no delimiters and no c quotation for strings
to facilitate debugging.
(tron, troff, TRACE): New.
Not very elegant, but until there is support for printf-formats in
libtextstyle, it shall be enough.
Currently we pass only the columns based on the screen-width, which is
important for the carets. But we don't pass the bytes-based columns,
which is important for the colors. Pass both.
* src/muscle-tab.c (muscle_boundary_grow): Also pass the byte-based column.
* src/location.c (location_caret): Clarify.
(boundary_set_from_string): Adjust to the new format.
* tests/diagnostics.at (Tabulations and multibyte characters from M4): New.
Locations issued from M4 need the byte-based column for the
diagnostics to work properly. Currently they were unassigned, which
typically resulted in partially non-colored diagnostics.
* src/location.c (boundary_set_from_string): Fix the parsed location.
* src/muscle-tab.c (muscle_percent_define_default): Set the byte values.
* tests/diagnostics.at (Locations from M4): New.
This is meant for developers, not end users, that's why I attached it
to --trace.
* src/getargs.h, src/getargs.c (trace_locations): New.
* src/location.c (location_print): Use it.
We were using the gnulib's gettext module with tricks in
bootstrap.conf to avoid useless files. Instead, use gnulib's
gettext-h module.
* .travis.yml: Force Gettext 0.18.3 on Trusty.
* bootstrap.conf: Use gettext-h instead of gettext.
(excluded_files): Remove.
* configure.ac (AM_GNU_GETTEXT_VERSION): Bump to 0.19.
Reported by Balázs Scheidler.
* data/skeletons/c.m4 (b4_location_type_define): Use api.location.type
if defined.
* doc/bison.texi: Document it.
* tests/local.at (AT_C_IF, AT_LANG_CASE): New.
Support Span in C.
* tests/calc.at (Span): Convert it to be usable in C and C++.
Check api.location.type with yacc.c and glr.c.
The "identifier and colon" of a rule is implemented as a single token,
but whose location is only that of the identifier (so that messages
about the lhs of a rule are accurate). When reducing empty rules, the
default location is the single point location on the end of the
previous symbol. As a consequence, when Bison parses a grammar, the
location of the right-hand side of an empty rule is based on the
lhs, *independently of the position of the colon*. And the colon can
be way farther, separated by comments, white spaces, including empty
lines.
As a result, some messages look really bad. For instance:
$ cat foo.y
%%
foo : /* empty */
bar
: /* empty */
gives
$ bison -Wall foo.y
foo.y:2.4: warning: empty rule without %empty [-Wempty-rule]
2 | foo : /* empty */
| ^
foo.y:3.4: warning: empty rule without %empty [-Wempty-rule]
3 | bar
| ^
The carets are not at the right column, not even the right line.
This commit passes the colon "again" after the "id colon" token, which
gives more accurate locations for these messages:
$ bison -Wall foo.y
foo.y:2.10: warning: empty rule without %empty [-Wempty-rule]
2 | foo : /* empty */
| ^
foo.y:4.2: warning: empty rule without %empty [-Wempty-rule]
4 | : /* empty */
| ^
* src/scan-gram.l (SC_AFTER_IDENTIFIER): Rollback the colon, so that
we scan it again afterwards.
(INITIAL): Scan colons.
* src/parse-gram.y (COLON): New.
(rules): Parse the colon after the rule's id_colon (and possible
named reference).
* tests/actions.at, tests/conflicts.at, tests/diagnostics.at,
* tests/existing.at: Adjust.
Because the fix-its were ready the character-based columns, but were
applied on byte-based columns, the result with multibyte characters or
tabs could be "interesting". For instance
%fixed-output_files
%fixed_output-files
%fixed-output-files
%define api.prefix {foo}
%no-default-prec
would give
%fixed-%fixed-output-files %fixed_output-files
%fixed-orefix= "foo"
o_default-prec
* src/fixits.c (fixit_print, fixits_run): Work on byte-base columns.
* tests/input.at: Check it.
Currently, when we quote the source file, we indent it with one space,
and preserve tabulations, so there is a discrepancy and the visual
rendering is bad. One way out is to indent with a tab instead of a
space, but then this space can be used for more information. This is
what GCC9 does. Let's play copy cats.
See
https://lists.gnu.org/archive/html/bison-patches/2019-04/msg00025.htmlhttps://developers.redhat.com/blog/2019/03/08/usability-improvements-in-gcc-9/https://gcc.gnu.org/onlinedocs/gccint/Guidelines-for-Diagnostics.html#Guidelines-for-Diagnostics
* src/location.c (location_caret): Prefix quoted lines with the line
number and a pipe, fitting 8 columns.
* tests/actions.at, tests/c++.at, tests/conflicts.at,
* tests/diagnostics.at, tests/input.at, tests/java.at,
* tests/named-refs.at, tests/reduce.at, tests/regression.at,
* tests/sets.at: Adjust expectations.
Partly by "./build-aux/update-test tests/testsuite.dir/*/testsuite.log"
repeatedly, and partly by hand.
This is a pity: efforts were invested in computing correctly the
number of screen columns consumed by multibyte characters, but the
routines that do that were fed by single-byte inputs...
As a consequence Bison never displayed correctly locations when there
are multibyte characters.
* src/scan-gram.l (mbchar): New.
Use it instead of . in the catch-all clause.
* tests/diagnostics.at (Tabulations): Enhance into...
(Tabulations and multibyte characters): this.
Single point locations (equal boundaries) are troublesome, and we were
incorrectly ending the style in their case. Which results in an abort
in libtextstyle.
There is also a confusion between columns as displayed on the
screen (which take into account multibyte characters and tabulations),
and the number of bytes. Counting the screen-column
incrementally (character by character) is uneasy (because of multibyte
characters), and I don't want to maintain a buffer of the current line
when displaying the diagnostic. So I believe the simplest solution is
to track the byte number in addition to the screen column.
* src/location.h, src/location.c (boundary): Add the byte-column.
Adjust dependencies.
* src/getargs.c, src/scan-gram.l: Adjust.
* tests/diagnostics.at: Check zero-width locations.
Enable checking of styles even when libtextstyle is not installed.
* src/getargs.h, src/getargs.c (style_debug): New.
(getargs_colors): Set it when --style=debug.
* src/complain.c (begin_use_class, end_use_class): Use it.
* tests/diagnostics.at: New.
* src/lalr.c: Move logs to a better place to understand the chronology
of events.
* src/symlist.c (symbol_list_syms_print): Don't dump core on type
elements.
The 'regen' commit in Bison's history are a nuisance. They are
especially big because of the #lines. Let's generate our parse
without these lines in the repository, but generate them in the
tarball.
* Makefile.am (AM_YFLAGS_WITH_LINES): New.
(AM_YFLAGS): Use it.
(dist-hook): Regenerate the parser with #lines.
Currently, with --no-lines, instead of "#line file line\n", we emit
"\n". Let's emit nothing.
* data/skeletons/bison.m4 (b4_syncline): Emit at end-of-line when enabled.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, src/output.c: Use dnl after b4_syncline to
avoid spurious empty lines.
* tests/synclines.at (Sync Lines): Make sure that --no-lines is like
grep -v #line.
* tests/calc.at: Make sure that a rich grammar file behaves properly
with %no-lines.
Currently we use the syncline to report errors about a symbol's
destructor/printer. This is not accurate (only file and line), and
this is incorrect: the file name is double quotes (a recent change,
needed to make sure we escape properly double quotes in it). And
worst of all: with --no-line, b4_syncline expands to nothing.
Rather, push the locations into the backend, and use them.
* src/muscle-tab.h, src/muscle-tab.c (muscle_location_grow): Make it
public.
* src/output.c (prepare_symbol_definitions): Use it to pubish the
location of the printer and destructor.
* data/skeletons/lalr1.java: Use complain_at instead of complain.
* tests/java.at (Java invalid directives): Adjust expectations.
* data/skeletons/bison.m4 (b4_symbol_action_location): Remove.
We should not use b4_syncline this way.
Fatal errors are inconvenient, and should be reserved to cases where
we cannot continue. Here, it could even be warnings actually: these
directives will simply be ignored.
* data/skeletons/lalr1.java: Prefer error (b4_complain) to fatal
errors (b4_fatal).
* tests/java.at (Java invalid directives): New.
I never understood why we book ngotos+1 slots for relations between
gotos: there are at most ngotos images, not ngotos+1 (and "includes"
does have cases where a goto is in relation with itself, so it's not
ngotos-1).
Maybe bbf37f2534 explains the +1: a bug
left us register a goto several times on occasion, and the +1 might
have been a means to avoid this problem in most cases. Now that this
bug is addressed, we should no longer overbook memory, if only for the
clarity of the code ("why ngotos+1 instead of ngotos?").
* src/lalr.c: A goto has at most ngotos images, not ngotos+1.
While at it, avoid useless repeated call to map_goto introduced in
bbf37f2534.
Reported by wcventure.
http://lists.gnu.org/archive/html/bug-bison/2019-03/msg00008.html
* src/symtab.c (complain_class_redeclared): Don't print empty
locations.
There can only be empty locations for predefined symbols. And the
only symbol that is lexically available is the error token. So this
appears to be the only possible way to have an error involving an
empty location.
* tests/input.at (Symbol class redefinition): Check it.
The "includes" relation [DeRemer 1982] is between gotos, so of course,
for a given goto, there cannot be more that ngotos (number of gotos)
images. But we manipulate the set of images of a goto as a list,
without checking that an image was not already introduced. So we can
"register" way more images than ngotos, leading to a crash (heap
buffer overflow).
Reported by wcventure.
http://lists.gnu.org/archive/html/bug-bison/2019-03/msg00007.html
For the records, this bug is present in the first committed version of
Bison.
* src/lalr.c (build_relations): Don't insert the same goto several
times.
* tests/sets.at (Build Relations): New.
It seems that not many people read these logs: the error was
introduced in 2001 (3067fbef53),
* src/gram.c (grammar_dump): Fix the headers of the table: remove
duplicate display of "Ritem Range".
While at it, remove duplicate display of the rule number (and remove
an incorrect comment about it: these numbers _are_ equal).
* tests/sets.at (Reduced Grammar): Use useless rule, nterm and token
in the example.
When we update some output format, too many adjustements must be made
by hand. This script updates most tests based on the actual output
made during the tests.
* build-aux/update-test: New.
* src/reader.c (grammar_rule_check_and_complete): When 'p' and 'lhs'
are aliases, prefer the latter, for clarity and consistency.
(grammar_current_rule_begin): Avoid 'p', current_rule suffices.
* src/gram.h, src/gram.c: Comment changes.
ptdr# calc.tab.c
Bruno Haible just added a default implementation of libtextstyle's
interface when the library is not available.
https://lists.gnu.org/archive/html/bison-patches/2019-03/msg00025.html
* gnulib: Update.
* bootstrap.conf: Replace libtextstyle with libtextstyle-optional.
* src/complain.c, src/getargs.c: Remove now useless cpp guards.
The timevar and bitset modules now use the c99 module which causes
$CXX to now include -std=gnu++11 when possible. Unfortunately, G++
4.7 does not implement [[noreturn]] in C++11 mode, so our tests of
glr.cc (which uses _Noreturn) fail with
input.cc:954:1: error: expected unqualified-id before '[' token
right before [[noreturn]]. 4.8 works fine.
* data/skeletons/c.m4 (b4_attribute_define): Do not use [[noreturn]]
with GCC 4.7.
See f8408562f8.
* tests/calc.at: Stop imitating the C API.
Prepare more tests to run in the future.
%verbose works as expected (what a surprise, it's unrelated to the
skeleton...).
The variable spec_defines_file denotes the name of the generated
header. Its name is derived from --defines/%defines, whose name in
turn is derived from the fact that the header, in Yacc, contained the
Not only does the header now contain a lot more than just the token
definitions, but we no longer even generate macros, but an enum...
Let's modernize our vocabulary.
* src/files.h, src/files.c (spec_defines_file): Rename as...
(spec_header_file): this.
Currently when --defines is used, we generate a header, and paste an
exact copy of it into the generated parser implementation file. Let's
provide a means to #include it instead.
We don't do it by default because of the Autotools' ylwrap. This
program wraps invocations of yacc (that uses a fixed output name:
y.tab.c, y.tab.h, y.output) to support a more modern naming
scheme (dir/foo.y -> dir/foo.tab.c, dir/foo.tab.h, etc.). It does
that by renaming the generated files, and then by running sed to
propagate these renamings inside the files themselves.
Unfortunately Automake's Makefiles uses Bison as if it were Yacc (with
--yacc or with -o y.tab.c) and invoke bison via ylwrap. As a
consequence, as far as Bison is concerned, the output files are
y.tab.c and y.tab.h, so it emits '#include "y.tab.h"'. So far, so
good. But now ylwrap processes this '#include "y.tab.h"' into
'#include "dir/foo.tab.h"', which is not guaranteed to always work.
So, let's do the Right Thing when the output file is not y.tab.c, in
which case the user should %define api.header.include. Binding this
behavior to --yacc is tempting, but we recently told people to stop
using --yacc (as it also enables the Yacc warnings), but rather to use
-o y.tab.c.
Yacc.c is the only skeleton concerned: all the others do include their
header.
* data/skeletons/yacc.c (b4_header_include_if): New.
(api.header.include): Provide a default value when the output is not
y.tab.c.
* src/parse-gram.y (api.header.include): Define.
Running with CC='gcc-mp-8 -fsanitize=undefined' revealed Undefined
Behaviors.
https://lists.gnu.org/archive/html/bison-patches/2019-03/msg00008.html
* src/state.c (errs_new): Don't call memcpy with NULL as source.
* src/location.c (add_column_width): Don't assume that the column
argument is nonnegative: the scanner sometimes "backtracks" (e.g., see
ROLLBACK_CURRENT_TOKEN and DEPRECATED) in which case we can have
negative column numbers (temporarily).
Found in test 3 (Invalid inputs).
* data/skeletons/yacc.c: here.
This is more logical for the time stamps, but it's also required by
following patches: the shared declarations are also in charge of
handling api.value.type=union. So far, they are run in the
implementation file in both cases (with or without header). But if we
run them only in the header, then the implementation file is emited
with incorrect support for api.value.type=union.
Arguably we should not have such dependencies. This is because we
have side-effects in our backend (redefining the symbols' type and
type_tag). In the future we should find a better solution for this,
without sacrificing the independence of the backend from bison
itself (i.e., I don't think we should handle api.value.type=union in
bison, leave it to m4).
Currently we generate things like:
#line 683 "src/parse-gram.y" /* yacc.c:316 */
The first part is of course very important: compilers point the users
to their grammar file rather than into the generated parser. The
second part points to the place in the skeletons that generated this
piece of code.
This dependency on the Bison skeletons generates lots of useless 'git
diff'. This location is useless for the regular user (who does not
care about the skeletons) and is actually not useful for Bison
developpers too (I never used this to locate the code in skeletons
that generated output). So disable it completely. If someone thinks
this was actually useful, a %define variable should be provided to
control the level of verbosity of '#line', in replacement of
--no-lines.
So now, generate:
#line 683 "src/parse-gram.y"
* data/skeletons/bison.m4 (b4_sync_end): Emit nothing.
Akim Demaille