This patch contains more fixes to prefer signed to unsigned
integer types, as modern tools like 'gcc -fsanitize=undefined'
can check for signed integer overflow but not unsigned overflow.
* NEWS: Document the API change.
* boostrap.conf (gnulib_modules): Add intprops.
* data/skeletons/glr.c: Include stddef.h and stdint.h,
since this skeleton can assume C99 or later.
(YYSIZEMAX): Now signed, and the minimum of SIZE_MAX and PTRDIFF_MAX.
(yybool) [!__cplusplus]: Now signed (which is how bool behaves).
(YYTRANSLATE): Avoid use of unsigned, and make the macro
safe even for values greater than UINT_MAX.
(yytnamerr, struct yyGLRState, struct yyGLRStateSet, struct yyGLRStack)
(yyaddDeferredAction, yyinitStateSet, yyinitGLRStack)
(yyexpandGLRStack, yymarkStackDeleted, yyremoveDeletes)
(yyglrShift, yyglrShiftDefer, yy_reduce_print, yydoAction)
(yyglrReduce, yysplitStack, yyreportTree, yycompressStack)
(yyprocessOneStack, yyreportSyntaxError, yyrecoverSyntaxError)
(yyparse, yy_yypstack, yypstack, yypdumpstack):
* tests/input.at (Torturing the Scanner):
Prefer ptrdiff_t to size_t.
* data/skeletons/c++.m4 (b4_yytranslate_define):
* src/AnnotationList.c (AnnotationList__computePredecessorAnnotations):
* src/AnnotationList.h (AnnotationIndex):
* src/InadequacyList.h (InadequacyListNodeCount):
* src/closure.c (closure_new):
* src/complain.c (error_message, complains, complain_indent)
(complain_args, duplicate_directive, duplicate_rule_directive):
* src/gram.c (nritems, ritem_print, grammar_dump):
* src/ielr.c (ielr_compute_ritem_sees_lookahead_set)
(ielr_item_has_lookahead, ielr_compute_annotation_lists)
(ielr_compute_lookaheads):
* src/location.c (columns, boundary_print, location_print):
* src/muscle-tab.c (muscle_percent_define_insert)
(muscle_percent_define_check_values):
* src/output.c (prepare_rules, prepare_actions):
* src/parse-gram.y (id, handle_require):
* src/reader.c (record_merge_function_type, packgram):
* src/reduce.c (nuseless_productions, nuseless_nonterminals)
(inaccessable_symbols):
* src/relation.c (relation_print):
* src/scan-code.l (variant, variant_table_size, variant_count)
(variant_add, get_at_spec, show_sub_message, show_sub_messages)
(parse_ref):
* src/scan-gram.l (<SC_ESCAPED_STRING,SC_ESCAPED_CHARACTER>)
(scan_integer, convert_ucn_to_byte, handle_syncline):
* src/scan-skel.l (at_complain):
* src/symtab.c (complain_symbol_redeclared)
(complain_semantic_type_redeclared, complain_class_redeclared)
(symbol_class_set, complain_user_token_number_redeclared):
* src/tables.c (conflict_tos, conflrow, conflict_table)
(conflict_list, save_row, pack_vector):
* tests/local.at (AT_YYLEX_DEFINE(c)):
Prefer signed to unsigned integer.
* data/skeletons/lalr1.cc (yy_lac_check_):
* tests/actions.at (_AT_CHECK_PRINTER_AND_DESTRUCTOR):
* tests/local.at (AT_YYLEX_DEFINE(c)):
Omit now-unnecessary casts.
* data/skeletons/location.cc (b4_location_define):
* doc/bison.texi (Mfcalc Lexer, C++ position, C++ location):
Prefer int to unsigned for line and column numbers.
Change example to abort explicitly on memory exhaustion,
and fix an off-by-one bug that led to undefined behavior.
* data/skeletons/stack.hh (stack::operator[]):
Also allow ptrdiff_t indexes.
(stack::pop, slice::slice, slice::operator[]):
Index arg is now ptrdiff_t, not int.
(stack::ssize): New method.
(slice::range_): Now ptrdiff_t, not int.
* data/skeletons/yacc.c (b4_state_num_type): Remove.
All uses replaced by b4_int_type.
(YY_CONVERT_INT_BEGIN, YY_CONVERT_INT_END): New macros.
(yylac, yyparse): Use them around conversions that -Wconversion
would give false alarms about. Omit unnecessary casts.
(yy_stack_print): Use int rather than unsigned, and omit
a cast that doesn’t seem to be needed here any more.
* examples/c++/variant.yy (yylex):
* examples/c++/variant-11.yy (yylex):
Omit no-longer-needed conversions to unsigned.
* src/InadequacyList.c (InadequacyList__new_conflict):
Don’t assume *node_count is unsigned.
* src/output.c (muscle_insert_unsigned_table):
Remove; no longer used.
* NEWS: Mention this.
* data/skeletons/c.m4 (b4_int_type):
Prefer char if it will do, and prefer signed types to unsigned if
either will do.
* data/skeletons/glr.c (yy_reduce_print): No need to
convert rule line to unsigned long.
(yyrecoverSyntaxError): Put action into an int to
avoid GCC warning of using a char subscript.
* data/skeletons/lalr1.cc (yy_lac_check_, yysyntax_error_):
Prefer ptrdiff_t to size_t.
* data/skeletons/yacc.c (b4_int_type):
Prefer signed types to unsigned if either will do.
* data/skeletons/yacc.c (b4_declare_parser_state_variables):
(YYSTACK_RELOCATE, YYCOPY, yy_lac_stack_realloc, yy_lac)
(yytnamerr, yysyntax_error, yyparse): Prefer ptrdiff_t to size_t.
(YYPTRDIFF_T, YYPTRDIFF_MAXIMUM): New macros.
(YYSIZE_T): Fix "! defined YYSIZE_T" typo.
(YYSIZE_MAXIMUM): Take the minimum of PTRDIFF_MAX and SIZE_MAX.
(YYSIZEOF): New macro.
(YYSTACK_GAP_MAXIMUM, YYSTACK_BYTES, YYSTACK_RELOCATE)
(yy_lac_stack_realloc, yyparse): Use it.
(YYCOPY, yy_lac_stack_realloc): Cast to YYSIZE_T to pacify GCC.
(yy_reduce_print): Use int instead of unsigned long when int
will do.
(yy_lac_stack_realloc): Prefer long to unsigned long when
either will do.
* tests/regression.at: Adjust to these changes.
Currently we properly use the "best" integral type for tables,
including those storing state numbers. However the variables for
state numbers used in yyparse (and its dependencies such as
yy_stack_print) still use int16_t invariably. As a consequence, very
large models overflow these variables.
Let's use the "best" type for these variables too. It turns out that
we can still use 16 bits for twice larger automata: stick to unsigned
types.
However using 'unsigned' when 16 bits are not enough is troublesome
and generates tons of warnings about signedness issues. Instead,
let's use 'int'.
Reported by Tom Kramer.
https://lists.gnu.org/archive/html/bug-bison/2019-09/msg00018.html
* data/skeletons/yacc.c (b4_state_num_type): New.
(yy_state_num): Be computed from YYNSTATES.
* tests/linear: New.
* tests/torture.at (State number type): New.
Use it.
Instead of
#define YYPACT_NINF -130
#define yypact_value_is_default(Yystate) \
(!!((Yystate) == (-130)))
generate
#define YYPACT_NINF (-130)
#define yypact_value_is_default(Yyn) \
((Yyn) == YYPACT_NINF)
* data/skeletons/c.m4 (b4_table_value_equals): Add support for $4.
* data/skeletons/glr.c, data/skeletons/yacc.c: Use it.
Also, use shorter macro argument names, the name of the macro is clear
enough.
* upstream/maint:
c++: add copy ctors for compatibility with the IAR compiler
CI: show git status
CI: disable ICC
tests: pass -jN from Make to the test suite
quotearg: avoid leaks
maint: post-release administrivia
Reported by Andreas Damm.
https://savannah.gnu.org/support/?110032
* data/skeletons/lalr1.cc (stack_symbol_type::operator=): New
overload, const, to please the IAR C++ compiler (version ca 2013).
This changes the traces from
Reading a token:
Now at end of input.
to
Reading a token:
Next token is token $end (7FFEE56E6474)
which is ok. Actually it is even better, as it gives the location
when locations are enabled, and is clearer when rules explicitly use
the EOF token.
* data/skeletons/lalr1.d (yytranslate_): Handle eof here, as is done
in lalr1.cc.
Bison used to feature %raw, documented as follows:
@item %raw
The output file @file{@var{name}.h} normally defines the tokens with
Yacc-compatible token numbers. If this option is specified, the
internal Bison numbers are used instead. (Yacc-compatible numbers start
at 257 except for single character tokens; Bison assigns token numbers
sequentially for all tokens starting at 3.)
Unfortunately, as far as I can tell, it never worked: token numbers
are indeed changed in the generated tables (from external token number
to internal), yet the code was still applying the mapping from
external token numbers to internal token numbers.
This commit reintroduces the feature as it was expected to be.
* data/skeletons/bison.m4 (b4_token_format): When api.token.raw is
enabled, use the internal token number.
* data/skeletons/yacc.c (yytranslate): Don't emit if api.token.raw is
enabled.
(YYTRANSLATE): Adjust.
The CI, with CC='gcc-7 -fsanitize=undefined,address
-fno-omit-frame-pointer', reports:
calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
../../tests/calc.at:867: cat stderr
--- expout 2019-09-05 20:30:37.887257545 +0000
+++ /home/travis/build/bison-3.4.1.72-79a1-dirty/_build/tests/testsuite.dir/at-groups/438/stdout 2019-09-05 20:30:37.887257545 +0000
@@ -1 +1,2 @@
syntax error
+calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
438. calc.at:867: 438. Calculator glr.cc (calc.at:867): FAILED (calc.at:867)
The problem is that yylookaheadNeeds is not initialized in
yyinitStateSet, and when it is copied, the value is not 0 or 1.
* data/skeletons/glr.c (yylookaheadNeeds): Initialize yylookaheadNeeds.
The CI, with CC='gcc-7 -fsanitize=undefined,address
-fno-omit-frame-pointer', reports:
calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
../../tests/calc.at:867: cat stderr
--- expout 2019-09-05 20:30:37.887257545 +0000
+++ /home/travis/build/bison-3.4.1.72-79a1-dirty/_build/tests/testsuite.dir/at-groups/438/stdout 2019-09-05 20:30:37.887257545 +0000
@@ -1 +1,2 @@
syntax error
+calc.cc:1652:50: runtime error: load of value 190, which is not a valid value for type 'bool'
438. calc.at:867: 438. Calculator glr.cc (calc.at:867): FAILED (calc.at:867)
The problem is that yylookaheadNeeds is not initialized in
yyinitStateSet, and when it is copied, the value is not 0 or 1.
* data/skeletons/glr.c (yylookaheadNeeds): Initialize yylookaheadNeeds.
Implement lookahead correction (LAC) for the C++ skeleton. LAC is a
mechanism to make sure that we report the correct list of expected
tokens if a syntax error occurs. So far, LAC was only supported for
the C skeleton "yacc.c".
* data/skeletons/lalr1.cc: Add LAC support.
* doc/bison.texi: Update.
* data/skeletons/yacc.c (yysyntax_error): Change the nesting of `m4`
conditions slightly to make it more readable.
The generated C code stays unchanged.
* configure.ac (DCFLAGS): Pass -g.
* data/skeletons/d.m4 (b4_locations_if): Remove, let bison.m4's one do
its job.
* data/skeletons/lalr1.d (position): Leave filename empty by default.
(position::toString): Don't print empty file names.
(location::this): New ctor.
(location::toString): Match the implementations of C/C++.
(yy_semantic_null): Leave undefined, the previous implementation does
not compile.
* tests/calc.at: Improve the implementation for D.
Enable more checks, in particular using locations.
* tests/local.at (AT_YYERROR_DEFINE(d)): Fix its implementation.
* data/skeletons/lalr1.java: Use more conventional function names for
Java.
Prefer < and <= to => and >.
Use the same approach for m4 quotation as in the other skeletons.
Fix indentation issues.
* tests/calc.at, tests/java.at, tests/javapush.at: Fix quotation style.
(main): Use 'args', not 'argv', the former seems more conventional and
is used elsewhere in Bison.
Prefer character literals to integers to denote characters.
* examples/java/Calc.y: Likewise.
This is an experiment. Maybe more styles will be used (in which case
a short-hand function will be useful), maybe it will be just reverted.
* data/bison-default.css (.traces0): New.
* src/lalr.c (lalr): Use it.
* data/diagnostics.css: Rename as...
* data/bison-default.css: this.
Add the GPL header.
This is the convention followed by Bruno Haible in gettext.
Adjust dependencies.
* src/complain.c (complain_init_color): Use BISON_STYLE instead of
BISON_DIAGNOSTICS_STYLE.
This makes reading the trace slightly easier. It would be very nice
to highlight the "big steps", especially reductions. But this is a
private experiment: do not use it.
* data/diagnostics.css (value): New.
* src/parse-gram.y: Use no delimiters and no c quotation for strings
to facilitate debugging.
(tron, troff, TRACE): New.
Not very elegant, but until there is support for printf-formats in
libtextstyle, it shall be enough.
Reported by Balázs Scheidler.
* data/skeletons/c.m4 (b4_location_type_define): Use api.location.type
if defined.
* doc/bison.texi: Document it.
* tests/local.at (AT_C_IF, AT_LANG_CASE): New.
Support Span in C.
* tests/calc.at (Span): Convert it to be usable in C and C++.
Check api.location.type with yacc.c and glr.c.
Currently, with --no-lines, instead of "#line file line\n", we emit
"\n". Let's emit nothing.
* data/skeletons/bison.m4 (b4_syncline): Emit at end-of-line when enabled.
* data/skeletons/bison.m4, data/skeletons/c.m4, data/skeletons/glr.cc,
* data/skeletons/lalr1.cc, src/output.c: Use dnl after b4_syncline to
avoid spurious empty lines.
* tests/synclines.at (Sync Lines): Make sure that --no-lines is like
grep -v #line.
* tests/calc.at: Make sure that a rich grammar file behaves properly
with %no-lines.
Currently we use the syncline to report errors about a symbol's
destructor/printer. This is not accurate (only file and line), and
this is incorrect: the file name is double quotes (a recent change,
needed to make sure we escape properly double quotes in it). And
worst of all: with --no-line, b4_syncline expands to nothing.
Rather, push the locations into the backend, and use them.
* src/muscle-tab.h, src/muscle-tab.c (muscle_location_grow): Make it
public.
* src/output.c (prepare_symbol_definitions): Use it to pubish the
location of the printer and destructor.
* data/skeletons/lalr1.java: Use complain_at instead of complain.
* tests/java.at (Java invalid directives): Adjust expectations.
* data/skeletons/bison.m4 (b4_symbol_action_location): Remove.
We should not use b4_syncline this way.
Fatal errors are inconvenient, and should be reserved to cases where
we cannot continue. Here, it could even be warnings actually: these
directives will simply be ignored.
* data/skeletons/lalr1.java: Prefer error (b4_complain) to fatal
errors (b4_fatal).
* tests/java.at (Java invalid directives): New.
