version 3.5

* NEWS: Record release date.

.prev-version

@@ -1 +1 @@
-3.4.91
+3.4.92

NEWS

@@ -1,6 +1,18 @@
 GNU Bison NEWS
 
-* Noteworthy changes in release 3.4.92 (2019-12-08) [beta]
+* Noteworthy changes in release 3.5 (2019-12-11) [stable]
+
+** Backward incompatible changes
+
+  Lone carriage-return characters (aka \r or ^M) in the grammar files are no
+  longer treated as end-of-lines.  This changes the diagnostics, and in
+  particular their locations.
+
+  In C++, line numbers and columns are now represented as 'int' not
+  'unsigned', so that integer overflow on positions is easily checkable via
+  'gcc -fsanitize=undefined' and the like.  This affects the API for
+  positions.  The default position and location classes now expose
+  'counter_type' (int), used to define line and column numbers.
 
 ** Deprecated features
 
@@ -8,7 +20,44 @@ GNU Bison NEWS
   obsoleted long ago by %printer, introduced in Bison 1.50 (November 2002).
   It is deprecated and its support will be removed eventually.
 
-** New Features
+** New features
+
+*** Lookahead correction in C++
+
+  Contributed by Adrian Vogelsgesang.
+
+  The C++ deterministic skeleton (lalr1.cc) now supports LAC, via the
+  %define variable parse.lac.
+
+*** Variable api.token.raw: Optimized token numbers (all skeletons)
+
+  In the generated parsers, tokens have two numbers: the "external" token
+  number as returned by yylex (which starts at 257), and the "internal"
+  symbol number (which starts at 3).  Each time yylex is called, a table
+  lookup maps the external token number to the internal symbol number.
+
+  When the %define variable api.token.raw is set, tokens are assigned their
+  internal number, which saves one table lookup per token, and also saves
+  the generation of the mapping table.
+
+  The gain is typically moderate, but in extreme cases (very simple user
+  actions), a 10% improvement can be observed.
+
+*** Generated parsers use better types for states
+
+  Stacks now use the best integral type for state numbers, instead of always
+  using 15 bits.  As a result "small" parsers now have a smaller memory
+  footprint (they use 8 bits), and there is support for large automata (16
+  bits), and extra large (using int, i.e., typically 31 bits).
+
+*** Generated parsers prefer signed integer types
+
+  Bison skeletons now prefer signed to unsigned integer types when either
+  will do, as the signed types are less error-prone and allow for better
+  checking with 'gcc -fsanitize=undefined'.  Also, the types chosen are now
+  portable to unusual machines where char, short and int are all the same
+  width.  On non-GNU platforms this may entail including <limits.h> and (if
+  available) <stdint.h> to define integer types and constants.
 
 *** A skeleton for the D programming language
 
@@ -24,30 +73,12 @@ GNU Bison NEWS
   The lalr1.d skeleton *is functional*, and works well, as demonstrated in
   examples/d/calc.d.  Please try it, enjoy it, and... commit to support it.
 
-** Changes
+*** Debug traces in Java
 
-*** Debugging glr.c and glr.cc
+  The Java backend no longer emits code and data for parser tracing if the
+  %define variable parse.trace is not defined.
 
-  The glr.c skeleton always had asserts to check its own behavior (not the
-  user's).  These assertions are now under the control of the parse.assert
-  %define variable (disabled by default).
-
-*** Clean up
-
-  Several new compiler warnings in the generated output have been avoided.
-  Some unused features are no longer emitted.  Cleaner generated code in
-  general.
-
-** Bug Fixes
-
-*** Crashes when reporting verbose error messages
-
-  In theory, parsers using %nonassoc could crash. This unlikely bug has been
-  fixed.
-
-* Noteworthy changes in release 3.4.91 (2019-11-20) [beta]
-
-** New Features
+** Diagnostics
 
 *** New diagnostic: -Wdangling-alias
 
@@ -98,7 +129,7 @@ GNU Bison NEWS
     %%
     expr:
 
-  gives, with -Wyacc
+  gives with -Wyacc
 
     input.y:2.15-20: warning: POSIX yacc reserves %type to nonterminals [-Wyacc]
         2 | %type  <ival> TOKEN1 TOKEN2 't'
@@ -110,50 +141,9 @@ GNU Bison NEWS
         2 | %type  <ival> TOKEN1 TOKEN2 't'
           |                      ^~~~~~
 
-* Noteworthy changes in release 3.4.90 (2019-10-29) [beta]
-
-** Backward incompatible changes
-
-  Lone carriage-return characters (aka \r or ^M) in the grammar files are no
-  longer treated as end-of-lines.  This changes the diagnostics, and in
-  particular their locations.
-
-  In C++, line numbers and columns are now represented as 'int' not
-  'unsigned', so that integer overflow on positions is easily checkable via
-  'gcc -fsanitize=undefined' and the like.  This affects the API for
-  positions.  The default position and location classes now expose
-  'counter_type' (int), used to define line and column numbers.
-
-** Bug fixes
-
-  In Java, %define api.prefix was ignored.  It now behaves as expected.
-
-** New features
-
-*** Lookahead correction in C++
-
-  Contributed by Adrian Vogelsgesang.
-
-  The C++ deterministic skeleton (lalr1.cc) now supports LAC, via the
-  %define variable parse.lac.
-
-*** Variable api.token.raw: Optimized token numbers (all skeletons)
-
-  In the generated parsers, tokens have two numbers: the "external" token
-  number as returned by yylex (which starts at 257), and the "internal"
-  symbol number (which starts at 3).  Each time yylex is called, a table
-  lookup maps the external token number to the internal symbol number.
-
-  When the %define variable api.token.raw is set, tokens are assigned their
-  internal number, which saves one table lookup per token, and also saves
-  the generation of the mapping table.
-
-  The gain is typically moderate, but in extreme cases (very simple user
-  actions), a 10% improvement can be observed.
-
 *** Diagnostics with insertion
 
-  The diagnostics now display suggestion below the underlined source.
+  The diagnostics now display the suggestion below the underlined source.
   Replacement for undeclared symbols are now also suggested.
 
     $ cat /tmp/foo.y
@@ -194,26 +184,28 @@ GNU Bison NEWS
         1 | %token FOO                     …
           |        ^~~
 
-*** Debug traces in Java
+** Changes
 
-  The Java backend no longer emits code and data for parser tracing if the
-  %define variable parse.trace is not defined.
+*** Debugging glr.c and glr.cc
 
-*** Generated parsers prefer signed integer types
+  The glr.c skeleton always had asserts to check its own behavior (not the
+  user's).  These assertions are now under the control of the parse.assert
+  %define variable (disabled by default).
 
-  Bison skeletons now prefer signed to unsigned integer types when either
-  will do, as the signed types are less error-prone and allow for better
-  checking with 'gcc -fsanitize=undefined'.  Also, the types chosen are now
-  portable to unusual machines where char, short and int are all the same
-  width.  On non-GNU platforms this may entail including <limits.h> and (if
-  available) <stdint.h> to define integer types and constants.
+*** Clean up
 
-*** Generated parsers use better types for states
+  Several new compiler warnings in the generated output have been avoided.
+  Some unused features are no longer emitted.  Cleaner generated code in
+  general.
 
-  Stacks now use the best integral type for state numbers, instead of always
-  using 15 bits.  As a result "small" parsers now have a smaller memory
-  footprint (they use 8 bits), and there is support for large automata (16
-  bits), and extra large (using int, i.e., typically 31 bits).
+** Bug Fixes
+
+  Portability issues in the test suite.
+
+  In theory, parsers using %nonassoc could crash when reporting verbose
+  error messages. This unlikely bug has been fixed.
+
+  In Java, %define api.prefix was ignored.  It now behaves as expected.
 
 * Noteworthy changes in release 3.4.2 (2019-09-12) [stable]
 

TODO

@@ -1,4 +1,4 @@
-* Bison 3.5
+* Bison 3.6
 ** doc
 I feel its ugly to use the GNU style to declare functions in the doc.  It
 generates tons of white space in the page, and may contribute to bad page
@@ -50,12 +50,72 @@ syntax error, unexpected $end, expecting ↦ or 🎅🐃 or '\n'
 While at it, we should stop using "$end" by default, in favor of "end of
 file", or "end of input", whatever.  See how lalr1.java does that.
 
+** consistency
+token vs terminal, variable vs non terminal.
+
+** Stop indentation in diagnostics
+Before Bison 2.7, we printed "flatly" the dependencies in long diagnostics:
+
+    input.y:2.7-12: %type redeclaration for exp
+    input.y:1.7-12: previous declaration
+
+In Bison 2.7, we indented them
+
+    input.y:2.7-12: error: %type redeclaration for exp
+    input.y:1.7-12:     previous declaration
+
+Later we quoted the source in the diagnostics, and today we have:
+
+    /tmp/foo.y:1.12-14: warning: symbol FOO redeclared [-Wother]
+        1 | %token FOO FOO
+          |            ^~~
+    /tmp/foo.y:1.8-10:      previous declaration
+        1 | %token FOO FOO
+          |        ^~~
+
+The indentation is no longer helping.  We should probably get rid of it, or
+maybe keep it only when -fno-caret. GCC displays this as a "note":
+
+    $ g++-mp-9 -Wall /tmp/foo.c -c
+    /tmp/foo.c:1:10: error: redefinition of 'int foo'
+        1 | int foo, foo;
+          |          ^~~
+    /tmp/foo.c:1:5: note: 'int foo' previously declared here
+        1 | int foo, foo;
+          |     ^~~
+
+Likewise for Clang, contrary to what I believed (because "note:" is written
+in black, so it doesn't show in my terminal :-)
+
+    $ clang++-mp-8.0 -Wall /tmp/foo.c -c
+    clang: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
+    /tmp/foo.c:1:10: error: redefinition of 'foo'
+    int foo, foo;
+             ^
+    /tmp/foo.c:1:5: note: previous definition is here
+    int foo, foo;
+        ^
+    1 error generated.
+
+See also the item "Complaint submessage indentation" below.
+
 ** api.token.raw
 Maybe we should exhibit the YYUNDEFTOK token.  It could also be assigned a
 semantic value so that yyerror could be used to report invalid lexemes.
 See also the item "$undefined" below.
 
-* Bison 3.6
+** C++
+Move to int everywhere instead of unsigned?  stack_size, etc.  The parser
+itself uses int (for yylen for instance), yet stack is based on size_t.
+
+Maybe locations should also move to ints.
+
+Paul Eggert already covered most of this.  But before publishing these
+changes, we need to ask our C++ users if they agree with that change, or if
+we need some migration path.  Could be a %define variable, or simply
+%require "3.5".
+
+* Bison 3.7
 ** Unit rules / Injection rules (Akim Demaille)
 Maybe we could expand unit rules (or "injections", see
 https://homepages.cwi.nl/~daybuild/daily-books/syntax/2-sdf/sdf.html), i.e.,
@@ -137,52 +197,6 @@ $ ./tests/testsuite -l | grep errors | sed q
 ** Get rid of YYPRINT and b4_toknum
 Besides yytoknum is wrong when api.token.raw is defined.
 
-** Stop indentation in diagnostics
-Before Bison 2.7, we printed "flatly" the dependencies in long diagnostics:
-
-    input.y:2.7-12: %type redeclaration for exp
-    input.y:1.7-12: previous declaration
-
-In Bison 2.7, we indented them
-
-    input.y:2.7-12: error: %type redeclaration for exp
-    input.y:1.7-12:     previous declaration
-
-Later we quoted the source in the diagnostics, and today we have:
-
-    /tmp/foo.y:1.12-14: warning: symbol FOO redeclared [-Wother]
-        1 | %token FOO FOO
-          |            ^~~
-    /tmp/foo.y:1.8-10:      previous declaration
-        1 | %token FOO FOO
-          |        ^~~
-
-The indentation is no longer helping.  We should probably get rid of it, or
-maybe keep it only when -fno-caret. GCC displays this as a "note":
-
-    $ g++-mp-9 -Wall /tmp/foo.c -c
-    /tmp/foo.c:1:10: error: redefinition of 'int foo'
-        1 | int foo, foo;
-          |          ^~~
-    /tmp/foo.c:1:5: note: 'int foo' previously declared here
-        1 | int foo, foo;
-          |     ^~~
-
-Likewise for Clang, contrary to what I believed (because "note:" is written
-in black, so it doesn't show in my terminal :-)
-
-    $ clang++-mp-8.0 -Wall /tmp/foo.c -c
-    clang: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
-    /tmp/foo.c:1:10: error: redefinition of 'foo'
-    int foo, foo;
-             ^
-    /tmp/foo.c:1:5: note: previous definition is here
-    int foo, foo;
-        ^
-    1 error generated.
-
-See also the item "Complaint submessage indentation" below.
-
 ** Better design for diagnostics
 The current implementation of diagnostics is adhoc, it grew organically.  It
 works as a series of calls to several functions, with dependency of the
@@ -217,20 +231,6 @@ page:
 
 https://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Message-Formatting-Options.html
 
-** consistency
-token vs terminal, variable vs non terminal.
-
-** C++
-Move to int everywhere instead of unsigned?  stack_size, etc.  The parser
-itself uses int (for yylen for instance), yet stack is based on size_t.
-
-Maybe locations should also move to ints.
-
-Paul Eggert already covered most of this.  But before publishing these
-changes, we need to ask our C++ users if they agree with that change, or if
-we need some migration path.  Could be a %define variable, or simply
-%require "3.5".
-
 ** Graphviz display code thoughts
 The code for the --graph option is over two files: print_graph, and
 graphviz. This is because Bison used to also produce VCG graphs, but since
@@ -603,23 +603,6 @@ to bison. If you're interested, I'll work on a patch.
 * Better graphics
 Equip the parser with a means to create the (visual) parse tree.
 
-* Complaint submessage indentation.
-We already have an implementation that works fairly well for named
-reference messages, but it would be nice to use it consistently for all
-submessages from Bison.  For example, the "previous definition"
-submessage or the list of correct values for a %define variable might
-look better with indentation.
-
-However, the current implementation makes the assumption that the
-location printed on the first line is not usually much shorter than the
-locations printed on the submessage lines that follow.  That assumption
-may not hold true as often for some kinds of submessages especially if
-we ever support multiple grammar files.
-
-Here's a proposal for how a new implementation might look:
-
-  http://lists.gnu.org/archive/html/bison-patches/2009-09/msg00086.html
-
 
 Local Variables:
 mode: outline

data/skeletons/bison.m4

@@ -975,8 +975,8 @@ m4_define([b4_percent_code_get],
 [m4_pushdef([b4_macro_name], [[b4_percent_code(]$1[)]])dnl
 m4_ifval([$1], [m4_define([b4_percent_code_bison_qualifiers(]$1[)])])dnl
 m4_ifdef(b4_macro_name,
-[b4_comment([m4_if([$#], [0], [[Unqualified %code]],
-                   [["%code ]$1["]])[ blocks.]])
+[b4_comment(m4_if([$#], [0], [[[Unqualified %code blocks.]]],
+                  [[["%code ]$1[" blocks.]]]))
 b4_user_code([m4_indir(b4_macro_name)])])dnl
 m4_popdef([b4_macro_name])])
 

data/skeletons/c++.m4

@@ -514,7 +514,7 @@ m4_define([b4_yytranslate_define],
   ]b4_parser_class[::yytranslate_ (int t)
   {
 ]b4_api_token_raw_if(
-[[    return static_cast<yy::parser::token_number_type> (t);]],
+[[    return static_cast<token_number_type> (t);]],
 [[    // YYTRANSLATE[TOKEN-NUM] -- Symbol number corresponding to
     // TOKEN-NUM as returned by yylex.
     static

data/skeletons/glr.cc

@@ -265,6 +265,12 @@ b4_percent_code_get([[requires]])[
 # pragma GCC diagnostic ignored "-Wold-style-cast"
 #endif
 
+// On MacOS, PTRDIFF_MAX is defined as long long, which Clang's
+// -pedantic reports as being a C++11 extension.
+#if defined __APPLE__ && YY_CPLUSPLUS < 201103L && 4 <= __clang_major__
+# pragma clang diagnostic ignored "-Wc++11-long-long"
+#endif
+
 // Whether we are compiled with exception support.
 #ifndef YY_EXCEPTIONS
 # if defined __GNUC__ && !defined __EXCEPTIONS

data/skeletons/lalr1.cc

@@ -269,7 +269,7 @@ m4_define([b4_shared_declarations],
 
     /// Convert a scanner token number \a t to a symbol number.
     /// In theory \a t should be a token_type, but character literals
-    /// are valid, yet not member of the token_type enum.
+    /// are valid, yet not members of the token_type enum.
     static token_number_type yytranslate_ (int t);
 
     // Tables.

src/parse-gram.c

@@ -1,4 +1,4 @@
-/* A Bison parser, made by GNU Bison 3.4.91.18-42a6c.  */
+/* A Bison parser, made by GNU Bison 3.4.92.  */
 
 /* Bison implementation for Yacc-like parsers in C
 
@@ -48,7 +48,7 @@
 #define YYBISON 1
 
 /* Bison version.  */
-#define YYBISON_VERSION "3.4.91.18-42a6c"
+#define YYBISON_VERSION "3.4.92"
 
 /* Skeleton name.  */
 #define YYSKELETON_NAME "yacc.c"
@@ -354,13 +354,6 @@ typedef int yy_state_fast_t;
     _Pragma ("GCC diagnostic ignored \"-Wmaybe-uninitialized\"")
 # define YY_IGNORE_MAYBE_UNINITIALIZED_END      \
     _Pragma ("GCC diagnostic pop")
-# if defined __cplusplus
-#  define YY_IGNORE_USELESS_CAST_BEGIN                          \
-     _Pragma ("GCC diagnostic push")                            \
-     _Pragma ("GCC diagnostic ignored \"-Wuseless-cast\"")
-#  define YY_IGNORE_USELESS_CAST_END            \
-     _Pragma ("GCC diagnostic pop")
-# endif
 #else
 # define YY_INITIAL_VALUE(Value) Value
 #endif
@@ -368,13 +361,21 @@ typedef int yy_state_fast_t;
 # define YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN
 # define YY_IGNORE_MAYBE_UNINITIALIZED_END
 #endif
+#ifndef YY_INITIAL_VALUE
+# define YY_INITIAL_VALUE(Value) /* Nothing. */
+#endif
+
+#if defined __cplusplus && defined __GNUC__ && ! defined __ICC && 6 <= __GNUC__
+# define YY_IGNORE_USELESS_CAST_BEGIN                          \
+    _Pragma ("GCC diagnostic push")                            \
+    _Pragma ("GCC diagnostic ignored \"-Wuseless-cast\"")
+# define YY_IGNORE_USELESS_CAST_END            \
+    _Pragma ("GCC diagnostic pop")
+#endif
 #ifndef YY_IGNORE_USELESS_CAST_BEGIN
 # define YY_IGNORE_USELESS_CAST_BEGIN
 # define YY_IGNORE_USELESS_CAST_END
 #endif
-#ifndef YY_INITIAL_VALUE
-# define YY_INITIAL_VALUE(Value) /* Nothing. */
-#endif
 
 
 #define YY_ASSERT(E) ((void) (0 && (E)))
@@ -3063,9 +3064,9 @@ handle_require (location const *loc, char const *version)
       return;
     }
 
-  /* Pretend to be at least 3.4, to check features published in 3.4
-     while developping it.  */
-  const char* api_version = "3.4";
+  /* Pretend to be at least 3.5, to check features published in that
+     version while developping it.  */
+  const char* api_version = "3.5";
   const char* package_version =
     0 < strverscmp (api_version, PACKAGE_VERSION)
     ? api_version : PACKAGE_VERSION;

src/parse-gram.h

@@ -1,4 +1,4 @@
-/* A Bison parser, made by GNU Bison 3.4.91.18-42a6c.  */
+/* A Bison parser, made by GNU Bison 3.4.92.  */
 
 /* Bison interface for Yacc-like parsers in C
 

src/parse-gram.y

@@ -1021,9 +1021,9 @@ handle_require (location const *loc, char const *version)
       return;
     }
 
-  /* Pretend to be at least 3.4, to check features published in 3.4
-     while developping it.  */
-  const char* api_version = "3.4";
+  /* Pretend to be at least 3.5, to check features published in that
+     version while developping it.  */
+  const char* api_version = "3.5";
   const char* package_version =
     0 < strverscmp (api_version, PACKAGE_VERSION)
     ? api_version : PACKAGE_VERSION;