c22902e360
In some rare conditions, the generated parser can be wrong when there
are useless tokens.
Reported by Balázs Scheidler.
https://github.com/akimd/bison/issues/72
Balázs managed to prove that the bug was introduced in
commit af1c6f973a
Author: Theophile Ranquet <ranquet@lrde.epita.fr>
Date: Tue Nov 13 10:38:49 2012 +0000
tables: use bitsets for a performance boost
Suggested by Yuri at
<http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00000.html>.
The improvement is marginal for most grammars, but notable for large
grammars (e.g., PosgreSQL's postgre.y), and very large for the
sample.y grammar submitted by Yuri in
http://lists.gnu.org/archive/html/bison-patches/2012-01/msg00012.html.
Measured with --trace=time -fsyntax-only.
parser action tables postgre.y sample.y
Before 0,129 (44%) 37,095 (99%)
After 0,117 (42%) 5,046 (93%)
* src/tables.c (pos): Replace this set of integer coded as an unsorted
array of integers with...
(pos_set): this bitset.
which was implemented long ago, but that I installed only recently
(March 2019), first published in v3.3.90.
That patch introduces a bitset to represent a set of integers. It
managed negative integers by using a (fixed) base (the smallest
integer to represent). It avoided negative accesses into the bitset
by ignoring integers smaller than the base, under the asumption that
these cases correspond to useless tokens that are ignored anyway.
While it turns out to be true for all the test cases in the test suite
(!), Balázs' use case demonstrates that it is not always the case.
So we need to be able to accept negative integers that are smaller
than the current base.
"Amusingly" enough, the aforementioned patch was visibly unsure about
itself:
/* Store PLACE into POS_SET. PLACE might not belong to the set
of possible values for instance with useless tokens. It
would be more satisfying to eliminate the need for this
'if'. */
This commit needs several improvements in the future:
- support from bitset for bit assignment and shifts
- amortized resizing of pos_set
- test cases
* src/tables.c (pos_set_base, pos_set_dump, pos_set_set, pos_set_test):
New.
Use them instead of using bitset_set and bitset_test directly.
GNU Bison is a general-purpose parser generator that converts an annotated context-free grammar into a deterministic LR or generalized LR (GLR) parser employing LALR(1) parser tables. Bison can also generate IELR(1) or canonical LR(1) parser tables. Once you are proficient with Bison, you can use it to develop a wide range of language parsers, from those used in simple desk calculators to complex programming languages.
Bison is upward compatible with Yacc: all properly-written Yacc grammars work with Bison with no change. Anyone familiar with Yacc should be able to use Bison with little trouble. You need to be fluent in C, C++ or Java programming in order to use Bison.
Bison and the parsers it generates are portable, they do not require any specific compilers.
GNU Bison's home page is https://gnu.org/software/bison/.
Installation
Build from git
The README-hacking.md file is about building, modifying and checking Bison. See its "Working from the Repository" section to build Bison from the git repo. Roughly, run:
$ git submodule update --init
$ ./bootstrap
then proceed with the usual configure && make steps.
Build from tarball
See the INSTALL file for generic compilation and installation instructions.
Bison requires GNU m4 1.4.6 or later. See https://ftp.gnu.org/gnu/m4/m4-1.4.6.tar.gz.
Running a non installed bison
Once you ran make, you might want to toy with this fresh bison before
installing it. In that case, do not use src/bison: it would use the
installed files (skeletons, etc.), not the local ones. Use tests/bison.
Colored diagnostics
As an experimental feature, diagnostics are now colored, controlled by the
--color and --style options.
To use them, install the libtextstyle library, 0.20.5 or newer, before configuring Bison. It is available from https://alpha.gnu.org/gnu/gettext/, for instance https://alpha.gnu.org/gnu/gettext/libtextstyle-0.20.5.tar.gz, or as part of Gettext 0.21 or newer, for instance https://ftp.gnu.org/gnu/gettext/gettext-0.21.tar.gz.
The option --color supports the following arguments:
- always, yes: Enable colors.
- never, no: Disable colors.
- auto, tty (default): Enable colors if the output device is a tty.
To customize the styles, create a CSS file, say bison-bw.css, similar to
/* bison-bw.css */
.warning { }
.error { font-weight: 800; text-decoration: underline; }
.note { }
then invoke bison with --style=bison-bw.css, or set the BISON_STYLE
environment variable to bison-bw.css.
In some diagnostics, bison uses libtextstyle to emit special escapes to
generate clickable hyperlinks. The environment variable
NO_TERM_HYPERLINKS can be used to suppress them. This may be useful for
terminal emulators which produce garbage output when they receive the escape
sequence for a hyperlink. Currently (as of 2020), this affects some versions
of emacs, guake, konsole, lxterminal, rxvt, yakuake.
Relocatability
If you pass --enable-relocatable to configure, Bison is relocatable.
A relocatable program can be moved or copied to a different location on the file system. It can also be used through mount points for network sharing. It is possible to make symlinks to the installed and moved programs, and invoke them through the symlink.
See "Enabling Relocatability" in the documentation.
Internationalization
Bison supports two catalogs: one for Bison itself (i.e., for the maintainer-side parser generation), and one for the generated parsers (i.e., for the user-side parser execution). The requirements between both differ: bison needs ngettext, the generated parsers do not. To simplify the build system, neither are installed if ngettext is not supported, even if generated parsers could have been localized. See http://lists.gnu.org/archive/html/bug-bison/2009-08/msg00006.html for more details.
Questions
See the section FAQ in the documentation (doc/bison.info) for frequently
asked questions. The documentation is also available in PDF and HTML,
provided you have a recent version of Texinfo installed: run make pdf or
make html.
If you have questions about using Bison and the documentation does not answer them, please send mail to help-bison@gnu.org.
Bug reports
Please send bug reports to bug-bison@gnu.org. Be sure to include the
version number from bison --version, and a complete, self-contained test
case in each bug report.
Copyright statements
For any copyright year range specified as YYYY-ZZZZ in this package, note that the range specifies every single year in that closed interval.