mirror of
https://git.savannah.gnu.org/git/bison.git
synced 2026-03-09 12:23:04 +00:00
7452 lines
256 KiB
Plaintext
7452 lines
256 KiB
Plaintext
\input texinfo @c -*-texinfo-*-
|
|
@comment %**start of header
|
|
@setfilename bison.info
|
|
@include version.texi
|
|
@settitle Bison @value{VERSION}
|
|
@setchapternewpage odd
|
|
|
|
@finalout
|
|
|
|
@c SMALL BOOK version
|
|
@c This edition has been formatted so that you can format and print it in
|
|
@c the smallbook format.
|
|
@c @smallbook
|
|
|
|
@c Set following if you have the new `shorttitlepage' command
|
|
@c @clear shorttitlepage-enabled
|
|
@c @set shorttitlepage-enabled
|
|
|
|
@c Set following if you want to document %default-prec and %no-default-prec.
|
|
@c This feature is experimental and may change in future Bison versions.
|
|
@c @set defaultprec
|
|
|
|
@c ISPELL CHECK: done, 14 Jan 1993 --bob
|
|
|
|
@c Check COPYRIGHT dates. should be updated in the titlepage, ifinfo
|
|
@c titlepage; should NOT be changed in the GPL. --mew
|
|
|
|
@c FIXME: I don't understand this `iftex'. Obsolete? --akim.
|
|
@iftex
|
|
@syncodeindex fn cp
|
|
@syncodeindex vr cp
|
|
@syncodeindex tp cp
|
|
@end iftex
|
|
@ifinfo
|
|
@synindex fn cp
|
|
@synindex vr cp
|
|
@synindex tp cp
|
|
@end ifinfo
|
|
@comment %**end of header
|
|
|
|
@copying
|
|
|
|
This manual is for @acronym{GNU} Bison (version @value{VERSION},
|
|
@value{UPDATED}), the @acronym{GNU} parser generator.
|
|
|
|
Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
|
|
1999, 2000, 2001, 2002, 2003, 2004 Free Software Foundation, Inc.
|
|
|
|
@quotation
|
|
Permission is granted to copy, distribute and/or modify this document
|
|
under the terms of the @acronym{GNU} Free Documentation License,
|
|
Version 1.1 or any later version published by the Free Software
|
|
Foundation; with no Invariant Sections, with the Front-Cover texts
|
|
being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
|
|
(a) below. A copy of the license is included in the section entitled
|
|
``@acronym{GNU} Free Documentation License.''
|
|
|
|
(a) The @acronym{FSF}'s Back-Cover Text is: ``You have freedom to copy
|
|
and modify this @acronym{GNU} Manual, like @acronym{GNU} software.
|
|
Copies published by the Free Software Foundation raise funds for
|
|
@acronym{GNU} development.''
|
|
@end quotation
|
|
@end copying
|
|
|
|
@dircategory GNU programming tools
|
|
@direntry
|
|
* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
|
|
@end direntry
|
|
|
|
@ifset shorttitlepage-enabled
|
|
@shorttitlepage Bison
|
|
@end ifset
|
|
@titlepage
|
|
@title Bison
|
|
@subtitle The Yacc-compatible Parser Generator
|
|
@subtitle @value{UPDATED}, Bison Version @value{VERSION}
|
|
|
|
@author by Charles Donnelly and Richard Stallman
|
|
|
|
@page
|
|
@vskip 0pt plus 1filll
|
|
@insertcopying
|
|
@sp 2
|
|
Published by the Free Software Foundation @*
|
|
59 Temple Place, Suite 330 @*
|
|
Boston, MA 02111-1307 USA @*
|
|
Printed copies are available from the Free Software Foundation.@*
|
|
@acronym{ISBN} 1-882114-44-2
|
|
@sp 2
|
|
Cover art by Etienne Suvasa.
|
|
@end titlepage
|
|
|
|
@contents
|
|
|
|
@ifnottex
|
|
@node Top
|
|
@top Bison
|
|
@insertcopying
|
|
@end ifnottex
|
|
|
|
@menu
|
|
* Introduction::
|
|
* Conditions::
|
|
* Copying:: The @acronym{GNU} General Public License says
|
|
how you can copy and share Bison
|
|
|
|
Tutorial sections:
|
|
* Concepts:: Basic concepts for understanding Bison.
|
|
* Examples:: Three simple explained examples of using Bison.
|
|
|
|
Reference sections:
|
|
* Grammar File:: Writing Bison declarations and rules.
|
|
* Interface:: C-language interface to the parser function @code{yyparse}.
|
|
* Algorithm:: How the Bison parser works at run-time.
|
|
* Error Recovery:: Writing rules for error recovery.
|
|
* Context Dependency:: What to do if your language syntax is too
|
|
messy for Bison to handle straightforwardly.
|
|
* Debugging:: Understanding or debugging Bison parsers.
|
|
* Invocation:: How to run Bison (to produce the parser source file).
|
|
* Table of Symbols:: All the keywords of the Bison language are explained.
|
|
* Glossary:: Basic concepts are explained.
|
|
* FAQ:: Frequently Asked Questions
|
|
* Copying This Manual:: License for copying this manual.
|
|
* Index:: Cross-references to the text.
|
|
|
|
@detailmenu
|
|
--- The Detailed Node Listing ---
|
|
|
|
The Concepts of Bison
|
|
|
|
* Language and Grammar:: Languages and context-free grammars,
|
|
as mathematical ideas.
|
|
* Grammar in Bison:: How we represent grammars for Bison's sake.
|
|
* Semantic Values:: Each token or syntactic grouping can have
|
|
a semantic value (the value of an integer,
|
|
the name of an identifier, etc.).
|
|
* Semantic Actions:: Each rule can have an action containing C code.
|
|
* GLR Parsers:: Writing parsers for general context-free languages.
|
|
* Simple GLR Parsers:: Using GLR in its simplest form.
|
|
* Locations Overview:: Tracking Locations.
|
|
* Bison Parser:: What are Bison's input and output,
|
|
how is the output used?
|
|
* Stages:: Stages in writing and running Bison grammars.
|
|
* Grammar Layout:: Overall structure of a Bison grammar file.
|
|
|
|
Examples
|
|
|
|
* RPN Calc:: Reverse polish notation calculator;
|
|
a first example with no operator precedence.
|
|
* Infix Calc:: Infix (algebraic) notation calculator.
|
|
Operator precedence is introduced.
|
|
* Simple Error Recovery:: Continuing after syntax errors.
|
|
* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
|
|
* Multi-function Calc:: Calculator with memory and trig functions.
|
|
It uses multiple data-types for semantic values.
|
|
* Exercises:: Ideas for improving the multi-function calculator.
|
|
|
|
Reverse Polish Notation Calculator
|
|
|
|
* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
|
|
* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
|
|
* Lexer: Rpcalc Lexer. The lexical analyzer.
|
|
* Main: Rpcalc Main. The controlling function.
|
|
* Error: Rpcalc Error. The error reporting function.
|
|
* Gen: Rpcalc Gen. Running Bison on the grammar file.
|
|
* Comp: Rpcalc Compile. Run the C compiler on the output code.
|
|
|
|
Grammar Rules for @code{rpcalc}
|
|
|
|
* Rpcalc Input::
|
|
* Rpcalc Line::
|
|
* Rpcalc Expr::
|
|
|
|
Location Tracking Calculator: @code{ltcalc}
|
|
|
|
* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
|
|
* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
|
|
* Lexer: Ltcalc Lexer. The lexical analyzer.
|
|
|
|
Multi-Function Calculator: @code{mfcalc}
|
|
|
|
* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
|
|
* Rules: Mfcalc Rules. Grammar rules for the calculator.
|
|
* Symtab: Mfcalc Symtab. Symbol table management subroutines.
|
|
|
|
Bison Grammar Files
|
|
|
|
* Grammar Outline:: Overall layout of the grammar file.
|
|
* Symbols:: Terminal and nonterminal symbols.
|
|
* Rules:: How to write grammar rules.
|
|
* Recursion:: Writing recursive rules.
|
|
* Semantics:: Semantic values and actions.
|
|
* Locations:: Locations and actions.
|
|
* Declarations:: All kinds of Bison declarations are described here.
|
|
* Multiple Parsers:: Putting more than one Bison parser in one program.
|
|
|
|
Outline of a Bison Grammar
|
|
|
|
* Prologue:: Syntax and usage of the prologue.
|
|
* Bison Declarations:: Syntax and usage of the Bison declarations section.
|
|
* Grammar Rules:: Syntax and usage of the grammar rules section.
|
|
* Epilogue:: Syntax and usage of the epilogue.
|
|
|
|
Defining Language Semantics
|
|
|
|
* Value Type:: Specifying one data type for all semantic values.
|
|
* Multiple Types:: Specifying several alternative data types.
|
|
* Actions:: An action is the semantic definition of a grammar rule.
|
|
* Action Types:: Specifying data types for actions to operate on.
|
|
* Mid-Rule Actions:: Most actions go at the end of a rule.
|
|
This says when, why and how to use the exceptional
|
|
action in the middle of a rule.
|
|
|
|
Tracking Locations
|
|
|
|
* Location Type:: Specifying a data type for locations.
|
|
* Actions and Locations:: Using locations in actions.
|
|
* Location Default Action:: Defining a general way to compute locations.
|
|
|
|
Bison Declarations
|
|
|
|
* Token Decl:: Declaring terminal symbols.
|
|
* Precedence Decl:: Declaring terminals with precedence and associativity.
|
|
* Union Decl:: Declaring the set of all semantic value types.
|
|
* Type Decl:: Declaring the choice of type for a nonterminal symbol.
|
|
* Destructor Decl:: Declaring how symbols are freed.
|
|
* Expect Decl:: Suppressing warnings about parsing conflicts.
|
|
* Start Decl:: Specifying the start symbol.
|
|
* Pure Decl:: Requesting a reentrant parser.
|
|
* Decl Summary:: Table of all Bison declarations.
|
|
|
|
Parser C-Language Interface
|
|
|
|
* Parser Function:: How to call @code{yyparse} and what it returns.
|
|
* Lexical:: You must supply a function @code{yylex}
|
|
which reads tokens.
|
|
* Error Reporting:: You must supply a function @code{yyerror}.
|
|
* Action Features:: Special features for use in actions.
|
|
|
|
The Lexical Analyzer Function @code{yylex}
|
|
|
|
* Calling Convention:: How @code{yyparse} calls @code{yylex}.
|
|
* Token Values:: How @code{yylex} must return the semantic value
|
|
of the token it has read.
|
|
* Token Locations:: How @code{yylex} must return the text location
|
|
(line number, etc.) of the token, if the
|
|
actions want that.
|
|
* Pure Calling:: How the calling convention differs
|
|
in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
|
|
|
|
The Bison Parser Algorithm
|
|
|
|
* Look-Ahead:: Parser looks one token ahead when deciding what to do.
|
|
* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
|
|
* Precedence:: Operator precedence works by resolving conflicts.
|
|
* Contextual Precedence:: When an operator's precedence depends on context.
|
|
* Parser States:: The parser is a finite-state-machine with stack.
|
|
* Reduce/Reduce:: When two rules are applicable in the same situation.
|
|
* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
|
|
* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
|
|
* Stack Overflow:: What happens when stack gets full. How to avoid it.
|
|
|
|
Operator Precedence
|
|
|
|
* Why Precedence:: An example showing why precedence is needed.
|
|
* Using Precedence:: How to specify precedence in Bison grammars.
|
|
* Precedence Examples:: How these features are used in the previous example.
|
|
* How Precedence:: How they work.
|
|
|
|
Handling Context Dependencies
|
|
|
|
* Semantic Tokens:: Token parsing can depend on the semantic context.
|
|
* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
|
|
* Tie-in Recovery:: Lexical tie-ins have implications for how
|
|
error recovery rules must be written.
|
|
|
|
Debugging Your Parser
|
|
|
|
* Understanding:: Understanding the structure of your parser.
|
|
* Tracing:: Tracing the execution of your parser.
|
|
|
|
Invoking Bison
|
|
|
|
* Bison Options:: All the options described in detail,
|
|
in alphabetical order by short options.
|
|
* Option Cross Key:: Alphabetical list of long options.
|
|
* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
|
|
|
|
Frequently Asked Questions
|
|
|
|
* Parser Stack Overflow:: Breaking the Stack Limits
|
|
* How Can I Reset the Parser:: @code{yyparse} Keeps some State
|
|
* Strings are Destroyed:: @code{yylval} Loses Track of Strings
|
|
* C++ Parsers:: Compiling Parsers with C++ Compilers
|
|
* Implementing Loops:: Control Flow in the Calculator
|
|
|
|
Copying This Manual
|
|
|
|
* GNU Free Documentation License:: License for copying this manual.
|
|
|
|
@end detailmenu
|
|
@end menu
|
|
|
|
@node Introduction
|
|
@unnumbered Introduction
|
|
@cindex introduction
|
|
|
|
@dfn{Bison} is a general-purpose parser generator that converts a
|
|
grammar description for an @acronym{LALR}(1) context-free grammar into a C
|
|
program to parse that grammar. Once you are proficient with Bison,
|
|
you may use it to develop a wide range of language parsers, from those
|
|
used in simple desk calculators to complex programming languages.
|
|
|
|
Bison is upward compatible with Yacc: all properly-written Yacc grammars
|
|
ought to work with Bison with no change. Anyone familiar with Yacc
|
|
should be able to use Bison with little trouble. You need to be fluent in
|
|
C programming in order to use Bison or to understand this manual.
|
|
|
|
We begin with tutorial chapters that explain the basic concepts of using
|
|
Bison and show three explained examples, each building on the last. If you
|
|
don't know Bison or Yacc, start by reading these chapters. Reference
|
|
chapters follow which describe specific aspects of Bison in detail.
|
|
|
|
Bison was written primarily by Robert Corbett; Richard Stallman made it
|
|
Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
|
|
multi-character string literals and other features.
|
|
|
|
This edition corresponds to version @value{VERSION} of Bison.
|
|
|
|
@node Conditions
|
|
@unnumbered Conditions for Using Bison
|
|
|
|
As of Bison version 1.24, we have changed the distribution terms for
|
|
@code{yyparse} to permit using Bison's output in nonfree programs when
|
|
Bison is generating C code for @acronym{LALR}(1) parsers. Formerly, these
|
|
parsers could be used only in programs that were free software.
|
|
|
|
The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
|
|
compiler, have never
|
|
had such a requirement. They could always be used for nonfree
|
|
software. The reason Bison was different was not due to a special
|
|
policy decision; it resulted from applying the usual General Public
|
|
License to all of the Bison source code.
|
|
|
|
The output of the Bison utility---the Bison parser file---contains a
|
|
verbatim copy of a sizable piece of Bison, which is the code for the
|
|
@code{yyparse} function. (The actions from your grammar are inserted
|
|
into this function at one point, but the rest of the function is not
|
|
changed.) When we applied the @acronym{GPL} terms to the code for
|
|
@code{yyparse},
|
|
the effect was to restrict the use of Bison output to free software.
|
|
|
|
We didn't change the terms because of sympathy for people who want to
|
|
make software proprietary. @strong{Software should be free.} But we
|
|
concluded that limiting Bison's use to free software was doing little to
|
|
encourage people to make other software free. So we decided to make the
|
|
practical conditions for using Bison match the practical conditions for
|
|
using the other @acronym{GNU} tools.
|
|
|
|
This exception applies only when Bison is generating C code for an
|
|
@acronym{LALR}(1) parser; otherwise, the @acronym{GPL} terms operate
|
|
as usual. You can
|
|
tell whether the exception applies to your @samp{.c} output file by
|
|
inspecting it to see whether it says ``As a special exception, when
|
|
this file is copied by Bison into a Bison output file, you may use
|
|
that output file without restriction.''
|
|
|
|
@include gpl.texi
|
|
|
|
@node Concepts
|
|
@chapter The Concepts of Bison
|
|
|
|
This chapter introduces many of the basic concepts without which the
|
|
details of Bison will not make sense. If you do not already know how to
|
|
use Bison or Yacc, we suggest you start by reading this chapter carefully.
|
|
|
|
@menu
|
|
* Language and Grammar:: Languages and context-free grammars,
|
|
as mathematical ideas.
|
|
* Grammar in Bison:: How we represent grammars for Bison's sake.
|
|
* Semantic Values:: Each token or syntactic grouping can have
|
|
a semantic value (the value of an integer,
|
|
the name of an identifier, etc.).
|
|
* Semantic Actions:: Each rule can have an action containing C code.
|
|
* GLR Parsers:: Writing parsers for general context-free languages.
|
|
* Simple GLR Parsers:: Using GLR in its simplest form.
|
|
* Locations Overview:: Tracking Locations.
|
|
* Bison Parser:: What are Bison's input and output,
|
|
how is the output used?
|
|
* Stages:: Stages in writing and running Bison grammars.
|
|
* Grammar Layout:: Overall structure of a Bison grammar file.
|
|
@end menu
|
|
|
|
@node Language and Grammar
|
|
@section Languages and Context-Free Grammars
|
|
|
|
@cindex context-free grammar
|
|
@cindex grammar, context-free
|
|
In order for Bison to parse a language, it must be described by a
|
|
@dfn{context-free grammar}. This means that you specify one or more
|
|
@dfn{syntactic groupings} and give rules for constructing them from their
|
|
parts. For example, in the C language, one kind of grouping is called an
|
|
`expression'. One rule for making an expression might be, ``An expression
|
|
can be made of a minus sign and another expression''. Another would be,
|
|
``An expression can be an integer''. As you can see, rules are often
|
|
recursive, but there must be at least one rule which leads out of the
|
|
recursion.
|
|
|
|
@cindex @acronym{BNF}
|
|
@cindex Backus-Naur form
|
|
The most common formal system for presenting such rules for humans to read
|
|
is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
|
|
order to specify the language Algol 60. Any grammar expressed in
|
|
@acronym{BNF} is a context-free grammar. The input to Bison is
|
|
essentially machine-readable @acronym{BNF}.
|
|
|
|
@cindex @acronym{LALR}(1) grammars
|
|
@cindex @acronym{LR}(1) grammars
|
|
There are various important subclasses of context-free grammar. Although it
|
|
can handle almost all context-free grammars, Bison is optimized for what
|
|
are called @acronym{LALR}(1) grammars.
|
|
In brief, in these grammars, it must be possible to
|
|
tell how to parse any portion of an input string with just a single
|
|
token of look-ahead. Strictly speaking, that is a description of an
|
|
@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
|
|
restrictions that are
|
|
hard to explain simply; but it is rare in actual practice to find an
|
|
@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
|
|
@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
|
|
more information on this.
|
|
|
|
@cindex @acronym{GLR} parsing
|
|
@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
|
|
@cindex ambiguous grammars
|
|
@cindex non-deterministic parsing
|
|
|
|
Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
|
|
roughly that the next grammar rule to apply at any point in the input is
|
|
uniquely determined by the preceding input and a fixed, finite portion
|
|
(called a @dfn{look-ahead}) of the remaining input. A context-free
|
|
grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
|
|
apply the grammar rules to get the some inputs. Even unambiguous
|
|
grammars can be @dfn{non-deterministic}, meaning that no fixed
|
|
look-ahead always suffices to determine the next grammar rule to apply.
|
|
With the proper declarations, Bison is also able to parse these more
|
|
general context-free grammars, using a technique known as @acronym{GLR}
|
|
parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
|
|
are able to handle any context-free grammar for which the number of
|
|
possible parses of any given string is finite.
|
|
|
|
@cindex symbols (abstract)
|
|
@cindex token
|
|
@cindex syntactic grouping
|
|
@cindex grouping, syntactic
|
|
In the formal grammatical rules for a language, each kind of syntactic
|
|
unit or grouping is named by a @dfn{symbol}. Those which are built by
|
|
grouping smaller constructs according to grammatical rules are called
|
|
@dfn{nonterminal symbols}; those which can't be subdivided are called
|
|
@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
|
|
corresponding to a single terminal symbol a @dfn{token}, and a piece
|
|
corresponding to a single nonterminal symbol a @dfn{grouping}.
|
|
|
|
We can use the C language as an example of what symbols, terminal and
|
|
nonterminal, mean. The tokens of C are identifiers, constants (numeric
|
|
and string), and the various keywords, arithmetic operators and
|
|
punctuation marks. So the terminal symbols of a grammar for C include
|
|
`identifier', `number', `string', plus one symbol for each keyword,
|
|
operator or punctuation mark: `if', `return', `const', `static', `int',
|
|
`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
|
|
(These tokens can be subdivided into characters, but that is a matter of
|
|
lexicography, not grammar.)
|
|
|
|
Here is a simple C function subdivided into tokens:
|
|
|
|
@ifinfo
|
|
@example
|
|
int /* @r{keyword `int'} */
|
|
square (int x) /* @r{identifier, open-paren, identifier,}
|
|
@r{identifier, close-paren} */
|
|
@{ /* @r{open-brace} */
|
|
return x * x; /* @r{keyword `return', identifier, asterisk,
|
|
identifier, semicolon} */
|
|
@} /* @r{close-brace} */
|
|
@end example
|
|
@end ifinfo
|
|
@ifnotinfo
|
|
@example
|
|
int /* @r{keyword `int'} */
|
|
square (int x) /* @r{identifier, open-paren, identifier, identifier, close-paren} */
|
|
@{ /* @r{open-brace} */
|
|
return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
|
|
@} /* @r{close-brace} */
|
|
@end example
|
|
@end ifnotinfo
|
|
|
|
The syntactic groupings of C include the expression, the statement, the
|
|
declaration, and the function definition. These are represented in the
|
|
grammar of C by nonterminal symbols `expression', `statement',
|
|
`declaration' and `function definition'. The full grammar uses dozens of
|
|
additional language constructs, each with its own nonterminal symbol, in
|
|
order to express the meanings of these four. The example above is a
|
|
function definition; it contains one declaration, and one statement. In
|
|
the statement, each @samp{x} is an expression and so is @samp{x * x}.
|
|
|
|
Each nonterminal symbol must have grammatical rules showing how it is made
|
|
out of simpler constructs. For example, one kind of C statement is the
|
|
@code{return} statement; this would be described with a grammar rule which
|
|
reads informally as follows:
|
|
|
|
@quotation
|
|
A `statement' can be made of a `return' keyword, an `expression' and a
|
|
`semicolon'.
|
|
@end quotation
|
|
|
|
@noindent
|
|
There would be many other rules for `statement', one for each kind of
|
|
statement in C.
|
|
|
|
@cindex start symbol
|
|
One nonterminal symbol must be distinguished as the special one which
|
|
defines a complete utterance in the language. It is called the @dfn{start
|
|
symbol}. In a compiler, this means a complete input program. In the C
|
|
language, the nonterminal symbol `sequence of definitions and declarations'
|
|
plays this role.
|
|
|
|
For example, @samp{1 + 2} is a valid C expression---a valid part of a C
|
|
program---but it is not valid as an @emph{entire} C program. In the
|
|
context-free grammar of C, this follows from the fact that `expression' is
|
|
not the start symbol.
|
|
|
|
The Bison parser reads a sequence of tokens as its input, and groups the
|
|
tokens using the grammar rules. If the input is valid, the end result is
|
|
that the entire token sequence reduces to a single grouping whose symbol is
|
|
the grammar's start symbol. If we use a grammar for C, the entire input
|
|
must be a `sequence of definitions and declarations'. If not, the parser
|
|
reports a syntax error.
|
|
|
|
@node Grammar in Bison
|
|
@section From Formal Rules to Bison Input
|
|
@cindex Bison grammar
|
|
@cindex grammar, Bison
|
|
@cindex formal grammar
|
|
|
|
A formal grammar is a mathematical construct. To define the language
|
|
for Bison, you must write a file expressing the grammar in Bison syntax:
|
|
a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
|
|
|
|
A nonterminal symbol in the formal grammar is represented in Bison input
|
|
as an identifier, like an identifier in C@. By convention, it should be
|
|
in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
|
|
|
|
The Bison representation for a terminal symbol is also called a @dfn{token
|
|
type}. Token types as well can be represented as C-like identifiers. By
|
|
convention, these identifiers should be upper case to distinguish them from
|
|
nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
|
|
@code{RETURN}. A terminal symbol that stands for a particular keyword in
|
|
the language should be named after that keyword converted to upper case.
|
|
The terminal symbol @code{error} is reserved for error recovery.
|
|
@xref{Symbols}.
|
|
|
|
A terminal symbol can also be represented as a character literal, just like
|
|
a C character constant. You should do this whenever a token is just a
|
|
single character (parenthesis, plus-sign, etc.): use that same character in
|
|
a literal as the terminal symbol for that token.
|
|
|
|
A third way to represent a terminal symbol is with a C string constant
|
|
containing several characters. @xref{Symbols}, for more information.
|
|
|
|
The grammar rules also have an expression in Bison syntax. For example,
|
|
here is the Bison rule for a C @code{return} statement. The semicolon in
|
|
quotes is a literal character token, representing part of the C syntax for
|
|
the statement; the naked semicolon, and the colon, are Bison punctuation
|
|
used in every rule.
|
|
|
|
@example
|
|
stmt: RETURN expr ';'
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
@xref{Rules, ,Syntax of Grammar Rules}.
|
|
|
|
@node Semantic Values
|
|
@section Semantic Values
|
|
@cindex semantic value
|
|
@cindex value, semantic
|
|
|
|
A formal grammar selects tokens only by their classifications: for example,
|
|
if a rule mentions the terminal symbol `integer constant', it means that
|
|
@emph{any} integer constant is grammatically valid in that position. The
|
|
precise value of the constant is irrelevant to how to parse the input: if
|
|
@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
|
|
grammatical.
|
|
|
|
But the precise value is very important for what the input means once it is
|
|
parsed. A compiler is useless if it fails to distinguish between 4, 1 and
|
|
3989 as constants in the program! Therefore, each token in a Bison grammar
|
|
has both a token type and a @dfn{semantic value}. @xref{Semantics,
|
|
,Defining Language Semantics},
|
|
for details.
|
|
|
|
The token type is a terminal symbol defined in the grammar, such as
|
|
@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
|
|
you need to know to decide where the token may validly appear and how to
|
|
group it with other tokens. The grammar rules know nothing about tokens
|
|
except their types.
|
|
|
|
The semantic value has all the rest of the information about the
|
|
meaning of the token, such as the value of an integer, or the name of an
|
|
identifier. (A token such as @code{','} which is just punctuation doesn't
|
|
need to have any semantic value.)
|
|
|
|
For example, an input token might be classified as token type
|
|
@code{INTEGER} and have the semantic value 4. Another input token might
|
|
have the same token type @code{INTEGER} but value 3989. When a grammar
|
|
rule says that @code{INTEGER} is allowed, either of these tokens is
|
|
acceptable because each is an @code{INTEGER}. When the parser accepts the
|
|
token, it keeps track of the token's semantic value.
|
|
|
|
Each grouping can also have a semantic value as well as its nonterminal
|
|
symbol. For example, in a calculator, an expression typically has a
|
|
semantic value that is a number. In a compiler for a programming
|
|
language, an expression typically has a semantic value that is a tree
|
|
structure describing the meaning of the expression.
|
|
|
|
@node Semantic Actions
|
|
@section Semantic Actions
|
|
@cindex semantic actions
|
|
@cindex actions, semantic
|
|
|
|
In order to be useful, a program must do more than parse input; it must
|
|
also produce some output based on the input. In a Bison grammar, a grammar
|
|
rule can have an @dfn{action} made up of C statements. Each time the
|
|
parser recognizes a match for that rule, the action is executed.
|
|
@xref{Actions}.
|
|
|
|
Most of the time, the purpose of an action is to compute the semantic value
|
|
of the whole construct from the semantic values of its parts. For example,
|
|
suppose we have a rule which says an expression can be the sum of two
|
|
expressions. When the parser recognizes such a sum, each of the
|
|
subexpressions has a semantic value which describes how it was built up.
|
|
The action for this rule should create a similar sort of value for the
|
|
newly recognized larger expression.
|
|
|
|
For example, here is a rule that says an expression can be the sum of
|
|
two subexpressions:
|
|
|
|
@example
|
|
expr: expr '+' expr @{ $$ = $1 + $3; @}
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
The action says how to produce the semantic value of the sum expression
|
|
from the values of the two subexpressions.
|
|
|
|
@node GLR Parsers
|
|
@section Writing @acronym{GLR} Parsers
|
|
@cindex @acronym{GLR} parsing
|
|
@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
|
|
@findex %glr-parser
|
|
@cindex conflicts
|
|
@cindex shift/reduce conflicts
|
|
|
|
In some grammars, there will be cases where Bison's standard
|
|
@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
|
|
certain grammar rule at a given point. That is, it may not be able to
|
|
decide (on the basis of the input read so far) which of two possible
|
|
reductions (applications of a grammar rule) applies, or whether to apply
|
|
a reduction or read more of the input and apply a reduction later in the
|
|
input. These are known respectively as @dfn{reduce/reduce} conflicts
|
|
(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
|
|
(@pxref{Shift/Reduce}).
|
|
|
|
To use a grammar that is not easily modified to be @acronym{LALR}(1), a
|
|
more general parsing algorithm is sometimes necessary. If you include
|
|
@code{%glr-parser} among the Bison declarations in your file
|
|
(@pxref{Grammar Outline}), the result will be a Generalized @acronym{LR}
|
|
(@acronym{GLR}) parser. These parsers handle Bison grammars that
|
|
contain no unresolved conflicts (i.e., after applying precedence
|
|
declarations) identically to @acronym{LALR}(1) parsers. However, when
|
|
faced with unresolved shift/reduce and reduce/reduce conflicts,
|
|
@acronym{GLR} parsers use the simple expedient of doing both,
|
|
effectively cloning the parser to follow both possibilities. Each of
|
|
the resulting parsers can again split, so that at any given time, there
|
|
can be any number of possible parses being explored. The parsers
|
|
proceed in lockstep; that is, all of them consume (shift) a given input
|
|
symbol before any of them proceed to the next. Each of the cloned
|
|
parsers eventually meets one of two possible fates: either it runs into
|
|
a parsing error, in which case it simply vanishes, or it merges with
|
|
another parser, because the two of them have reduced the input to an
|
|
identical set of symbols.
|
|
|
|
During the time that there are multiple parsers, semantic actions are
|
|
recorded, but not performed. When a parser disappears, its recorded
|
|
semantic actions disappear as well, and are never performed. When a
|
|
reduction makes two parsers identical, causing them to merge, Bison
|
|
records both sets of semantic actions. Whenever the last two parsers
|
|
merge, reverting to the single-parser case, Bison resolves all the
|
|
outstanding actions either by precedences given to the grammar rules
|
|
involved, or by performing both actions, and then calling a designated
|
|
user-defined function on the resulting values to produce an arbitrary
|
|
merged result.
|
|
|
|
Let's consider an example, vastly simplified from a C++ grammar.
|
|
|
|
@example
|
|
%@{
|
|
#include <stdio.h>
|
|
#define YYSTYPE char const *
|
|
int yylex (void);
|
|
void yyerror (char const *);
|
|
%@}
|
|
|
|
%token TYPENAME ID
|
|
|
|
%right '='
|
|
%left '+'
|
|
|
|
%glr-parser
|
|
|
|
%%
|
|
|
|
prog :
|
|
| prog stmt @{ printf ("\n"); @}
|
|
;
|
|
|
|
stmt : expr ';' %dprec 1
|
|
| decl %dprec 2
|
|
;
|
|
|
|
expr : ID @{ printf ("%s ", $$); @}
|
|
| TYPENAME '(' expr ')'
|
|
@{ printf ("%s <cast> ", $1); @}
|
|
| expr '+' expr @{ printf ("+ "); @}
|
|
| expr '=' expr @{ printf ("= "); @}
|
|
;
|
|
|
|
decl : TYPENAME declarator ';'
|
|
@{ printf ("%s <declare> ", $1); @}
|
|
| TYPENAME declarator '=' expr ';'
|
|
@{ printf ("%s <init-declare> ", $1); @}
|
|
;
|
|
|
|
declarator : ID @{ printf ("\"%s\" ", $1); @}
|
|
| '(' declarator ')'
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
This models a problematic part of the C++ grammar---the ambiguity between
|
|
certain declarations and statements. For example,
|
|
|
|
@example
|
|
T (x) = y+z;
|
|
@end example
|
|
|
|
@noindent
|
|
parses as either an @code{expr} or a @code{stmt}
|
|
(assuming that @samp{T} is recognized as a @code{TYPENAME} and
|
|
@samp{x} as an @code{ID}).
|
|
Bison detects this as a reduce/reduce conflict between the rules
|
|
@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
|
|
time it encounters @code{x} in the example above. The two @code{%dprec}
|
|
declarations, however, give precedence to interpreting the example as a
|
|
@code{decl}, which implies that @code{x} is a declarator.
|
|
The parser therefore prints
|
|
|
|
@example
|
|
"x" y z + T <init-declare>
|
|
@end example
|
|
|
|
Consider a different input string for this parser:
|
|
|
|
@example
|
|
T (x) + y;
|
|
@end example
|
|
|
|
@noindent
|
|
Here, there is no ambiguity (this cannot be parsed as a declaration).
|
|
However, at the time the Bison parser encounters @code{x}, it does not
|
|
have enough information to resolve the reduce/reduce conflict (again,
|
|
between @code{x} as an @code{expr} or a @code{declarator}). In this
|
|
case, no precedence declaration is used. Instead, the parser splits
|
|
into two, one assuming that @code{x} is an @code{expr}, and the other
|
|
assuming @code{x} is a @code{declarator}. The second of these parsers
|
|
then vanishes when it sees @code{+}, and the parser prints
|
|
|
|
@example
|
|
x T <cast> y +
|
|
@end example
|
|
|
|
Suppose that instead of resolving the ambiguity, you wanted to see all
|
|
the possibilities. For this purpose, we must @dfn{merge} the semantic
|
|
actions of the two possible parsers, rather than choosing one over the
|
|
other. To do so, you could change the declaration of @code{stmt} as
|
|
follows:
|
|
|
|
@example
|
|
stmt : expr ';' %merge <stmtMerge>
|
|
| decl %merge <stmtMerge>
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
|
|
and define the @code{stmtMerge} function as:
|
|
|
|
@example
|
|
static YYSTYPE
|
|
stmtMerge (YYSTYPE x0, YYSTYPE x1)
|
|
@{
|
|
printf ("<OR> ");
|
|
return "";
|
|
@}
|
|
@end example
|
|
|
|
@noindent
|
|
with an accompanying forward declaration
|
|
in the C declarations at the beginning of the file:
|
|
|
|
@example
|
|
%@{
|
|
#define YYSTYPE char const *
|
|
static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
|
|
%@}
|
|
@end example
|
|
|
|
@noindent
|
|
With these declarations, the resulting parser will parse the first example
|
|
as both an @code{expr} and a @code{decl}, and print
|
|
|
|
@example
|
|
"x" y z + T <init-declare> x T <cast> y z + = <OR>
|
|
@end example
|
|
|
|
@sp 1
|
|
|
|
@cindex @code{incline}
|
|
@cindex @acronym{GLR} parsers and @code{inline}
|
|
The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
|
|
later. In addition, they use the @code{inline} keyword, which is not
|
|
C89, but is C99 and is a common extension in pre-C99 compilers. It is
|
|
up to the user of these parsers to handle
|
|
portability issues. For instance, if using Autoconf and the Autoconf
|
|
macro @code{AC_C_INLINE}, a mere
|
|
|
|
@example
|
|
%@{
|
|
#include <config.h>
|
|
%@}
|
|
@end example
|
|
|
|
@noindent
|
|
will suffice. Otherwise, we suggest
|
|
|
|
@example
|
|
%@{
|
|
#if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
|
|
#define inline
|
|
#endif
|
|
%@}
|
|
@end example
|
|
|
|
@node Simple GLR Parsers
|
|
@section Using @acronym{GLR} in its Simplest Form
|
|
@cindex @acronym{GLR} parsing, unambiguous grammars
|
|
@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
|
|
@findex %glr-parser
|
|
@findex %expect-rr
|
|
@cindex conflicts
|
|
@cindex reduce/reduce conflicts
|
|
|
|
The C++ example for @acronym{GLR} (@pxref{GLR Parsers}) explains how to use
|
|
the @acronym{GLR} parsing algorithm with some advanced features such as
|
|
@samp{%dprec} and @samp{%merge} to handle syntactically ambiguous
|
|
grammars. However, the @acronym{GLR} algorithm can also be used in a simpler
|
|
way to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
|
|
Such grammars typically require more than one symbol of lookahead,
|
|
or (in rare cases) fall into the category of grammars in which the
|
|
@acronym{LALR}(1) algorithm throws away too much information (they are in
|
|
@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
|
|
|
|
Here is an example of this situation, using a problem that
|
|
arises in the declaration of enumerated and subrange types in the
|
|
programming language Pascal. These declarations look like this:
|
|
|
|
@example
|
|
type subrange = lo .. hi;
|
|
type enum = (a, b, c);
|
|
@end example
|
|
|
|
@noindent
|
|
The original language standard allows only numeric
|
|
literals and constant identifiers for the subrange bounds (@samp{lo}
|
|
and @samp{hi}), but Extended Pascal (ISO/IEC 10206:1990) and many other
|
|
Pascal implementations allow arbitrary expressions there. This gives
|
|
rise to the following situation, containing a superfluous pair of
|
|
parentheses:
|
|
|
|
@example
|
|
type subrange = (a) .. b;
|
|
@end example
|
|
|
|
@noindent
|
|
Compare this to the following declaration of an enumerated
|
|
type with only one value:
|
|
|
|
@example
|
|
type enum = (a);
|
|
@end example
|
|
|
|
@noindent
|
|
(These declarations are contrived, but they are syntactically
|
|
valid, and more-complicated cases can come up in practical programs.)
|
|
|
|
These two declarations look identical until the @samp{..} token.
|
|
With normal @acronym{LALR}(1) one-token look-ahead it is not
|
|
possible to decide between the two forms when the identifier
|
|
@samp{a} is parsed. It is, however, desirable
|
|
for a parser to decide this, since in the latter case
|
|
@samp{a} must become a new identifier to represent the enumeration
|
|
value, while in the former case @samp{a} must be evaluated with its
|
|
current meaning, which may be a constant or even a function call.
|
|
|
|
You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
|
|
to be resolved later, but this typically requires substantial
|
|
contortions in both semantic actions and large parts of the
|
|
grammar, where the parentheses are nested in the recursive rules for
|
|
expressions.
|
|
|
|
You might think of using the lexer to distinguish between the two
|
|
forms by returning different tokens for currently defined and
|
|
undefined identifiers. But if these declarations occur in a local
|
|
scope, and @samp{a} is defined in an outer scope, then both forms
|
|
are possible---either locally redefining @samp{a}, or using the
|
|
value of @samp{a} from the outer scope. So this approach cannot
|
|
work.
|
|
|
|
A solution to this problem is to use a @acronym{GLR} parser in its simplest
|
|
form, i.e., without using special features such as @samp{%dprec} and
|
|
@samp{%merge}. When the @acronym{GLR} parser reaches the critical state, it
|
|
simply splits into two branches and pursues both syntax rules
|
|
simultaneously. Sooner or later, one of them runs into a parsing
|
|
error. If there is a @samp{..} token before the next
|
|
@samp{;}, the rule for enumerated types fails since it cannot
|
|
accept @samp{..} anywhere; otherwise, the subrange type rule
|
|
fails since it requires a @samp{..} token. So one of the branches
|
|
fails silently, and the other one continues normally, performing
|
|
all the intermediate actions that were postponed during the split.
|
|
|
|
If the input is syntactically incorrect, both branches fail and the parser
|
|
reports a syntax error as usual.
|
|
|
|
The effect of all this is that the parser seems to ``guess'' the
|
|
correct branch to take, or in other words, it seems to use more
|
|
look-ahead than the underlying @acronym{LALR}(1) algorithm actually allows
|
|
for. In this example, @acronym{LALR}(2) would suffice, but also some cases
|
|
that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
|
|
|
|
Since there can be only two branches and at least one of them
|
|
must fail, you need not worry about merging the branches by
|
|
using dynamic precedence or @samp{%merge}.
|
|
|
|
Another potential problem of @acronym{GLR} does not arise here, either. In
|
|
general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
|
|
and the current Bison parser even takes exponential time and space
|
|
for some grammars. In practice, this rarely happens, and for many
|
|
grammars it is possible to prove that it cannot happen. In
|
|
in the present example, there is only one conflict between two
|
|
rules, and the type-declaration context where the conflict
|
|
arises cannot be nested. So the number of
|
|
branches that can exist at any time is limited by the constant 2,
|
|
and the parsing time is still linear.
|
|
|
|
So here we have a case where we can use the benefits of @acronym{GLR}, almost
|
|
without disadvantages. There are two things to note, though.
|
|
First, one should carefully analyze the conflicts reported by
|
|
Bison to make sure that @acronym{GLR} splitting is done only where it is
|
|
intended to be. A @acronym{GLR} parser splitting inadvertently may cause
|
|
problems less obvious than an @acronym{LALR} parser statically choosing the
|
|
wrong alternative in a conflict.
|
|
|
|
Second, interactions with the lexer (@pxref{Semantic Tokens}) must
|
|
be considered with great care. Since a split parser consumes tokens
|
|
without performing any actions during the split, the lexer cannot
|
|
obtain information via parser actions. Some cases of
|
|
lexer interactions can simply be eliminated by using @acronym{GLR}, i.e.,
|
|
shifting the complications from the lexer to the parser. Remaining
|
|
cases have to be checked for safety.
|
|
|
|
In our example, it would be safe for the lexer to return tokens
|
|
based on their current meanings in some symbol table, because no new
|
|
symbols are defined in the middle of a type declaration. Though it
|
|
is possible for a parser to define the enumeration
|
|
constants as they are parsed, before the type declaration is
|
|
completed, it actually makes no difference since they cannot be used
|
|
within the same enumerated type declaration.
|
|
|
|
Here is a Bison grammar corresponding to the example above. It
|
|
parses a vastly simplified form of Pascal type declarations.
|
|
|
|
@example
|
|
%token TYPE DOTDOT ID
|
|
|
|
@group
|
|
%left '+' '-'
|
|
%left '*' '/'
|
|
@end group
|
|
|
|
%%
|
|
|
|
@group
|
|
type_decl:
|
|
TYPE ID '=' type ';'
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
type: '(' id_list ')'
|
|
| expr DOTDOT expr
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
id_list: ID
|
|
| id_list ',' ID
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
expr: '(' expr ')'
|
|
| expr '+' expr
|
|
| expr '-' expr
|
|
| expr '*' expr
|
|
| expr '/' expr
|
|
| ID
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
|
|
about one reduce/reduce conflict. In the conflicting situation the
|
|
parser chooses one of the alternatives, arbitrarily the one
|
|
declared first. Therefore the following correct input is not
|
|
recognized:
|
|
|
|
@example
|
|
type t = (a) .. b;
|
|
@end example
|
|
|
|
The parser can be turned into a @acronym{GLR} parser, while also telling Bison
|
|
to be silent about the one known reduce/reduce conflict, simply by
|
|
adding these two declarations to the Bison input file:
|
|
|
|
@example
|
|
%glr-parser
|
|
%expect-rr 1
|
|
@end example
|
|
|
|
@noindent
|
|
No change in the grammar itself is required. Now the
|
|
parser recognizes all valid declarations, according to the
|
|
limited syntax above, transparently. In fact, the user does not even
|
|
notice when the parser splits.
|
|
|
|
@node Locations Overview
|
|
@section Locations
|
|
@cindex location
|
|
@cindex textual location
|
|
@cindex location, textual
|
|
|
|
Many applications, like interpreters or compilers, have to produce verbose
|
|
and useful error messages. To achieve this, one must be able to keep track of
|
|
the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
|
|
Bison provides a mechanism for handling these locations.
|
|
|
|
Each token has a semantic value. In a similar fashion, each token has an
|
|
associated location, but the type of locations is the same for all tokens and
|
|
groupings. Moreover, the output parser is equipped with a default data
|
|
structure for storing locations (@pxref{Locations}, for more details).
|
|
|
|
Like semantic values, locations can be reached in actions using a dedicated
|
|
set of constructs. In the example above, the location of the whole grouping
|
|
is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
|
|
@code{@@3}.
|
|
|
|
When a rule is matched, a default action is used to compute the semantic value
|
|
of its left hand side (@pxref{Actions}). In the same way, another default
|
|
action is used for locations. However, the action for locations is general
|
|
enough for most cases, meaning there is usually no need to describe for each
|
|
rule how @code{@@$} should be formed. When building a new location for a given
|
|
grouping, the default behavior of the output parser is to take the beginning
|
|
of the first symbol, and the end of the last symbol.
|
|
|
|
@node Bison Parser
|
|
@section Bison Output: the Parser File
|
|
@cindex Bison parser
|
|
@cindex Bison utility
|
|
@cindex lexical analyzer, purpose
|
|
@cindex parser
|
|
|
|
When you run Bison, you give it a Bison grammar file as input. The output
|
|
is a C source file that parses the language described by the grammar.
|
|
This file is called a @dfn{Bison parser}. Keep in mind that the Bison
|
|
utility and the Bison parser are two distinct programs: the Bison utility
|
|
is a program whose output is the Bison parser that becomes part of your
|
|
program.
|
|
|
|
The job of the Bison parser is to group tokens into groupings according to
|
|
the grammar rules---for example, to build identifiers and operators into
|
|
expressions. As it does this, it runs the actions for the grammar rules it
|
|
uses.
|
|
|
|
The tokens come from a function called the @dfn{lexical analyzer} that
|
|
you must supply in some fashion (such as by writing it in C). The Bison
|
|
parser calls the lexical analyzer each time it wants a new token. It
|
|
doesn't know what is ``inside'' the tokens (though their semantic values
|
|
may reflect this). Typically the lexical analyzer makes the tokens by
|
|
parsing characters of text, but Bison does not depend on this.
|
|
@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
|
|
|
|
The Bison parser file is C code which defines a function named
|
|
@code{yyparse} which implements that grammar. This function does not make
|
|
a complete C program: you must supply some additional functions. One is
|
|
the lexical analyzer. Another is an error-reporting function which the
|
|
parser calls to report an error. In addition, a complete C program must
|
|
start with a function called @code{main}; you have to provide this, and
|
|
arrange for it to call @code{yyparse} or the parser will never run.
|
|
@xref{Interface, ,Parser C-Language Interface}.
|
|
|
|
Aside from the token type names and the symbols in the actions you
|
|
write, all symbols defined in the Bison parser file itself
|
|
begin with @samp{yy} or @samp{YY}. This includes interface functions
|
|
such as the lexical analyzer function @code{yylex}, the error reporting
|
|
function @code{yyerror} and the parser function @code{yyparse} itself.
|
|
This also includes numerous identifiers used for internal purposes.
|
|
Therefore, you should avoid using C identifiers starting with @samp{yy}
|
|
or @samp{YY} in the Bison grammar file except for the ones defined in
|
|
this manual.
|
|
|
|
In some cases the Bison parser file includes system headers, and in
|
|
those cases your code should respect the identifiers reserved by those
|
|
headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>},
|
|
@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
|
|
declare memory allocators and related types. Other system headers may
|
|
be included if you define @code{YYDEBUG} to a nonzero value
|
|
(@pxref{Tracing, ,Tracing Your Parser}).
|
|
|
|
@node Stages
|
|
@section Stages in Using Bison
|
|
@cindex stages in using Bison
|
|
@cindex using Bison
|
|
|
|
The actual language-design process using Bison, from grammar specification
|
|
to a working compiler or interpreter, has these parts:
|
|
|
|
@enumerate
|
|
@item
|
|
Formally specify the grammar in a form recognized by Bison
|
|
(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
|
|
in the language, describe the action that is to be taken when an
|
|
instance of that rule is recognized. The action is described by a
|
|
sequence of C statements.
|
|
|
|
@item
|
|
Write a lexical analyzer to process input and pass tokens to the parser.
|
|
The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
|
|
Lexical Analyzer Function @code{yylex}}). It could also be produced
|
|
using Lex, but the use of Lex is not discussed in this manual.
|
|
|
|
@item
|
|
Write a controlling function that calls the Bison-produced parser.
|
|
|
|
@item
|
|
Write error-reporting routines.
|
|
@end enumerate
|
|
|
|
To turn this source code as written into a runnable program, you
|
|
must follow these steps:
|
|
|
|
@enumerate
|
|
@item
|
|
Run Bison on the grammar to produce the parser.
|
|
|
|
@item
|
|
Compile the code output by Bison, as well as any other source files.
|
|
|
|
@item
|
|
Link the object files to produce the finished product.
|
|
@end enumerate
|
|
|
|
@node Grammar Layout
|
|
@section The Overall Layout of a Bison Grammar
|
|
@cindex grammar file
|
|
@cindex file format
|
|
@cindex format of grammar file
|
|
@cindex layout of Bison grammar
|
|
|
|
The input file for the Bison utility is a @dfn{Bison grammar file}. The
|
|
general form of a Bison grammar file is as follows:
|
|
|
|
@example
|
|
%@{
|
|
@var{Prologue}
|
|
%@}
|
|
|
|
@var{Bison declarations}
|
|
|
|
%%
|
|
@var{Grammar rules}
|
|
%%
|
|
@var{Epilogue}
|
|
@end example
|
|
|
|
@noindent
|
|
The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
|
|
in every Bison grammar file to separate the sections.
|
|
|
|
The prologue may define types and variables used in the actions. You can
|
|
also use preprocessor commands to define macros used there, and use
|
|
@code{#include} to include header files that do any of these things.
|
|
You need to declare the lexical analyzer @code{yylex} and the error
|
|
printer @code{yyerror} here, along with any other global identifiers
|
|
used by the actions in the grammar rules.
|
|
|
|
The Bison declarations declare the names of the terminal and nonterminal
|
|
symbols, and may also describe operator precedence and the data types of
|
|
semantic values of various symbols.
|
|
|
|
The grammar rules define how to construct each nonterminal symbol from its
|
|
parts.
|
|
|
|
The epilogue can contain any code you want to use. Often the
|
|
definitions of functions declared in the prologue go here. In a
|
|
simple program, all the rest of the program can go here.
|
|
|
|
@node Examples
|
|
@chapter Examples
|
|
@cindex simple examples
|
|
@cindex examples, simple
|
|
|
|
Now we show and explain three sample programs written using Bison: a
|
|
reverse polish notation calculator, an algebraic (infix) notation
|
|
calculator, and a multi-function calculator. All three have been tested
|
|
under BSD Unix 4.3; each produces a usable, though limited, interactive
|
|
desk-top calculator.
|
|
|
|
These examples are simple, but Bison grammars for real programming
|
|
languages are written the same way.
|
|
@ifinfo
|
|
You can copy these examples out of the Info file and into a source file
|
|
to try them.
|
|
@end ifinfo
|
|
|
|
@menu
|
|
* RPN Calc:: Reverse polish notation calculator;
|
|
a first example with no operator precedence.
|
|
* Infix Calc:: Infix (algebraic) notation calculator.
|
|
Operator precedence is introduced.
|
|
* Simple Error Recovery:: Continuing after syntax errors.
|
|
* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
|
|
* Multi-function Calc:: Calculator with memory and trig functions.
|
|
It uses multiple data-types for semantic values.
|
|
* Exercises:: Ideas for improving the multi-function calculator.
|
|
@end menu
|
|
|
|
@node RPN Calc
|
|
@section Reverse Polish Notation Calculator
|
|
@cindex reverse polish notation
|
|
@cindex polish notation calculator
|
|
@cindex @code{rpcalc}
|
|
@cindex calculator, simple
|
|
|
|
The first example is that of a simple double-precision @dfn{reverse polish
|
|
notation} calculator (a calculator using postfix operators). This example
|
|
provides a good starting point, since operator precedence is not an issue.
|
|
The second example will illustrate how operator precedence is handled.
|
|
|
|
The source code for this calculator is named @file{rpcalc.y}. The
|
|
@samp{.y} extension is a convention used for Bison input files.
|
|
|
|
@menu
|
|
* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
|
|
* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
|
|
* Lexer: Rpcalc Lexer. The lexical analyzer.
|
|
* Main: Rpcalc Main. The controlling function.
|
|
* Error: Rpcalc Error. The error reporting function.
|
|
* Gen: Rpcalc Gen. Running Bison on the grammar file.
|
|
* Comp: Rpcalc Compile. Run the C compiler on the output code.
|
|
@end menu
|
|
|
|
@node Rpcalc Decls
|
|
@subsection Declarations for @code{rpcalc}
|
|
|
|
Here are the C and Bison declarations for the reverse polish notation
|
|
calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
|
|
|
|
@example
|
|
/* Reverse polish notation calculator. */
|
|
|
|
%@{
|
|
#define YYSTYPE double
|
|
#include <math.h>
|
|
int yylex (void);
|
|
void yyerror (char const *);
|
|
%@}
|
|
|
|
%token NUM
|
|
|
|
%% /* Grammar rules and actions follow. */
|
|
@end example
|
|
|
|
The declarations section (@pxref{Prologue, , The prologue}) contains two
|
|
preprocessor directives and two forward declarations.
|
|
|
|
The @code{#define} directive defines the macro @code{YYSTYPE}, thus
|
|
specifying the C data type for semantic values of both tokens and
|
|
groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
|
|
Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
|
|
don't define it, @code{int} is the default. Because we specify
|
|
@code{double}, each token and each expression has an associated value,
|
|
which is a floating point number.
|
|
|
|
The @code{#include} directive is used to declare the exponentiation
|
|
function @code{pow}.
|
|
|
|
The forward declarations for @code{yylex} and @code{yyerror} are
|
|
needed because the C language requires that functions be declared
|
|
before they are used. These functions will be defined in the
|
|
epilogue, but the parser calls them so they must be declared in the
|
|
prologue.
|
|
|
|
The second section, Bison declarations, provides information to Bison
|
|
about the token types (@pxref{Bison Declarations, ,The Bison
|
|
Declarations Section}). Each terminal symbol that is not a
|
|
single-character literal must be declared here. (Single-character
|
|
literals normally don't need to be declared.) In this example, all the
|
|
arithmetic operators are designated by single-character literals, so the
|
|
only terminal symbol that needs to be declared is @code{NUM}, the token
|
|
type for numeric constants.
|
|
|
|
@node Rpcalc Rules
|
|
@subsection Grammar Rules for @code{rpcalc}
|
|
|
|
Here are the grammar rules for the reverse polish notation calculator.
|
|
|
|
@example
|
|
input: /* empty */
|
|
| input line
|
|
;
|
|
|
|
line: '\n'
|
|
| exp '\n' @{ printf ("\t%.10g\n", $1); @}
|
|
;
|
|
|
|
exp: NUM @{ $$ = $1; @}
|
|
| exp exp '+' @{ $$ = $1 + $2; @}
|
|
| exp exp '-' @{ $$ = $1 - $2; @}
|
|
| exp exp '*' @{ $$ = $1 * $2; @}
|
|
| exp exp '/' @{ $$ = $1 / $2; @}
|
|
/* Exponentiation */
|
|
| exp exp '^' @{ $$ = pow ($1, $2); @}
|
|
/* Unary minus */
|
|
| exp 'n' @{ $$ = -$1; @}
|
|
;
|
|
%%
|
|
@end example
|
|
|
|
The groupings of the rpcalc ``language'' defined here are the expression
|
|
(given the name @code{exp}), the line of input (@code{line}), and the
|
|
complete input transcript (@code{input}). Each of these nonterminal
|
|
symbols has several alternate rules, joined by the @samp{|} punctuator
|
|
which is read as ``or''. The following sections explain what these rules
|
|
mean.
|
|
|
|
The semantics of the language is determined by the actions taken when a
|
|
grouping is recognized. The actions are the C code that appears inside
|
|
braces. @xref{Actions}.
|
|
|
|
You must specify these actions in C, but Bison provides the means for
|
|
passing semantic values between the rules. In each action, the
|
|
pseudo-variable @code{$$} stands for the semantic value for the grouping
|
|
that the rule is going to construct. Assigning a value to @code{$$} is the
|
|
main job of most actions. The semantic values of the components of the
|
|
rule are referred to as @code{$1}, @code{$2}, and so on.
|
|
|
|
@menu
|
|
* Rpcalc Input::
|
|
* Rpcalc Line::
|
|
* Rpcalc Expr::
|
|
@end menu
|
|
|
|
@node Rpcalc Input
|
|
@subsubsection Explanation of @code{input}
|
|
|
|
Consider the definition of @code{input}:
|
|
|
|
@example
|
|
input: /* empty */
|
|
| input line
|
|
;
|
|
@end example
|
|
|
|
This definition reads as follows: ``A complete input is either an empty
|
|
string, or a complete input followed by an input line''. Notice that
|
|
``complete input'' is defined in terms of itself. This definition is said
|
|
to be @dfn{left recursive} since @code{input} appears always as the
|
|
leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
|
|
|
|
The first alternative is empty because there are no symbols between the
|
|
colon and the first @samp{|}; this means that @code{input} can match an
|
|
empty string of input (no tokens). We write the rules this way because it
|
|
is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
|
|
It's conventional to put an empty alternative first and write the comment
|
|
@samp{/* empty */} in it.
|
|
|
|
The second alternate rule (@code{input line}) handles all nontrivial input.
|
|
It means, ``After reading any number of lines, read one more line if
|
|
possible.'' The left recursion makes this rule into a loop. Since the
|
|
first alternative matches empty input, the loop can be executed zero or
|
|
more times.
|
|
|
|
The parser function @code{yyparse} continues to process input until a
|
|
grammatical error is seen or the lexical analyzer says there are no more
|
|
input tokens; we will arrange for the latter to happen at end-of-input.
|
|
|
|
@node Rpcalc Line
|
|
@subsubsection Explanation of @code{line}
|
|
|
|
Now consider the definition of @code{line}:
|
|
|
|
@example
|
|
line: '\n'
|
|
| exp '\n' @{ printf ("\t%.10g\n", $1); @}
|
|
;
|
|
@end example
|
|
|
|
The first alternative is a token which is a newline character; this means
|
|
that rpcalc accepts a blank line (and ignores it, since there is no
|
|
action). The second alternative is an expression followed by a newline.
|
|
This is the alternative that makes rpcalc useful. The semantic value of
|
|
the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
|
|
question is the first symbol in the alternative. The action prints this
|
|
value, which is the result of the computation the user asked for.
|
|
|
|
This action is unusual because it does not assign a value to @code{$$}. As
|
|
a consequence, the semantic value associated with the @code{line} is
|
|
uninitialized (its value will be unpredictable). This would be a bug if
|
|
that value were ever used, but we don't use it: once rpcalc has printed the
|
|
value of the user's input line, that value is no longer needed.
|
|
|
|
@node Rpcalc Expr
|
|
@subsubsection Explanation of @code{expr}
|
|
|
|
The @code{exp} grouping has several rules, one for each kind of expression.
|
|
The first rule handles the simplest expressions: those that are just numbers.
|
|
The second handles an addition-expression, which looks like two expressions
|
|
followed by a plus-sign. The third handles subtraction, and so on.
|
|
|
|
@example
|
|
exp: NUM
|
|
| exp exp '+' @{ $$ = $1 + $2; @}
|
|
| exp exp '-' @{ $$ = $1 - $2; @}
|
|
@dots{}
|
|
;
|
|
@end example
|
|
|
|
We have used @samp{|} to join all the rules for @code{exp}, but we could
|
|
equally well have written them separately:
|
|
|
|
@example
|
|
exp: NUM ;
|
|
exp: exp exp '+' @{ $$ = $1 + $2; @} ;
|
|
exp: exp exp '-' @{ $$ = $1 - $2; @} ;
|
|
@dots{}
|
|
@end example
|
|
|
|
Most of the rules have actions that compute the value of the expression in
|
|
terms of the value of its parts. For example, in the rule for addition,
|
|
@code{$1} refers to the first component @code{exp} and @code{$2} refers to
|
|
the second one. The third component, @code{'+'}, has no meaningful
|
|
associated semantic value, but if it had one you could refer to it as
|
|
@code{$3}. When @code{yyparse} recognizes a sum expression using this
|
|
rule, the sum of the two subexpressions' values is produced as the value of
|
|
the entire expression. @xref{Actions}.
|
|
|
|
You don't have to give an action for every rule. When a rule has no
|
|
action, Bison by default copies the value of @code{$1} into @code{$$}.
|
|
This is what happens in the first rule (the one that uses @code{NUM}).
|
|
|
|
The formatting shown here is the recommended convention, but Bison does
|
|
not require it. You can add or change white space as much as you wish.
|
|
For example, this:
|
|
|
|
@example
|
|
exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
|
|
@end example
|
|
|
|
@noindent
|
|
means the same thing as this:
|
|
|
|
@example
|
|
exp: NUM
|
|
| exp exp '+' @{ $$ = $1 + $2; @}
|
|
| @dots{}
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
The latter, however, is much more readable.
|
|
|
|
@node Rpcalc Lexer
|
|
@subsection The @code{rpcalc} Lexical Analyzer
|
|
@cindex writing a lexical analyzer
|
|
@cindex lexical analyzer, writing
|
|
|
|
The lexical analyzer's job is low-level parsing: converting characters
|
|
or sequences of characters into tokens. The Bison parser gets its
|
|
tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
|
|
Analyzer Function @code{yylex}}.
|
|
|
|
Only a simple lexical analyzer is needed for the @acronym{RPN}
|
|
calculator. This
|
|
lexical analyzer skips blanks and tabs, then reads in numbers as
|
|
@code{double} and returns them as @code{NUM} tokens. Any other character
|
|
that isn't part of a number is a separate token. Note that the token-code
|
|
for such a single-character token is the character itself.
|
|
|
|
The return value of the lexical analyzer function is a numeric code which
|
|
represents a token type. The same text used in Bison rules to stand for
|
|
this token type is also a C expression for the numeric code for the type.
|
|
This works in two ways. If the token type is a character literal, then its
|
|
numeric code is that of the character; you can use the same
|
|
character literal in the lexical analyzer to express the number. If the
|
|
token type is an identifier, that identifier is defined by Bison as a C
|
|
macro whose definition is the appropriate number. In this example,
|
|
therefore, @code{NUM} becomes a macro for @code{yylex} to use.
|
|
|
|
The semantic value of the token (if it has one) is stored into the
|
|
global variable @code{yylval}, which is where the Bison parser will look
|
|
for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
|
|
defined at the beginning of the grammar; @pxref{Rpcalc Decls,
|
|
,Declarations for @code{rpcalc}}.)
|
|
|
|
A token type code of zero is returned if the end-of-input is encountered.
|
|
(Bison recognizes any nonpositive value as indicating end-of-input.)
|
|
|
|
Here is the code for the lexical analyzer:
|
|
|
|
@example
|
|
@group
|
|
/* The lexical analyzer returns a double floating point
|
|
number on the stack and the token NUM, or the numeric code
|
|
of the character read if not a number. It skips all blanks
|
|
and tabs, and returns 0 for end-of-input. */
|
|
|
|
#include <ctype.h>
|
|
@end group
|
|
|
|
@group
|
|
int
|
|
yylex (void)
|
|
@{
|
|
int c;
|
|
|
|
/* Skip white space. */
|
|
while ((c = getchar ()) == ' ' || c == '\t')
|
|
;
|
|
@end group
|
|
@group
|
|
/* Process numbers. */
|
|
if (c == '.' || isdigit (c))
|
|
@{
|
|
ungetc (c, stdin);
|
|
scanf ("%lf", &yylval);
|
|
return NUM;
|
|
@}
|
|
@end group
|
|
@group
|
|
/* Return end-of-input. */
|
|
if (c == EOF)
|
|
return 0;
|
|
/* Return a single char. */
|
|
return c;
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
@node Rpcalc Main
|
|
@subsection The Controlling Function
|
|
@cindex controlling function
|
|
@cindex main function in simple example
|
|
|
|
In keeping with the spirit of this example, the controlling function is
|
|
kept to the bare minimum. The only requirement is that it call
|
|
@code{yyparse} to start the process of parsing.
|
|
|
|
@example
|
|
@group
|
|
int
|
|
main (void)
|
|
@{
|
|
return yyparse ();
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
@node Rpcalc Error
|
|
@subsection The Error Reporting Routine
|
|
@cindex error reporting routine
|
|
|
|
When @code{yyparse} detects a syntax error, it calls the error reporting
|
|
function @code{yyerror} to print an error message (usually but not
|
|
always @code{"syntax error"}). It is up to the programmer to supply
|
|
@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
|
|
here is the definition we will use:
|
|
|
|
@example
|
|
@group
|
|
#include <stdio.h>
|
|
|
|
/* Called by yyparse on error. */
|
|
void
|
|
yyerror (char const *s)
|
|
@{
|
|
fprintf (stderr, "%s\n", s);
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
After @code{yyerror} returns, the Bison parser may recover from the error
|
|
and continue parsing if the grammar contains a suitable error rule
|
|
(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
|
|
have not written any error rules in this example, so any invalid input will
|
|
cause the calculator program to exit. This is not clean behavior for a
|
|
real calculator, but it is adequate for the first example.
|
|
|
|
@node Rpcalc Gen
|
|
@subsection Running Bison to Make the Parser
|
|
@cindex running Bison (introduction)
|
|
|
|
Before running Bison to produce a parser, we need to decide how to
|
|
arrange all the source code in one or more source files. For such a
|
|
simple example, the easiest thing is to put everything in one file. The
|
|
definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
|
|
end, in the epilogue of the file
|
|
(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
|
|
|
|
For a large project, you would probably have several source files, and use
|
|
@code{make} to arrange to recompile them.
|
|
|
|
With all the source in a single file, you use the following command to
|
|
convert it into a parser file:
|
|
|
|
@example
|
|
bison @var{file_name}.y
|
|
@end example
|
|
|
|
@noindent
|
|
In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
|
|
@sc{calc}ulator''). Bison produces a file named @file{@var{file_name}.tab.c},
|
|
removing the @samp{.y} from the original file name. The file output by
|
|
Bison contains the source code for @code{yyparse}. The additional
|
|
functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
|
|
are copied verbatim to the output.
|
|
|
|
@node Rpcalc Compile
|
|
@subsection Compiling the Parser File
|
|
@cindex compiling the parser
|
|
|
|
Here is how to compile and run the parser file:
|
|
|
|
@example
|
|
@group
|
|
# @r{List files in current directory.}
|
|
$ @kbd{ls}
|
|
rpcalc.tab.c rpcalc.y
|
|
@end group
|
|
|
|
@group
|
|
# @r{Compile the Bison parser.}
|
|
# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
|
|
$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
|
|
@end group
|
|
|
|
@group
|
|
# @r{List files again.}
|
|
$ @kbd{ls}
|
|
rpcalc rpcalc.tab.c rpcalc.y
|
|
@end group
|
|
@end example
|
|
|
|
The file @file{rpcalc} now contains the executable code. Here is an
|
|
example session using @code{rpcalc}.
|
|
|
|
@example
|
|
$ @kbd{rpcalc}
|
|
@kbd{4 9 +}
|
|
13
|
|
@kbd{3 7 + 3 4 5 *+-}
|
|
-13
|
|
@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
|
|
13
|
|
@kbd{5 6 / 4 n +}
|
|
-3.166666667
|
|
@kbd{3 4 ^} @r{Exponentiation}
|
|
81
|
|
@kbd{^D} @r{End-of-file indicator}
|
|
$
|
|
@end example
|
|
|
|
@node Infix Calc
|
|
@section Infix Notation Calculator: @code{calc}
|
|
@cindex infix notation calculator
|
|
@cindex @code{calc}
|
|
@cindex calculator, infix notation
|
|
|
|
We now modify rpcalc to handle infix operators instead of postfix. Infix
|
|
notation involves the concept of operator precedence and the need for
|
|
parentheses nested to arbitrary depth. Here is the Bison code for
|
|
@file{calc.y}, an infix desk-top calculator.
|
|
|
|
@example
|
|
/* Infix notation calculator. */
|
|
|
|
%@{
|
|
#define YYSTYPE double
|
|
#include <math.h>
|
|
#include <stdio.h>
|
|
int yylex (void);
|
|
void yyerror (char const *);
|
|
%@}
|
|
|
|
/* Bison declarations. */
|
|
%token NUM
|
|
%left '-' '+'
|
|
%left '*' '/'
|
|
%left NEG /* negation--unary minus */
|
|
%right '^' /* exponentiation */
|
|
|
|
%% /* The grammar follows. */
|
|
input: /* empty */
|
|
| input line
|
|
;
|
|
|
|
line: '\n'
|
|
| exp '\n' @{ printf ("\t%.10g\n", $1); @}
|
|
;
|
|
|
|
exp: NUM @{ $$ = $1; @}
|
|
| exp '+' exp @{ $$ = $1 + $3; @}
|
|
| exp '-' exp @{ $$ = $1 - $3; @}
|
|
| exp '*' exp @{ $$ = $1 * $3; @}
|
|
| exp '/' exp @{ $$ = $1 / $3; @}
|
|
| '-' exp %prec NEG @{ $$ = -$2; @}
|
|
| exp '^' exp @{ $$ = pow ($1, $3); @}
|
|
| '(' exp ')' @{ $$ = $2; @}
|
|
;
|
|
%%
|
|
@end example
|
|
|
|
@noindent
|
|
The functions @code{yylex}, @code{yyerror} and @code{main} can be the
|
|
same as before.
|
|
|
|
There are two important new features shown in this code.
|
|
|
|
In the second section (Bison declarations), @code{%left} declares token
|
|
types and says they are left-associative operators. The declarations
|
|
@code{%left} and @code{%right} (right associativity) take the place of
|
|
@code{%token} which is used to declare a token type name without
|
|
associativity. (These tokens are single-character literals, which
|
|
ordinarily don't need to be declared. We declare them here to specify
|
|
the associativity.)
|
|
|
|
Operator precedence is determined by the line ordering of the
|
|
declarations; the higher the line number of the declaration (lower on
|
|
the page or screen), the higher the precedence. Hence, exponentiation
|
|
has the highest precedence, unary minus (@code{NEG}) is next, followed
|
|
by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
|
|
Precedence}.
|
|
|
|
The other important new feature is the @code{%prec} in the grammar
|
|
section for the unary minus operator. The @code{%prec} simply instructs
|
|
Bison that the rule @samp{| '-' exp} has the same precedence as
|
|
@code{NEG}---in this case the next-to-highest. @xref{Contextual
|
|
Precedence, ,Context-Dependent Precedence}.
|
|
|
|
Here is a sample run of @file{calc.y}:
|
|
|
|
@need 500
|
|
@example
|
|
$ @kbd{calc}
|
|
@kbd{4 + 4.5 - (34/(8*3+-3))}
|
|
6.880952381
|
|
@kbd{-56 + 2}
|
|
-54
|
|
@kbd{3 ^ 2}
|
|
9
|
|
@end example
|
|
|
|
@node Simple Error Recovery
|
|
@section Simple Error Recovery
|
|
@cindex error recovery, simple
|
|
|
|
Up to this point, this manual has not addressed the issue of @dfn{error
|
|
recovery}---how to continue parsing after the parser detects a syntax
|
|
error. All we have handled is error reporting with @code{yyerror}.
|
|
Recall that by default @code{yyparse} returns after calling
|
|
@code{yyerror}. This means that an erroneous input line causes the
|
|
calculator program to exit. Now we show how to rectify this deficiency.
|
|
|
|
The Bison language itself includes the reserved word @code{error}, which
|
|
may be included in the grammar rules. In the example below it has
|
|
been added to one of the alternatives for @code{line}:
|
|
|
|
@example
|
|
@group
|
|
line: '\n'
|
|
| exp '\n' @{ printf ("\t%.10g\n", $1); @}
|
|
| error '\n' @{ yyerrok; @}
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
This addition to the grammar allows for simple error recovery in the
|
|
event of a syntax error. If an expression that cannot be evaluated is
|
|
read, the error will be recognized by the third rule for @code{line},
|
|
and parsing will continue. (The @code{yyerror} function is still called
|
|
upon to print its message as well.) The action executes the statement
|
|
@code{yyerrok}, a macro defined automatically by Bison; its meaning is
|
|
that error recovery is complete (@pxref{Error Recovery}). Note the
|
|
difference between @code{yyerrok} and @code{yyerror}; neither one is a
|
|
misprint.
|
|
|
|
This form of error recovery deals with syntax errors. There are other
|
|
kinds of errors; for example, division by zero, which raises an exception
|
|
signal that is normally fatal. A real calculator program must handle this
|
|
signal and use @code{longjmp} to return to @code{main} and resume parsing
|
|
input lines; it would also have to discard the rest of the current line of
|
|
input. We won't discuss this issue further because it is not specific to
|
|
Bison programs.
|
|
|
|
@node Location Tracking Calc
|
|
@section Location Tracking Calculator: @code{ltcalc}
|
|
@cindex location tracking calculator
|
|
@cindex @code{ltcalc}
|
|
@cindex calculator, location tracking
|
|
|
|
This example extends the infix notation calculator with location
|
|
tracking. This feature will be used to improve the error messages. For
|
|
the sake of clarity, this example is a simple integer calculator, since
|
|
most of the work needed to use locations will be done in the lexical
|
|
analyzer.
|
|
|
|
@menu
|
|
* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
|
|
* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
|
|
* Lexer: Ltcalc Lexer. The lexical analyzer.
|
|
@end menu
|
|
|
|
@node Ltcalc Decls
|
|
@subsection Declarations for @code{ltcalc}
|
|
|
|
The C and Bison declarations for the location tracking calculator are
|
|
the same as the declarations for the infix notation calculator.
|
|
|
|
@example
|
|
/* Location tracking calculator. */
|
|
|
|
%@{
|
|
#define YYSTYPE int
|
|
#include <math.h>
|
|
int yylex (void);
|
|
void yyerror (char const *);
|
|
%@}
|
|
|
|
/* Bison declarations. */
|
|
%token NUM
|
|
|
|
%left '-' '+'
|
|
%left '*' '/'
|
|
%left NEG
|
|
%right '^'
|
|
|
|
%% /* The grammar follows. */
|
|
@end example
|
|
|
|
@noindent
|
|
Note there are no declarations specific to locations. Defining a data
|
|
type for storing locations is not needed: we will use the type provided
|
|
by default (@pxref{Location Type, ,Data Types of Locations}), which is a
|
|
four member structure with the following integer fields:
|
|
@code{first_line}, @code{first_column}, @code{last_line} and
|
|
@code{last_column}.
|
|
|
|
@node Ltcalc Rules
|
|
@subsection Grammar Rules for @code{ltcalc}
|
|
|
|
Whether handling locations or not has no effect on the syntax of your
|
|
language. Therefore, grammar rules for this example will be very close
|
|
to those of the previous example: we will only modify them to benefit
|
|
from the new information.
|
|
|
|
Here, we will use locations to report divisions by zero, and locate the
|
|
wrong expressions or subexpressions.
|
|
|
|
@example
|
|
@group
|
|
input : /* empty */
|
|
| input line
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
line : '\n'
|
|
| exp '\n' @{ printf ("%d\n", $1); @}
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
exp : NUM @{ $$ = $1; @}
|
|
| exp '+' exp @{ $$ = $1 + $3; @}
|
|
| exp '-' exp @{ $$ = $1 - $3; @}
|
|
| exp '*' exp @{ $$ = $1 * $3; @}
|
|
@end group
|
|
@group
|
|
| exp '/' exp
|
|
@{
|
|
if ($3)
|
|
$$ = $1 / $3;
|
|
else
|
|
@{
|
|
$$ = 1;
|
|
fprintf (stderr, "%d.%d-%d.%d: division by zero",
|
|
@@3.first_line, @@3.first_column,
|
|
@@3.last_line, @@3.last_column);
|
|
@}
|
|
@}
|
|
@end group
|
|
@group
|
|
| '-' exp %preg NEG @{ $$ = -$2; @}
|
|
| exp '^' exp @{ $$ = pow ($1, $3); @}
|
|
| '(' exp ')' @{ $$ = $2; @}
|
|
@end group
|
|
@end example
|
|
|
|
This code shows how to reach locations inside of semantic actions, by
|
|
using the pseudo-variables @code{@@@var{n}} for rule components, and the
|
|
pseudo-variable @code{@@$} for groupings.
|
|
|
|
We don't need to assign a value to @code{@@$}: the output parser does it
|
|
automatically. By default, before executing the C code of each action,
|
|
@code{@@$} is set to range from the beginning of @code{@@1} to the end
|
|
of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
|
|
can be redefined (@pxref{Location Default Action, , Default Action for
|
|
Locations}), and for very specific rules, @code{@@$} can be computed by
|
|
hand.
|
|
|
|
@node Ltcalc Lexer
|
|
@subsection The @code{ltcalc} Lexical Analyzer.
|
|
|
|
Until now, we relied on Bison's defaults to enable location
|
|
tracking. The next step is to rewrite the lexical analyzer, and make it
|
|
able to feed the parser with the token locations, as it already does for
|
|
semantic values.
|
|
|
|
To this end, we must take into account every single character of the
|
|
input text, to avoid the computed locations of being fuzzy or wrong:
|
|
|
|
@example
|
|
@group
|
|
int
|
|
yylex (void)
|
|
@{
|
|
int c;
|
|
@end group
|
|
|
|
@group
|
|
/* Skip white space. */
|
|
while ((c = getchar ()) == ' ' || c == '\t')
|
|
++yylloc.last_column;
|
|
@end group
|
|
|
|
@group
|
|
/* Step. */
|
|
yylloc.first_line = yylloc.last_line;
|
|
yylloc.first_column = yylloc.last_column;
|
|
@end group
|
|
|
|
@group
|
|
/* Process numbers. */
|
|
if (isdigit (c))
|
|
@{
|
|
yylval = c - '0';
|
|
++yylloc.last_column;
|
|
while (isdigit (c = getchar ()))
|
|
@{
|
|
++yylloc.last_column;
|
|
yylval = yylval * 10 + c - '0';
|
|
@}
|
|
ungetc (c, stdin);
|
|
return NUM;
|
|
@}
|
|
@end group
|
|
|
|
/* Return end-of-input. */
|
|
if (c == EOF)
|
|
return 0;
|
|
|
|
/* Return a single char, and update location. */
|
|
if (c == '\n')
|
|
@{
|
|
++yylloc.last_line;
|
|
yylloc.last_column = 0;
|
|
@}
|
|
else
|
|
++yylloc.last_column;
|
|
return c;
|
|
@}
|
|
@end example
|
|
|
|
Basically, the lexical analyzer performs the same processing as before:
|
|
it skips blanks and tabs, and reads numbers or single-character tokens.
|
|
In addition, it updates @code{yylloc}, the global variable (of type
|
|
@code{YYLTYPE}) containing the token's location.
|
|
|
|
Now, each time this function returns a token, the parser has its number
|
|
as well as its semantic value, and its location in the text. The last
|
|
needed change is to initialize @code{yylloc}, for example in the
|
|
controlling function:
|
|
|
|
@example
|
|
@group
|
|
int
|
|
main (void)
|
|
@{
|
|
yylloc.first_line = yylloc.last_line = 1;
|
|
yylloc.first_column = yylloc.last_column = 0;
|
|
return yyparse ();
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
Remember that computing locations is not a matter of syntax. Every
|
|
character must be associated to a location update, whether it is in
|
|
valid input, in comments, in literal strings, and so on.
|
|
|
|
@node Multi-function Calc
|
|
@section Multi-Function Calculator: @code{mfcalc}
|
|
@cindex multi-function calculator
|
|
@cindex @code{mfcalc}
|
|
@cindex calculator, multi-function
|
|
|
|
Now that the basics of Bison have been discussed, it is time to move on to
|
|
a more advanced problem. The above calculators provided only five
|
|
functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
|
|
be nice to have a calculator that provides other mathematical functions such
|
|
as @code{sin}, @code{cos}, etc.
|
|
|
|
It is easy to add new operators to the infix calculator as long as they are
|
|
only single-character literals. The lexical analyzer @code{yylex} passes
|
|
back all nonnumber characters as tokens, so new grammar rules suffice for
|
|
adding a new operator. But we want something more flexible: built-in
|
|
functions whose syntax has this form:
|
|
|
|
@example
|
|
@var{function_name} (@var{argument})
|
|
@end example
|
|
|
|
@noindent
|
|
At the same time, we will add memory to the calculator, by allowing you
|
|
to create named variables, store values in them, and use them later.
|
|
Here is a sample session with the multi-function calculator:
|
|
|
|
@example
|
|
$ @kbd{mfcalc}
|
|
@kbd{pi = 3.141592653589}
|
|
3.1415926536
|
|
@kbd{sin(pi)}
|
|
0.0000000000
|
|
@kbd{alpha = beta1 = 2.3}
|
|
2.3000000000
|
|
@kbd{alpha}
|
|
2.3000000000
|
|
@kbd{ln(alpha)}
|
|
0.8329091229
|
|
@kbd{exp(ln(beta1))}
|
|
2.3000000000
|
|
$
|
|
@end example
|
|
|
|
Note that multiple assignment and nested function calls are permitted.
|
|
|
|
@menu
|
|
* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
|
|
* Rules: Mfcalc Rules. Grammar rules for the calculator.
|
|
* Symtab: Mfcalc Symtab. Symbol table management subroutines.
|
|
@end menu
|
|
|
|
@node Mfcalc Decl
|
|
@subsection Declarations for @code{mfcalc}
|
|
|
|
Here are the C and Bison declarations for the multi-function calculator.
|
|
|
|
@smallexample
|
|
@group
|
|
%@{
|
|
#include <math.h> /* For math functions, cos(), sin(), etc. */
|
|
#include "calc.h" /* Contains definition of `symrec'. */
|
|
int yylex (void);
|
|
void yyerror (char const *);
|
|
%@}
|
|
@end group
|
|
@group
|
|
%union @{
|
|
double val; /* For returning numbers. */
|
|
symrec *tptr; /* For returning symbol-table pointers. */
|
|
@}
|
|
@end group
|
|
%token <val> NUM /* Simple double precision number. */
|
|
%token <tptr> VAR FNCT /* Variable and Function. */
|
|
%type <val> exp
|
|
|
|
@group
|
|
%right '='
|
|
%left '-' '+'
|
|
%left '*' '/'
|
|
%left NEG /* negation--unary minus */
|
|
%right '^' /* exponentiation */
|
|
@end group
|
|
%% /* The grammar follows. */
|
|
@end smallexample
|
|
|
|
The above grammar introduces only two new features of the Bison language.
|
|
These features allow semantic values to have various data types
|
|
(@pxref{Multiple Types, ,More Than One Value Type}).
|
|
|
|
The @code{%union} declaration specifies the entire list of possible types;
|
|
this is instead of defining @code{YYSTYPE}. The allowable types are now
|
|
double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
|
|
the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
|
|
|
|
Since values can now have various types, it is necessary to associate a
|
|
type with each grammar symbol whose semantic value is used. These symbols
|
|
are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
|
|
declarations are augmented with information about their data type (placed
|
|
between angle brackets).
|
|
|
|
The Bison construct @code{%type} is used for declaring nonterminal
|
|
symbols, just as @code{%token} is used for declaring token types. We
|
|
have not used @code{%type} before because nonterminal symbols are
|
|
normally declared implicitly by the rules that define them. But
|
|
@code{exp} must be declared explicitly so we can specify its value type.
|
|
@xref{Type Decl, ,Nonterminal Symbols}.
|
|
|
|
@node Mfcalc Rules
|
|
@subsection Grammar Rules for @code{mfcalc}
|
|
|
|
Here are the grammar rules for the multi-function calculator.
|
|
Most of them are copied directly from @code{calc}; three rules,
|
|
those which mention @code{VAR} or @code{FNCT}, are new.
|
|
|
|
@smallexample
|
|
@group
|
|
input: /* empty */
|
|
| input line
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
line:
|
|
'\n'
|
|
| exp '\n' @{ printf ("\t%.10g\n", $1); @}
|
|
| error '\n' @{ yyerrok; @}
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
exp: NUM @{ $$ = $1; @}
|
|
| VAR @{ $$ = $1->value.var; @}
|
|
| VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
|
|
| FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
|
|
| exp '+' exp @{ $$ = $1 + $3; @}
|
|
| exp '-' exp @{ $$ = $1 - $3; @}
|
|
| exp '*' exp @{ $$ = $1 * $3; @}
|
|
| exp '/' exp @{ $$ = $1 / $3; @}
|
|
| '-' exp %prec NEG @{ $$ = -$2; @}
|
|
| exp '^' exp @{ $$ = pow ($1, $3); @}
|
|
| '(' exp ')' @{ $$ = $2; @}
|
|
;
|
|
@end group
|
|
/* End of grammar. */
|
|
%%
|
|
@end smallexample
|
|
|
|
@node Mfcalc Symtab
|
|
@subsection The @code{mfcalc} Symbol Table
|
|
@cindex symbol table example
|
|
|
|
The multi-function calculator requires a symbol table to keep track of the
|
|
names and meanings of variables and functions. This doesn't affect the
|
|
grammar rules (except for the actions) or the Bison declarations, but it
|
|
requires some additional C functions for support.
|
|
|
|
The symbol table itself consists of a linked list of records. Its
|
|
definition, which is kept in the header @file{calc.h}, is as follows. It
|
|
provides for either functions or variables to be placed in the table.
|
|
|
|
@smallexample
|
|
@group
|
|
/* Function type. */
|
|
typedef double (*func_t) (double);
|
|
@end group
|
|
|
|
@group
|
|
/* Data type for links in the chain of symbols. */
|
|
struct symrec
|
|
@{
|
|
char *name; /* name of symbol */
|
|
int type; /* type of symbol: either VAR or FNCT */
|
|
union
|
|
@{
|
|
double var; /* value of a VAR */
|
|
func_t fnctptr; /* value of a FNCT */
|
|
@} value;
|
|
struct symrec *next; /* link field */
|
|
@};
|
|
@end group
|
|
|
|
@group
|
|
typedef struct symrec symrec;
|
|
|
|
/* The symbol table: a chain of `struct symrec'. */
|
|
extern symrec *sym_table;
|
|
|
|
symrec *putsym (char const *, func_t);
|
|
symrec *getsym (char const *);
|
|
@end group
|
|
@end smallexample
|
|
|
|
The new version of @code{main} includes a call to @code{init_table}, a
|
|
function that initializes the symbol table. Here it is, and
|
|
@code{init_table} as well:
|
|
|
|
@smallexample
|
|
#include <stdio.h>
|
|
|
|
@group
|
|
/* Called by yyparse on error. */
|
|
void
|
|
yyerror (char const *s)
|
|
@{
|
|
printf ("%s\n", s);
|
|
@}
|
|
@end group
|
|
|
|
@group
|
|
struct init
|
|
@{
|
|
char const *fname;
|
|
double (*fnct) (double);
|
|
@};
|
|
@end group
|
|
|
|
@group
|
|
struct init const arith_fncts[] =
|
|
@{
|
|
"sin", sin,
|
|
"cos", cos,
|
|
"atan", atan,
|
|
"ln", log,
|
|
"exp", exp,
|
|
"sqrt", sqrt,
|
|
0, 0
|
|
@};
|
|
@end group
|
|
|
|
@group
|
|
/* The symbol table: a chain of `struct symrec'. */
|
|
symrec *sym_table;
|
|
@end group
|
|
|
|
@group
|
|
/* Put arithmetic functions in table. */
|
|
void
|
|
init_table (void)
|
|
@{
|
|
int i;
|
|
symrec *ptr;
|
|
for (i = 0; arith_fncts[i].fname != 0; i++)
|
|
@{
|
|
ptr = putsym (arith_fncts[i].fname, FNCT);
|
|
ptr->value.fnctptr = arith_fncts[i].fnct;
|
|
@}
|
|
@}
|
|
@end group
|
|
|
|
@group
|
|
int
|
|
main (void)
|
|
@{
|
|
init_table ();
|
|
return yyparse ();
|
|
@}
|
|
@end group
|
|
@end smallexample
|
|
|
|
By simply editing the initialization list and adding the necessary include
|
|
files, you can add additional functions to the calculator.
|
|
|
|
Two important functions allow look-up and installation of symbols in the
|
|
symbol table. The function @code{putsym} is passed a name and the type
|
|
(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
|
|
linked to the front of the list, and a pointer to the object is returned.
|
|
The function @code{getsym} is passed the name of the symbol to look up. If
|
|
found, a pointer to that symbol is returned; otherwise zero is returned.
|
|
|
|
@smallexample
|
|
symrec *
|
|
putsym (char const *sym_name, int sym_type)
|
|
@{
|
|
symrec *ptr;
|
|
ptr = (symrec *) malloc (sizeof (symrec));
|
|
ptr->name = (char *) malloc (strlen (sym_name) + 1);
|
|
strcpy (ptr->name,sym_name);
|
|
ptr->type = sym_type;
|
|
ptr->value.var = 0; /* Set value to 0 even if fctn. */
|
|
ptr->next = (struct symrec *)sym_table;
|
|
sym_table = ptr;
|
|
return ptr;
|
|
@}
|
|
|
|
symrec *
|
|
getsym (char const *sym_name)
|
|
@{
|
|
symrec *ptr;
|
|
for (ptr = sym_table; ptr != (symrec *) 0;
|
|
ptr = (symrec *)ptr->next)
|
|
if (strcmp (ptr->name,sym_name) == 0)
|
|
return ptr;
|
|
return 0;
|
|
@}
|
|
@end smallexample
|
|
|
|
The function @code{yylex} must now recognize variables, numeric values, and
|
|
the single-character arithmetic operators. Strings of alphanumeric
|
|
characters with a leading non-digit are recognized as either variables or
|
|
functions depending on what the symbol table says about them.
|
|
|
|
The string is passed to @code{getsym} for look up in the symbol table. If
|
|
the name appears in the table, a pointer to its location and its type
|
|
(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
|
|
already in the table, then it is installed as a @code{VAR} using
|
|
@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
|
|
returned to @code{yyparse}.
|
|
|
|
No change is needed in the handling of numeric values and arithmetic
|
|
operators in @code{yylex}.
|
|
|
|
@smallexample
|
|
@group
|
|
#include <ctype.h>
|
|
@end group
|
|
|
|
@group
|
|
int
|
|
yylex (void)
|
|
@{
|
|
int c;
|
|
|
|
/* Ignore white space, get first nonwhite character. */
|
|
while ((c = getchar ()) == ' ' || c == '\t');
|
|
|
|
if (c == EOF)
|
|
return 0;
|
|
@end group
|
|
|
|
@group
|
|
/* Char starts a number => parse the number. */
|
|
if (c == '.' || isdigit (c))
|
|
@{
|
|
ungetc (c, stdin);
|
|
scanf ("%lf", &yylval.val);
|
|
return NUM;
|
|
@}
|
|
@end group
|
|
|
|
@group
|
|
/* Char starts an identifier => read the name. */
|
|
if (isalpha (c))
|
|
@{
|
|
symrec *s;
|
|
static char *symbuf = 0;
|
|
static int length = 0;
|
|
int i;
|
|
@end group
|
|
|
|
@group
|
|
/* Initially make the buffer long enough
|
|
for a 40-character symbol name. */
|
|
if (length == 0)
|
|
length = 40, symbuf = (char *)malloc (length + 1);
|
|
|
|
i = 0;
|
|
do
|
|
@end group
|
|
@group
|
|
@{
|
|
/* If buffer is full, make it bigger. */
|
|
if (i == length)
|
|
@{
|
|
length *= 2;
|
|
symbuf = (char *) realloc (symbuf, length + 1);
|
|
@}
|
|
/* Add this character to the buffer. */
|
|
symbuf[i++] = c;
|
|
/* Get another character. */
|
|
c = getchar ();
|
|
@}
|
|
@end group
|
|
@group
|
|
while (isalnum (c));
|
|
|
|
ungetc (c, stdin);
|
|
symbuf[i] = '\0';
|
|
@end group
|
|
|
|
@group
|
|
s = getsym (symbuf);
|
|
if (s == 0)
|
|
s = putsym (symbuf, VAR);
|
|
yylval.tptr = s;
|
|
return s->type;
|
|
@}
|
|
|
|
/* Any other character is a token by itself. */
|
|
return c;
|
|
@}
|
|
@end group
|
|
@end smallexample
|
|
|
|
This program is both powerful and flexible. You may easily add new
|
|
functions, and it is a simple job to modify this code to install
|
|
predefined variables such as @code{pi} or @code{e} as well.
|
|
|
|
@node Exercises
|
|
@section Exercises
|
|
@cindex exercises
|
|
|
|
@enumerate
|
|
@item
|
|
Add some new functions from @file{math.h} to the initialization list.
|
|
|
|
@item
|
|
Add another array that contains constants and their values. Then
|
|
modify @code{init_table} to add these constants to the symbol table.
|
|
It will be easiest to give the constants type @code{VAR}.
|
|
|
|
@item
|
|
Make the program report an error if the user refers to an
|
|
uninitialized variable in any way except to store a value in it.
|
|
@end enumerate
|
|
|
|
@node Grammar File
|
|
@chapter Bison Grammar Files
|
|
|
|
Bison takes as input a context-free grammar specification and produces a
|
|
C-language function that recognizes correct instances of the grammar.
|
|
|
|
The Bison grammar input file conventionally has a name ending in @samp{.y}.
|
|
@xref{Invocation, ,Invoking Bison}.
|
|
|
|
@menu
|
|
* Grammar Outline:: Overall layout of the grammar file.
|
|
* Symbols:: Terminal and nonterminal symbols.
|
|
* Rules:: How to write grammar rules.
|
|
* Recursion:: Writing recursive rules.
|
|
* Semantics:: Semantic values and actions.
|
|
* Locations:: Locations and actions.
|
|
* Declarations:: All kinds of Bison declarations are described here.
|
|
* Multiple Parsers:: Putting more than one Bison parser in one program.
|
|
@end menu
|
|
|
|
@node Grammar Outline
|
|
@section Outline of a Bison Grammar
|
|
|
|
A Bison grammar file has four main sections, shown here with the
|
|
appropriate delimiters:
|
|
|
|
@example
|
|
%@{
|
|
@var{Prologue}
|
|
%@}
|
|
|
|
@var{Bison declarations}
|
|
|
|
%%
|
|
@var{Grammar rules}
|
|
%%
|
|
|
|
@var{Epilogue}
|
|
@end example
|
|
|
|
Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
|
|
As a @acronym{GNU} extension, @samp{//} introduces a comment that
|
|
continues until end of line.
|
|
|
|
@menu
|
|
* Prologue:: Syntax and usage of the prologue.
|
|
* Bison Declarations:: Syntax and usage of the Bison declarations section.
|
|
* Grammar Rules:: Syntax and usage of the grammar rules section.
|
|
* Epilogue:: Syntax and usage of the epilogue.
|
|
@end menu
|
|
|
|
@node Prologue
|
|
@subsection The prologue
|
|
@cindex declarations section
|
|
@cindex Prologue
|
|
@cindex declarations
|
|
|
|
The @var{Prologue} section contains macro definitions and
|
|
declarations of functions and variables that are used in the actions in the
|
|
grammar rules. These are copied to the beginning of the parser file so
|
|
that they precede the definition of @code{yyparse}. You can use
|
|
@samp{#include} to get the declarations from a header file. If you don't
|
|
need any C declarations, you may omit the @samp{%@{} and @samp{%@}}
|
|
delimiters that bracket this section.
|
|
|
|
You may have more than one @var{Prologue} section, intermixed with the
|
|
@var{Bison declarations}. This allows you to have C and Bison
|
|
declarations that refer to each other. For example, the @code{%union}
|
|
declaration may use types defined in a header file, and you may wish to
|
|
prototype functions that take arguments of type @code{YYSTYPE}. This
|
|
can be done with two @var{Prologue} blocks, one before and one after the
|
|
@code{%union} declaration.
|
|
|
|
@smallexample
|
|
%@{
|
|
#include <stdio.h>
|
|
#include "ptypes.h"
|
|
%@}
|
|
|
|
%union @{
|
|
long int n;
|
|
tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
|
|
@}
|
|
|
|
%@{
|
|
static void print_token_value (FILE *, int, YYSTYPE);
|
|
#define YYPRINT(F, N, L) print_token_value (F, N, L)
|
|
%@}
|
|
|
|
@dots{}
|
|
@end smallexample
|
|
|
|
@node Bison Declarations
|
|
@subsection The Bison Declarations Section
|
|
@cindex Bison declarations (introduction)
|
|
@cindex declarations, Bison (introduction)
|
|
|
|
The @var{Bison declarations} section contains declarations that define
|
|
terminal and nonterminal symbols, specify precedence, and so on.
|
|
In some simple grammars you may not need any declarations.
|
|
@xref{Declarations, ,Bison Declarations}.
|
|
|
|
@node Grammar Rules
|
|
@subsection The Grammar Rules Section
|
|
@cindex grammar rules section
|
|
@cindex rules section for grammar
|
|
|
|
The @dfn{grammar rules} section contains one or more Bison grammar
|
|
rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
|
|
|
|
There must always be at least one grammar rule, and the first
|
|
@samp{%%} (which precedes the grammar rules) may never be omitted even
|
|
if it is the first thing in the file.
|
|
|
|
@node Epilogue
|
|
@subsection The epilogue
|
|
@cindex additional C code section
|
|
@cindex epilogue
|
|
@cindex C code, section for additional
|
|
|
|
The @var{Epilogue} is copied verbatim to the end of the parser file, just as
|
|
the @var{Prologue} is copied to the beginning. This is the most convenient
|
|
place to put anything that you want to have in the parser file but which need
|
|
not come before the definition of @code{yyparse}. For example, the
|
|
definitions of @code{yylex} and @code{yyerror} often go here. Because
|
|
C requires functions to be declared before being used, you often need
|
|
to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
|
|
even if you define them int he Epilogue.
|
|
@xref{Interface, ,Parser C-Language Interface}.
|
|
|
|
If the last section is empty, you may omit the @samp{%%} that separates it
|
|
from the grammar rules.
|
|
|
|
The Bison parser itself contains many macros and identifiers whose
|
|
names start with @samp{yy} or @samp{YY}, so it is a
|
|
good idea to avoid using any such names (except those documented in this
|
|
manual) in the epilogue of the grammar file.
|
|
|
|
@node Symbols
|
|
@section Symbols, Terminal and Nonterminal
|
|
@cindex nonterminal symbol
|
|
@cindex terminal symbol
|
|
@cindex token type
|
|
@cindex symbol
|
|
|
|
@dfn{Symbols} in Bison grammars represent the grammatical classifications
|
|
of the language.
|
|
|
|
A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
|
|
class of syntactically equivalent tokens. You use the symbol in grammar
|
|
rules to mean that a token in that class is allowed. The symbol is
|
|
represented in the Bison parser by a numeric code, and the @code{yylex}
|
|
function returns a token type code to indicate what kind of token has been
|
|
read. You don't need to know what the code value is; you can use the
|
|
symbol to stand for it.
|
|
|
|
A @dfn{nonterminal symbol} stands for a class of syntactically equivalent
|
|
groupings. The symbol name is used in writing grammar rules. By convention,
|
|
it should be all lower case.
|
|
|
|
Symbol names can contain letters, digits (not at the beginning),
|
|
underscores and periods. Periods make sense only in nonterminals.
|
|
|
|
There are three ways of writing terminal symbols in the grammar:
|
|
|
|
@itemize @bullet
|
|
@item
|
|
A @dfn{named token type} is written with an identifier, like an
|
|
identifier in C@. By convention, it should be all upper case. Each
|
|
such name must be defined with a Bison declaration such as
|
|
@code{%token}. @xref{Token Decl, ,Token Type Names}.
|
|
|
|
@item
|
|
@cindex character token
|
|
@cindex literal token
|
|
@cindex single-character literal
|
|
A @dfn{character token type} (or @dfn{literal character token}) is
|
|
written in the grammar using the same syntax used in C for character
|
|
constants; for example, @code{'+'} is a character token type. A
|
|
character token type doesn't need to be declared unless you need to
|
|
specify its semantic value data type (@pxref{Value Type, ,Data Types of
|
|
Semantic Values}), associativity, or precedence (@pxref{Precedence,
|
|
,Operator Precedence}).
|
|
|
|
By convention, a character token type is used only to represent a
|
|
token that consists of that particular character. Thus, the token
|
|
type @code{'+'} is used to represent the character @samp{+} as a
|
|
token. Nothing enforces this convention, but if you depart from it,
|
|
your program will confuse other readers.
|
|
|
|
All the usual escape sequences used in character literals in C can be
|
|
used in Bison as well, but you must not use the null character as a
|
|
character literal because its numeric code, zero, signifies
|
|
end-of-input (@pxref{Calling Convention, ,Calling Convention
|
|
for @code{yylex}}). Also, unlike standard C, trigraphs have no
|
|
special meaning in Bison character literals, nor is backslash-newline
|
|
allowed.
|
|
|
|
@item
|
|
@cindex string token
|
|
@cindex literal string token
|
|
@cindex multicharacter literal
|
|
A @dfn{literal string token} is written like a C string constant; for
|
|
example, @code{"<="} is a literal string token. A literal string token
|
|
doesn't need to be declared unless you need to specify its semantic
|
|
value data type (@pxref{Value Type}), associativity, or precedence
|
|
(@pxref{Precedence}).
|
|
|
|
You can associate the literal string token with a symbolic name as an
|
|
alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
|
|
Declarations}). If you don't do that, the lexical analyzer has to
|
|
retrieve the token number for the literal string token from the
|
|
@code{yytname} table (@pxref{Calling Convention}).
|
|
|
|
@strong{Warning}: literal string tokens do not work in Yacc.
|
|
|
|
By convention, a literal string token is used only to represent a token
|
|
that consists of that particular string. Thus, you should use the token
|
|
type @code{"<="} to represent the string @samp{<=} as a token. Bison
|
|
does not enforce this convention, but if you depart from it, people who
|
|
read your program will be confused.
|
|
|
|
All the escape sequences used in string literals in C can be used in
|
|
Bison as well, except that you must not use a null character within a
|
|
string literal. Also, unlike Standard C, trigraphs have no special
|
|
meaning in Bison string literals, nor is backslash-newline allowed. A
|
|
literal string token must contain two or more characters; for a token
|
|
containing just one character, use a character token (see above).
|
|
@end itemize
|
|
|
|
How you choose to write a terminal symbol has no effect on its
|
|
grammatical meaning. That depends only on where it appears in rules and
|
|
on when the parser function returns that symbol.
|
|
|
|
The value returned by @code{yylex} is always one of the terminal
|
|
symbols, except that a zero or negative value signifies end-of-input.
|
|
Whichever way you write the token type in the grammar rules, you write
|
|
it the same way in the definition of @code{yylex}. The numeric code
|
|
for a character token type is simply the positive numeric code of the
|
|
character, so @code{yylex} can use the identical value to generate the
|
|
requisite code, though you may need to convert it to @code{unsigned
|
|
char} to avoid sign-extension on hosts where @code{char} is signed.
|
|
Each named token type becomes a C macro in
|
|
the parser file, so @code{yylex} can use the name to stand for the code.
|
|
(This is why periods don't make sense in terminal symbols.)
|
|
@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
|
|
|
|
If @code{yylex} is defined in a separate file, you need to arrange for the
|
|
token-type macro definitions to be available there. Use the @samp{-d}
|
|
option when you run Bison, so that it will write these macro definitions
|
|
into a separate header file @file{@var{name}.tab.h} which you can include
|
|
in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
|
|
|
|
If you want to write a grammar that is portable to any Standard C
|
|
host, you must use only non-null character tokens taken from the basic
|
|
execution character set of Standard C@. This set consists of the ten
|
|
digits, the 52 lower- and upper-case English letters, and the
|
|
characters in the following C-language string:
|
|
|
|
@example
|
|
"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
|
|
@end example
|
|
|
|
The @code{yylex} function and Bison must use a consistent character
|
|
set and encoding for character tokens. For example, if you run Bison in an
|
|
@acronym{ASCII} environment, but then compile and run the resulting program
|
|
in an environment that uses an incompatible character set like
|
|
@acronym{EBCDIC}, the resulting program may not work because the
|
|
tables generated by Bison will assume @acronym{ASCII} numeric values for
|
|
character tokens. It is standard
|
|
practice for software distributions to contain C source files that
|
|
were generated by Bison in an @acronym{ASCII} environment, so installers on
|
|
platforms that are incompatible with @acronym{ASCII} must rebuild those
|
|
files before compiling them.
|
|
|
|
The symbol @code{error} is a terminal symbol reserved for error recovery
|
|
(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
|
|
In particular, @code{yylex} should never return this value. The default
|
|
value of the error token is 256, unless you explicitly assigned 256 to
|
|
one of your tokens with a @code{%token} declaration.
|
|
|
|
@node Rules
|
|
@section Syntax of Grammar Rules
|
|
@cindex rule syntax
|
|
@cindex grammar rule syntax
|
|
@cindex syntax of grammar rules
|
|
|
|
A Bison grammar rule has the following general form:
|
|
|
|
@example
|
|
@group
|
|
@var{result}: @var{components}@dots{}
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
where @var{result} is the nonterminal symbol that this rule describes,
|
|
and @var{components} are various terminal and nonterminal symbols that
|
|
are put together by this rule (@pxref{Symbols}).
|
|
|
|
For example,
|
|
|
|
@example
|
|
@group
|
|
exp: exp '+' exp
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
says that two groupings of type @code{exp}, with a @samp{+} token in between,
|
|
can be combined into a larger grouping of type @code{exp}.
|
|
|
|
White space in rules is significant only to separate symbols. You can add
|
|
extra white space as you wish.
|
|
|
|
Scattered among the components can be @var{actions} that determine
|
|
the semantics of the rule. An action looks like this:
|
|
|
|
@example
|
|
@{@var{C statements}@}
|
|
@end example
|
|
|
|
@noindent
|
|
Usually there is only one action and it follows the components.
|
|
@xref{Actions}.
|
|
|
|
@findex |
|
|
Multiple rules for the same @var{result} can be written separately or can
|
|
be joined with the vertical-bar character @samp{|} as follows:
|
|
|
|
@ifinfo
|
|
@example
|
|
@var{result}: @var{rule1-components}@dots{}
|
|
| @var{rule2-components}@dots{}
|
|
@dots{}
|
|
;
|
|
@end example
|
|
@end ifinfo
|
|
@iftex
|
|
@example
|
|
@group
|
|
@var{result}: @var{rule1-components}@dots{}
|
|
| @var{rule2-components}@dots{}
|
|
@dots{}
|
|
;
|
|
@end group
|
|
@end example
|
|
@end iftex
|
|
|
|
@noindent
|
|
They are still considered distinct rules even when joined in this way.
|
|
|
|
If @var{components} in a rule is empty, it means that @var{result} can
|
|
match the empty string. For example, here is how to define a
|
|
comma-separated sequence of zero or more @code{exp} groupings:
|
|
|
|
@example
|
|
@group
|
|
expseq: /* empty */
|
|
| expseq1
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
expseq1: exp
|
|
| expseq1 ',' exp
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
It is customary to write a comment @samp{/* empty */} in each rule
|
|
with no components.
|
|
|
|
@node Recursion
|
|
@section Recursive Rules
|
|
@cindex recursive rule
|
|
|
|
A rule is called @dfn{recursive} when its @var{result} nonterminal appears
|
|
also on its right hand side. Nearly all Bison grammars need to use
|
|
recursion, because that is the only way to define a sequence of any number
|
|
of a particular thing. Consider this recursive definition of a
|
|
comma-separated sequence of one or more expressions:
|
|
|
|
@example
|
|
@group
|
|
expseq1: exp
|
|
| expseq1 ',' exp
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@cindex left recursion
|
|
@cindex right recursion
|
|
@noindent
|
|
Since the recursive use of @code{expseq1} is the leftmost symbol in the
|
|
right hand side, we call this @dfn{left recursion}. By contrast, here
|
|
the same construct is defined using @dfn{right recursion}:
|
|
|
|
@example
|
|
@group
|
|
expseq1: exp
|
|
| exp ',' expseq1
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
Any kind of sequence can be defined using either left recursion or right
|
|
recursion, but you should always use left recursion, because it can
|
|
parse a sequence of any number of elements with bounded stack space.
|
|
Right recursion uses up space on the Bison stack in proportion to the
|
|
number of elements in the sequence, because all the elements must be
|
|
shifted onto the stack before the rule can be applied even once.
|
|
@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
|
|
of this.
|
|
|
|
@cindex mutual recursion
|
|
@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
|
|
rule does not appear directly on its right hand side, but does appear
|
|
in rules for other nonterminals which do appear on its right hand
|
|
side.
|
|
|
|
For example:
|
|
|
|
@example
|
|
@group
|
|
expr: primary
|
|
| primary '+' primary
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
primary: constant
|
|
| '(' expr ')'
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
defines two mutually-recursive nonterminals, since each refers to the
|
|
other.
|
|
|
|
@node Semantics
|
|
@section Defining Language Semantics
|
|
@cindex defining language semantics
|
|
@cindex language semantics, defining
|
|
|
|
The grammar rules for a language determine only the syntax. The semantics
|
|
are determined by the semantic values associated with various tokens and
|
|
groupings, and by the actions taken when various groupings are recognized.
|
|
|
|
For example, the calculator calculates properly because the value
|
|
associated with each expression is the proper number; it adds properly
|
|
because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
|
|
the numbers associated with @var{x} and @var{y}.
|
|
|
|
@menu
|
|
* Value Type:: Specifying one data type for all semantic values.
|
|
* Multiple Types:: Specifying several alternative data types.
|
|
* Actions:: An action is the semantic definition of a grammar rule.
|
|
* Action Types:: Specifying data types for actions to operate on.
|
|
* Mid-Rule Actions:: Most actions go at the end of a rule.
|
|
This says when, why and how to use the exceptional
|
|
action in the middle of a rule.
|
|
@end menu
|
|
|
|
@node Value Type
|
|
@subsection Data Types of Semantic Values
|
|
@cindex semantic value type
|
|
@cindex value type, semantic
|
|
@cindex data types of semantic values
|
|
@cindex default data type
|
|
|
|
In a simple program it may be sufficient to use the same data type for
|
|
the semantic values of all language constructs. This was true in the
|
|
@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
|
|
Notation Calculator}).
|
|
|
|
Bison's default is to use type @code{int} for all semantic values. To
|
|
specify some other type, define @code{YYSTYPE} as a macro, like this:
|
|
|
|
@example
|
|
#define YYSTYPE double
|
|
@end example
|
|
|
|
@noindent
|
|
This macro definition must go in the prologue of the grammar file
|
|
(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
|
|
|
|
@node Multiple Types
|
|
@subsection More Than One Value Type
|
|
|
|
In most programs, you will need different data types for different kinds
|
|
of tokens and groupings. For example, a numeric constant may need type
|
|
@code{int} or @code{long int}, while a string constant needs type @code{char *},
|
|
and an identifier might need a pointer to an entry in the symbol table.
|
|
|
|
To use more than one data type for semantic values in one parser, Bison
|
|
requires you to do two things:
|
|
|
|
@itemize @bullet
|
|
@item
|
|
Specify the entire collection of possible data types, with the
|
|
@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
|
|
Value Types}).
|
|
|
|
@item
|
|
Choose one of those types for each symbol (terminal or nonterminal) for
|
|
which semantic values are used. This is done for tokens with the
|
|
@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
|
|
and for groupings with the @code{%type} Bison declaration (@pxref{Type
|
|
Decl, ,Nonterminal Symbols}).
|
|
@end itemize
|
|
|
|
@node Actions
|
|
@subsection Actions
|
|
@cindex action
|
|
@vindex $$
|
|
@vindex $@var{n}
|
|
|
|
An action accompanies a syntactic rule and contains C code to be executed
|
|
each time an instance of that rule is recognized. The task of most actions
|
|
is to compute a semantic value for the grouping built by the rule from the
|
|
semantic values associated with tokens or smaller groupings.
|
|
|
|
An action consists of C statements surrounded by braces, much like a
|
|
compound statement in C@. An action can contain any sequence of C
|
|
statements. Bison does not look for trigraphs, though, so if your C
|
|
code uses trigraphs you should ensure that they do not affect the
|
|
nesting of braces or the boundaries of comments, strings, or character
|
|
literals.
|
|
|
|
An action can be placed at any position in the rule;
|
|
it is executed at that position. Most rules have just one action at the
|
|
end of the rule, following all the components. Actions in the middle of
|
|
a rule are tricky and used only for special purposes (@pxref{Mid-Rule
|
|
Actions, ,Actions in Mid-Rule}).
|
|
|
|
The C code in an action can refer to the semantic values of the components
|
|
matched by the rule with the construct @code{$@var{n}}, which stands for
|
|
the value of the @var{n}th component. The semantic value for the grouping
|
|
being constructed is @code{$$}. Bison translates both of these
|
|
constructs into expressions of the appropriate type when it copies the
|
|
actions into the parser file. @code{$$} is translated to a modifiable
|
|
lvalue, so it can be assigned to.
|
|
|
|
Here is a typical example:
|
|
|
|
@example
|
|
@group
|
|
exp: @dots{}
|
|
| exp '+' exp
|
|
@{ $$ = $1 + $3; @}
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
This rule constructs an @code{exp} from two smaller @code{exp} groupings
|
|
connected by a plus-sign token. In the action, @code{$1} and @code{$3}
|
|
refer to the semantic values of the two component @code{exp} groupings,
|
|
which are the first and third symbols on the right hand side of the rule.
|
|
The sum is stored into @code{$$} so that it becomes the semantic value of
|
|
the addition-expression just recognized by the rule. If there were a
|
|
useful semantic value associated with the @samp{+} token, it could be
|
|
referred to as @code{$2}.
|
|
|
|
Note that the vertical-bar character @samp{|} is really a rule
|
|
separator, and actions are attached to a single rule. This is a
|
|
difference with tools like Flex, for which @samp{|} stands for either
|
|
``or'', or ``the same action as that of the next rule''. In the
|
|
following example, the action is triggered only when @samp{b} is found:
|
|
|
|
@example
|
|
@group
|
|
a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
|
|
@end group
|
|
@end example
|
|
|
|
@cindex default action
|
|
If you don't specify an action for a rule, Bison supplies a default:
|
|
@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
|
|
becomes the value of the whole rule. Of course, the default action is
|
|
valid only if the two data types match. There is no meaningful default
|
|
action for an empty rule; every empty rule must have an explicit action
|
|
unless the rule's value does not matter.
|
|
|
|
@code{$@var{n}} with @var{n} zero or negative is allowed for reference
|
|
to tokens and groupings on the stack @emph{before} those that match the
|
|
current rule. This is a very risky practice, and to use it reliably
|
|
you must be certain of the context in which the rule is applied. Here
|
|
is a case in which you can use this reliably:
|
|
|
|
@example
|
|
@group
|
|
foo: expr bar '+' expr @{ @dots{} @}
|
|
| expr bar '-' expr @{ @dots{} @}
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
bar: /* empty */
|
|
@{ previous_expr = $0; @}
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
As long as @code{bar} is used only in the fashion shown here, @code{$0}
|
|
always refers to the @code{expr} which precedes @code{bar} in the
|
|
definition of @code{foo}.
|
|
|
|
@node Action Types
|
|
@subsection Data Types of Values in Actions
|
|
@cindex action data types
|
|
@cindex data types in actions
|
|
|
|
If you have chosen a single data type for semantic values, the @code{$$}
|
|
and @code{$@var{n}} constructs always have that data type.
|
|
|
|
If you have used @code{%union} to specify a variety of data types, then you
|
|
must declare a choice among these types for each terminal or nonterminal
|
|
symbol that can have a semantic value. Then each time you use @code{$$} or
|
|
@code{$@var{n}}, its data type is determined by which symbol it refers to
|
|
in the rule. In this example,
|
|
|
|
@example
|
|
@group
|
|
exp: @dots{}
|
|
| exp '+' exp
|
|
@{ $$ = $1 + $3; @}
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
|
|
have the data type declared for the nonterminal symbol @code{exp}. If
|
|
@code{$2} were used, it would have the data type declared for the
|
|
terminal symbol @code{'+'}, whatever that might be.
|
|
|
|
Alternatively, you can specify the data type when you refer to the value,
|
|
by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
|
|
reference. For example, if you have defined types as shown here:
|
|
|
|
@example
|
|
@group
|
|
%union @{
|
|
int itype;
|
|
double dtype;
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
then you can write @code{$<itype>1} to refer to the first subunit of the
|
|
rule as an integer, or @code{$<dtype>1} to refer to it as a double.
|
|
|
|
@node Mid-Rule Actions
|
|
@subsection Actions in Mid-Rule
|
|
@cindex actions in mid-rule
|
|
@cindex mid-rule actions
|
|
|
|
Occasionally it is useful to put an action in the middle of a rule.
|
|
These actions are written just like usual end-of-rule actions, but they
|
|
are executed before the parser even recognizes the following components.
|
|
|
|
A mid-rule action may refer to the components preceding it using
|
|
@code{$@var{n}}, but it may not refer to subsequent components because
|
|
it is run before they are parsed.
|
|
|
|
The mid-rule action itself counts as one of the components of the rule.
|
|
This makes a difference when there is another action later in the same rule
|
|
(and usually there is another at the end): you have to count the actions
|
|
along with the symbols when working out which number @var{n} to use in
|
|
@code{$@var{n}}.
|
|
|
|
The mid-rule action can also have a semantic value. The action can set
|
|
its value with an assignment to @code{$$}, and actions later in the rule
|
|
can refer to the value using @code{$@var{n}}. Since there is no symbol
|
|
to name the action, there is no way to declare a data type for the value
|
|
in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
|
|
specify a data type each time you refer to this value.
|
|
|
|
There is no way to set the value of the entire rule with a mid-rule
|
|
action, because assignments to @code{$$} do not have that effect. The
|
|
only way to set the value for the entire rule is with an ordinary action
|
|
at the end of the rule.
|
|
|
|
Here is an example from a hypothetical compiler, handling a @code{let}
|
|
statement that looks like @samp{let (@var{variable}) @var{statement}} and
|
|
serves to create a variable named @var{variable} temporarily for the
|
|
duration of @var{statement}. To parse this construct, we must put
|
|
@var{variable} into the symbol table while @var{statement} is parsed, then
|
|
remove it afterward. Here is how it is done:
|
|
|
|
@example
|
|
@group
|
|
stmt: LET '(' var ')'
|
|
@{ $<context>$ = push_context ();
|
|
declare_variable ($3); @}
|
|
stmt @{ $$ = $6;
|
|
pop_context ($<context>5); @}
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
As soon as @samp{let (@var{variable})} has been recognized, the first
|
|
action is run. It saves a copy of the current semantic context (the
|
|
list of accessible variables) as its semantic value, using alternative
|
|
@code{context} in the data-type union. Then it calls
|
|
@code{declare_variable} to add the new variable to that list. Once the
|
|
first action is finished, the embedded statement @code{stmt} can be
|
|
parsed. Note that the mid-rule action is component number 5, so the
|
|
@samp{stmt} is component number 6.
|
|
|
|
After the embedded statement is parsed, its semantic value becomes the
|
|
value of the entire @code{let}-statement. Then the semantic value from the
|
|
earlier action is used to restore the prior list of variables. This
|
|
removes the temporary @code{let}-variable from the list so that it won't
|
|
appear to exist while the rest of the program is parsed.
|
|
|
|
Taking action before a rule is completely recognized often leads to
|
|
conflicts since the parser must commit to a parse in order to execute the
|
|
action. For example, the following two rules, without mid-rule actions,
|
|
can coexist in a working parser because the parser can shift the open-brace
|
|
token and look at what follows before deciding whether there is a
|
|
declaration or not:
|
|
|
|
@example
|
|
@group
|
|
compound: '@{' declarations statements '@}'
|
|
| '@{' statements '@}'
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
But when we add a mid-rule action as follows, the rules become nonfunctional:
|
|
|
|
@example
|
|
@group
|
|
compound: @{ prepare_for_local_variables (); @}
|
|
'@{' declarations statements '@}'
|
|
@end group
|
|
@group
|
|
| '@{' statements '@}'
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
Now the parser is forced to decide whether to run the mid-rule action
|
|
when it has read no farther than the open-brace. In other words, it
|
|
must commit to using one rule or the other, without sufficient
|
|
information to do it correctly. (The open-brace token is what is called
|
|
the @dfn{look-ahead} token at this time, since the parser is still
|
|
deciding what to do about it. @xref{Look-Ahead, ,Look-Ahead Tokens}.)
|
|
|
|
You might think that you could correct the problem by putting identical
|
|
actions into the two rules, like this:
|
|
|
|
@example
|
|
@group
|
|
compound: @{ prepare_for_local_variables (); @}
|
|
'@{' declarations statements '@}'
|
|
| @{ prepare_for_local_variables (); @}
|
|
'@{' statements '@}'
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
But this does not help, because Bison does not realize that the two actions
|
|
are identical. (Bison never tries to understand the C code in an action.)
|
|
|
|
If the grammar is such that a declaration can be distinguished from a
|
|
statement by the first token (which is true in C), then one solution which
|
|
does work is to put the action after the open-brace, like this:
|
|
|
|
@example
|
|
@group
|
|
compound: '@{' @{ prepare_for_local_variables (); @}
|
|
declarations statements '@}'
|
|
| '@{' statements '@}'
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
Now the first token of the following declaration or statement,
|
|
which would in any case tell Bison which rule to use, can still do so.
|
|
|
|
Another solution is to bury the action inside a nonterminal symbol which
|
|
serves as a subroutine:
|
|
|
|
@example
|
|
@group
|
|
subroutine: /* empty */
|
|
@{ prepare_for_local_variables (); @}
|
|
;
|
|
|
|
@end group
|
|
|
|
@group
|
|
compound: subroutine
|
|
'@{' declarations statements '@}'
|
|
| subroutine
|
|
'@{' statements '@}'
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
Now Bison can execute the action in the rule for @code{subroutine} without
|
|
deciding which rule for @code{compound} it will eventually use. Note that
|
|
the action is now at the end of its rule. Any mid-rule action can be
|
|
converted to an end-of-rule action in this way, and this is what Bison
|
|
actually does to implement mid-rule actions.
|
|
|
|
@node Locations
|
|
@section Tracking Locations
|
|
@cindex location
|
|
@cindex textual location
|
|
@cindex location, textual
|
|
|
|
Though grammar rules and semantic actions are enough to write a fully
|
|
functional parser, it can be useful to process some additional information,
|
|
especially symbol locations.
|
|
|
|
The way locations are handled is defined by providing a data type, and
|
|
actions to take when rules are matched.
|
|
|
|
@menu
|
|
* Location Type:: Specifying a data type for locations.
|
|
* Actions and Locations:: Using locations in actions.
|
|
* Location Default Action:: Defining a general way to compute locations.
|
|
@end menu
|
|
|
|
@node Location Type
|
|
@subsection Data Type of Locations
|
|
@cindex data type of locations
|
|
@cindex default location type
|
|
|
|
Defining a data type for locations is much simpler than for semantic values,
|
|
since all tokens and groupings always use the same type.
|
|
|
|
The type of locations is specified by defining a macro called @code{YYLTYPE}.
|
|
When @code{YYLTYPE} is not defined, Bison uses a default structure type with
|
|
four members:
|
|
|
|
@example
|
|
typedef struct YYLTYPE
|
|
@{
|
|
int first_line;
|
|
int first_column;
|
|
int last_line;
|
|
int last_column;
|
|
@} YYLTYPE;
|
|
@end example
|
|
|
|
@node Actions and Locations
|
|
@subsection Actions and Locations
|
|
@cindex location actions
|
|
@cindex actions, location
|
|
@vindex @@$
|
|
@vindex @@@var{n}
|
|
|
|
Actions are not only useful for defining language semantics, but also for
|
|
describing the behavior of the output parser with locations.
|
|
|
|
The most obvious way for building locations of syntactic groupings is very
|
|
similar to the way semantic values are computed. In a given rule, several
|
|
constructs can be used to access the locations of the elements being matched.
|
|
The location of the @var{n}th component of the right hand side is
|
|
@code{@@@var{n}}, while the location of the left hand side grouping is
|
|
@code{@@$}.
|
|
|
|
Here is a basic example using the default data type for locations:
|
|
|
|
@example
|
|
@group
|
|
exp: @dots{}
|
|
| exp '/' exp
|
|
@{
|
|
@@$.first_column = @@1.first_column;
|
|
@@$.first_line = @@1.first_line;
|
|
@@$.last_column = @@3.last_column;
|
|
@@$.last_line = @@3.last_line;
|
|
if ($3)
|
|
$$ = $1 / $3;
|
|
else
|
|
@{
|
|
$$ = 1;
|
|
fprintf (stderr,
|
|
"Division by zero, l%d,c%d-l%d,c%d",
|
|
@@3.first_line, @@3.first_column,
|
|
@@3.last_line, @@3.last_column);
|
|
@}
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
As for semantic values, there is a default action for locations that is
|
|
run each time a rule is matched. It sets the beginning of @code{@@$} to the
|
|
beginning of the first symbol, and the end of @code{@@$} to the end of the
|
|
last symbol.
|
|
|
|
With this default action, the location tracking can be fully automatic. The
|
|
example above simply rewrites this way:
|
|
|
|
@example
|
|
@group
|
|
exp: @dots{}
|
|
| exp '/' exp
|
|
@{
|
|
if ($3)
|
|
$$ = $1 / $3;
|
|
else
|
|
@{
|
|
$$ = 1;
|
|
fprintf (stderr,
|
|
"Division by zero, l%d,c%d-l%d,c%d",
|
|
@@3.first_line, @@3.first_column,
|
|
@@3.last_line, @@3.last_column);
|
|
@}
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
@node Location Default Action
|
|
@subsection Default Action for Locations
|
|
@vindex YYLLOC_DEFAULT
|
|
|
|
Actually, actions are not the best place to compute locations. Since
|
|
locations are much more general than semantic values, there is room in
|
|
the output parser to redefine the default action to take for each
|
|
rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
|
|
matched, before the associated action is run. It is also invoked
|
|
while processing a syntax error, to compute the error's location.
|
|
|
|
Most of the time, this macro is general enough to suppress location
|
|
dedicated code from semantic actions.
|
|
|
|
The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
|
|
the location of the grouping (the result of the computation). When a
|
|
rule is matched, the second parameter is an array holding locations of
|
|
all right hand side elements of the rule being matched, and the third
|
|
parameter is the size of the rule's right hand side. When processing
|
|
a syntax error, the second parameter is an array holding locations of
|
|
the symbols that were discarded during error processing, and the third
|
|
parameter is the number of discarded symbols.
|
|
|
|
By default, @code{YYLLOC_DEFAULT} is defined this way for simple
|
|
@acronym{LALR}(1) parsers:
|
|
|
|
@example
|
|
@group
|
|
# define YYLLOC_DEFAULT(Current, Rhs, N) \
|
|
((Current).first_line = (Rhs)[1].first_line, \
|
|
(Current).first_column = (Rhs)[1].first_column, \
|
|
(Current).last_line = (Rhs)[N].last_line, \
|
|
(Current).last_column = (Rhs)[N].last_column)
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
and like this for @acronym{GLR} parsers:
|
|
|
|
@example
|
|
@group
|
|
# define YYLLOC_DEFAULT(yyCurrent, yyRhs, YYN) \
|
|
((yyCurrent).first_line = YYRHSLOC(yyRhs, 1).first_line, \
|
|
(yyCurrent).first_column = YYRHSLOC(yyRhs, 1).first_column, \
|
|
(yyCurrent).last_line = YYRHSLOC(yyRhs, YYN).last_line, \
|
|
(yyCurrent).last_column = YYRHSLOC(yyRhs, YYN).last_column)
|
|
@end group
|
|
@end example
|
|
|
|
When defining @code{YYLLOC_DEFAULT}, you should consider that:
|
|
|
|
@itemize @bullet
|
|
@item
|
|
All arguments are free of side-effects. However, only the first one (the
|
|
result) should be modified by @code{YYLLOC_DEFAULT}.
|
|
|
|
@item
|
|
For consistency with semantic actions, valid indexes for the location
|
|
array range from 1 to @var{n}.
|
|
|
|
@item
|
|
Your macro should parenthesize its arguments, if need be, since the
|
|
actual arguments may not be surrounded by parentheses. Also, your
|
|
macro should expand to something that can be used as a single
|
|
statement when it is followed by a semicolon.
|
|
@end itemize
|
|
|
|
@node Declarations
|
|
@section Bison Declarations
|
|
@cindex declarations, Bison
|
|
@cindex Bison declarations
|
|
|
|
The @dfn{Bison declarations} section of a Bison grammar defines the symbols
|
|
used in formulating the grammar and the data types of semantic values.
|
|
@xref{Symbols}.
|
|
|
|
All token type names (but not single-character literal tokens such as
|
|
@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
|
|
declared if you need to specify which data type to use for the semantic
|
|
value (@pxref{Multiple Types, ,More Than One Value Type}).
|
|
|
|
The first rule in the file also specifies the start symbol, by default.
|
|
If you want some other symbol to be the start symbol, you must declare
|
|
it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
|
|
Grammars}).
|
|
|
|
@menu
|
|
* Token Decl:: Declaring terminal symbols.
|
|
* Precedence Decl:: Declaring terminals with precedence and associativity.
|
|
* Union Decl:: Declaring the set of all semantic value types.
|
|
* Type Decl:: Declaring the choice of type for a nonterminal symbol.
|
|
* Destructor Decl:: Declaring how symbols are freed.
|
|
* Expect Decl:: Suppressing warnings about parsing conflicts.
|
|
* Start Decl:: Specifying the start symbol.
|
|
* Pure Decl:: Requesting a reentrant parser.
|
|
* Decl Summary:: Table of all Bison declarations.
|
|
@end menu
|
|
|
|
@node Token Decl
|
|
@subsection Token Type Names
|
|
@cindex declaring token type names
|
|
@cindex token type names, declaring
|
|
@cindex declaring literal string tokens
|
|
@findex %token
|
|
|
|
The basic way to declare a token type name (terminal symbol) is as follows:
|
|
|
|
@example
|
|
%token @var{name}
|
|
@end example
|
|
|
|
Bison will convert this into a @code{#define} directive in
|
|
the parser, so that the function @code{yylex} (if it is in this file)
|
|
can use the name @var{name} to stand for this token type's code.
|
|
|
|
Alternatively, you can use @code{%left}, @code{%right}, or
|
|
@code{%nonassoc} instead of @code{%token}, if you wish to specify
|
|
associativity and precedence. @xref{Precedence Decl, ,Operator
|
|
Precedence}.
|
|
|
|
You can explicitly specify the numeric code for a token type by appending
|
|
a decimal or hexadecimal integer value in the field immediately
|
|
following the token name:
|
|
|
|
@example
|
|
%token NUM 300
|
|
%token XNUM 0x12d // a GNU extension
|
|
@end example
|
|
|
|
@noindent
|
|
It is generally best, however, to let Bison choose the numeric codes for
|
|
all token types. Bison will automatically select codes that don't conflict
|
|
with each other or with normal characters.
|
|
|
|
In the event that the stack type is a union, you must augment the
|
|
@code{%token} or other token declaration to include the data type
|
|
alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
|
|
Than One Value Type}).
|
|
|
|
For example:
|
|
|
|
@example
|
|
@group
|
|
%union @{ /* define stack type */
|
|
double val;
|
|
symrec *tptr;
|
|
@}
|
|
%token <val> NUM /* define token NUM and its type */
|
|
@end group
|
|
@end example
|
|
|
|
You can associate a literal string token with a token type name by
|
|
writing the literal string at the end of a @code{%token}
|
|
declaration which declares the name. For example:
|
|
|
|
@example
|
|
%token arrow "=>"
|
|
@end example
|
|
|
|
@noindent
|
|
For example, a grammar for the C language might specify these names with
|
|
equivalent literal string tokens:
|
|
|
|
@example
|
|
%token <operator> OR "||"
|
|
%token <operator> LE 134 "<="
|
|
%left OR "<="
|
|
@end example
|
|
|
|
@noindent
|
|
Once you equate the literal string and the token name, you can use them
|
|
interchangeably in further declarations or the grammar rules. The
|
|
@code{yylex} function can use the token name or the literal string to
|
|
obtain the token type code number (@pxref{Calling Convention}).
|
|
|
|
@node Precedence Decl
|
|
@subsection Operator Precedence
|
|
@cindex precedence declarations
|
|
@cindex declaring operator precedence
|
|
@cindex operator precedence, declaring
|
|
|
|
Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
|
|
declare a token and specify its precedence and associativity, all at
|
|
once. These are called @dfn{precedence declarations}.
|
|
@xref{Precedence, ,Operator Precedence}, for general information on
|
|
operator precedence.
|
|
|
|
The syntax of a precedence declaration is the same as that of
|
|
@code{%token}: either
|
|
|
|
@example
|
|
%left @var{symbols}@dots{}
|
|
@end example
|
|
|
|
@noindent
|
|
or
|
|
|
|
@example
|
|
%left <@var{type}> @var{symbols}@dots{}
|
|
@end example
|
|
|
|
And indeed any of these declarations serves the purposes of @code{%token}.
|
|
But in addition, they specify the associativity and relative precedence for
|
|
all the @var{symbols}:
|
|
|
|
@itemize @bullet
|
|
@item
|
|
The associativity of an operator @var{op} determines how repeated uses
|
|
of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
|
|
@var{z}} is parsed by grouping @var{x} with @var{y} first or by
|
|
grouping @var{y} with @var{z} first. @code{%left} specifies
|
|
left-associativity (grouping @var{x} with @var{y} first) and
|
|
@code{%right} specifies right-associativity (grouping @var{y} with
|
|
@var{z} first). @code{%nonassoc} specifies no associativity, which
|
|
means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
|
|
considered a syntax error.
|
|
|
|
@item
|
|
The precedence of an operator determines how it nests with other operators.
|
|
All the tokens declared in a single precedence declaration have equal
|
|
precedence and nest together according to their associativity.
|
|
When two tokens declared in different precedence declarations associate,
|
|
the one declared later has the higher precedence and is grouped first.
|
|
@end itemize
|
|
|
|
@node Union Decl
|
|
@subsection The Collection of Value Types
|
|
@cindex declaring value types
|
|
@cindex value types, declaring
|
|
@findex %union
|
|
|
|
The @code{%union} declaration specifies the entire collection of possible
|
|
data types for semantic values. The keyword @code{%union} is followed by a
|
|
pair of braces containing the same thing that goes inside a @code{union} in
|
|
C.
|
|
|
|
For example:
|
|
|
|
@example
|
|
@group
|
|
%union @{
|
|
double val;
|
|
symrec *tptr;
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
This says that the two alternative types are @code{double} and @code{symrec
|
|
*}. They are given names @code{val} and @code{tptr}; these names are used
|
|
in the @code{%token} and @code{%type} declarations to pick one of the types
|
|
for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
|
|
|
|
As an extension to @acronym{POSIX}, a tag is allowed after the
|
|
@code{union}. For example:
|
|
|
|
@example
|
|
@group
|
|
%union value @{
|
|
double val;
|
|
symrec *tptr;
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
specifies the union tag @code{value}, so the corresponding C type is
|
|
@code{union value}. If you do not specify a tag, it defaults to
|
|
@code{YYSTYPE}.
|
|
|
|
Note that, unlike making a @code{union} declaration in C, you need not write
|
|
a semicolon after the closing brace.
|
|
|
|
@node Type Decl
|
|
@subsection Nonterminal Symbols
|
|
@cindex declaring value types, nonterminals
|
|
@cindex value types, nonterminals, declaring
|
|
@findex %type
|
|
|
|
@noindent
|
|
When you use @code{%union} to specify multiple value types, you must
|
|
declare the value type of each nonterminal symbol for which values are
|
|
used. This is done with a @code{%type} declaration, like this:
|
|
|
|
@example
|
|
%type <@var{type}> @var{nonterminal}@dots{}
|
|
@end example
|
|
|
|
@noindent
|
|
Here @var{nonterminal} is the name of a nonterminal symbol, and
|
|
@var{type} is the name given in the @code{%union} to the alternative
|
|
that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
|
|
can give any number of nonterminal symbols in the same @code{%type}
|
|
declaration, if they have the same value type. Use spaces to separate
|
|
the symbol names.
|
|
|
|
You can also declare the value type of a terminal symbol. To do this,
|
|
use the same @code{<@var{type}>} construction in a declaration for the
|
|
terminal symbol. All kinds of token declarations allow
|
|
@code{<@var{type}>}.
|
|
|
|
@node Destructor Decl
|
|
@subsection Freeing Discarded Symbols
|
|
@cindex freeing discarded symbols
|
|
@findex %destructor
|
|
|
|
Some symbols can be discarded by the parser, typically during error
|
|
recovery (@pxref{Error Recovery}). Basically, during error recovery,
|
|
embarrassing symbols already pushed on the stack, and embarrassing
|
|
tokens coming from the rest of the file are thrown away until the parser
|
|
falls on its feet. If these symbols convey heap based information, this
|
|
memory is lost. While this behavior is tolerable for batch parsers,
|
|
such as in compilers, it is unacceptable for parsers that can
|
|
possibility ``never end'' such as shells, or implementations of
|
|
communication protocols.
|
|
|
|
The @code{%destructor} directive allows for the definition of code that
|
|
is called when a symbol is thrown away.
|
|
|
|
@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
|
|
@findex %destructor
|
|
Declare that the @var{code} must be invoked for each of the
|
|
@var{symbols} that will be discarded by the parser. The @var{code}
|
|
should use @code{$$} to designate the semantic value associated to the
|
|
@var{symbols}. The additional parser parameters are also available
|
|
(@pxref{Parser Function, , The Parser Function @code{yyparse}}).
|
|
|
|
@strong{Warning:} as of Bison 1.875, this feature is still considered as
|
|
experimental, as there was not enough user feedback. In particular,
|
|
the syntax might still change.
|
|
@end deffn
|
|
|
|
For instance:
|
|
|
|
@smallexample
|
|
%union
|
|
@{
|
|
char *string;
|
|
@}
|
|
%token <string> STRING
|
|
%type <string> string
|
|
%destructor @{ free ($$); @} STRING string
|
|
@end smallexample
|
|
|
|
@noindent
|
|
guarantees that when a @code{STRING} or a @code{string} will be discarded,
|
|
its associated memory will be freed.
|
|
|
|
Note that in the future, Bison might also consider that right hand side
|
|
members that are not mentioned in the action can be destroyed. For
|
|
instance, in:
|
|
|
|
@smallexample
|
|
comment: "/*" STRING "*/";
|
|
@end smallexample
|
|
|
|
@noindent
|
|
the parser is entitled to destroy the semantic value of the
|
|
@code{string}. Of course, this will not apply to the default action;
|
|
compare:
|
|
|
|
@smallexample
|
|
typeless: string; // $$ = $1 does not apply; $1 is destroyed.
|
|
typefull: string; // $$ = $1 applies, $1 is not destroyed.
|
|
@end smallexample
|
|
|
|
@node Expect Decl
|
|
@subsection Suppressing Conflict Warnings
|
|
@cindex suppressing conflict warnings
|
|
@cindex preventing warnings about conflicts
|
|
@cindex warnings, preventing
|
|
@cindex conflicts, suppressing warnings of
|
|
@findex %expect
|
|
@findex %expect-rr
|
|
|
|
Bison normally warns if there are any conflicts in the grammar
|
|
(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
|
|
have harmless shift/reduce conflicts which are resolved in a predictable
|
|
way and would be difficult to eliminate. It is desirable to suppress
|
|
the warning about these conflicts unless the number of conflicts
|
|
changes. You can do this with the @code{%expect} declaration.
|
|
|
|
The declaration looks like this:
|
|
|
|
@example
|
|
%expect @var{n}
|
|
@end example
|
|
|
|
Here @var{n} is a decimal integer. The declaration says there should be
|
|
no warning if there are @var{n} shift/reduce conflicts and no
|
|
reduce/reduce conflicts. The usual warning is
|
|
given if there are either more or fewer conflicts, or if there are any
|
|
reduce/reduce conflicts.
|
|
|
|
For normal LALR(1) parsers, reduce/reduce conflicts are more serious,
|
|
and should be eliminated entirely. Bison will always report
|
|
reduce/reduce conflicts for these parsers. With GLR parsers, however,
|
|
both shift/reduce and reduce/reduce are routine (otherwise, there
|
|
would be no need to use GLR parsing). Therefore, it is also possible
|
|
to specify an expected number of reduce/reduce conflicts in GLR
|
|
parsers, using the declaration:
|
|
|
|
@example
|
|
%expect-rr @var{n}
|
|
@end example
|
|
|
|
In general, using @code{%expect} involves these steps:
|
|
|
|
@itemize @bullet
|
|
@item
|
|
Compile your grammar without @code{%expect}. Use the @samp{-v} option
|
|
to get a verbose list of where the conflicts occur. Bison will also
|
|
print the number of conflicts.
|
|
|
|
@item
|
|
Check each of the conflicts to make sure that Bison's default
|
|
resolution is what you really want. If not, rewrite the grammar and
|
|
go back to the beginning.
|
|
|
|
@item
|
|
Add an @code{%expect} declaration, copying the number @var{n} from the
|
|
number which Bison printed.
|
|
@end itemize
|
|
|
|
Now Bison will stop annoying you if you do not change the number of
|
|
conflicts, but it will warn you again if changes in the grammar result
|
|
in more or fewer conflicts.
|
|
|
|
@node Start Decl
|
|
@subsection The Start-Symbol
|
|
@cindex declaring the start symbol
|
|
@cindex start symbol, declaring
|
|
@cindex default start symbol
|
|
@findex %start
|
|
|
|
Bison assumes by default that the start symbol for the grammar is the first
|
|
nonterminal specified in the grammar specification section. The programmer
|
|
may override this restriction with the @code{%start} declaration as follows:
|
|
|
|
@example
|
|
%start @var{symbol}
|
|
@end example
|
|
|
|
@node Pure Decl
|
|
@subsection A Pure (Reentrant) Parser
|
|
@cindex reentrant parser
|
|
@cindex pure parser
|
|
@findex %pure-parser
|
|
|
|
A @dfn{reentrant} program is one which does not alter in the course of
|
|
execution; in other words, it consists entirely of @dfn{pure} (read-only)
|
|
code. Reentrancy is important whenever asynchronous execution is possible;
|
|
for example, a non-reentrant program may not be safe to call from a signal
|
|
handler. In systems with multiple threads of control, a non-reentrant
|
|
program must be called only within interlocks.
|
|
|
|
Normally, Bison generates a parser which is not reentrant. This is
|
|
suitable for most uses, and it permits compatibility with Yacc. (The
|
|
standard Yacc interfaces are inherently nonreentrant, because they use
|
|
statically allocated variables for communication with @code{yylex},
|
|
including @code{yylval} and @code{yylloc}.)
|
|
|
|
Alternatively, you can generate a pure, reentrant parser. The Bison
|
|
declaration @code{%pure-parser} says that you want the parser to be
|
|
reentrant. It looks like this:
|
|
|
|
@example
|
|
%pure-parser
|
|
@end example
|
|
|
|
The result is that the communication variables @code{yylval} and
|
|
@code{yylloc} become local variables in @code{yyparse}, and a different
|
|
calling convention is used for the lexical analyzer function
|
|
@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
|
|
Parsers}, for the details of this. The variable @code{yynerrs} also
|
|
becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error
|
|
Reporting Function @code{yyerror}}). The convention for calling
|
|
@code{yyparse} itself is unchanged.
|
|
|
|
Whether the parser is pure has nothing to do with the grammar rules.
|
|
You can generate either a pure parser or a nonreentrant parser from any
|
|
valid grammar.
|
|
|
|
@node Decl Summary
|
|
@subsection Bison Declaration Summary
|
|
@cindex Bison declaration summary
|
|
@cindex declaration summary
|
|
@cindex summary, Bison declaration
|
|
|
|
Here is a summary of the declarations used to define a grammar:
|
|
|
|
@deffn {Directive} %union
|
|
Declare the collection of data types that semantic values may have
|
|
(@pxref{Union Decl, ,The Collection of Value Types}).
|
|
@end deffn
|
|
|
|
@deffn {Directive} %token
|
|
Declare a terminal symbol (token type name) with no precedence
|
|
or associativity specified (@pxref{Token Decl, ,Token Type Names}).
|
|
@end deffn
|
|
|
|
@deffn {Directive} %right
|
|
Declare a terminal symbol (token type name) that is right-associative
|
|
(@pxref{Precedence Decl, ,Operator Precedence}).
|
|
@end deffn
|
|
|
|
@deffn {Directive} %left
|
|
Declare a terminal symbol (token type name) that is left-associative
|
|
(@pxref{Precedence Decl, ,Operator Precedence}).
|
|
@end deffn
|
|
|
|
@deffn {Directive} %nonassoc
|
|
Declare a terminal symbol (token type name) that is nonassociative
|
|
(@pxref{Precedence Decl, ,Operator Precedence}).
|
|
Using it in a way that would be associative is a syntax error.
|
|
@end deffn
|
|
|
|
@ifset defaultprec
|
|
@deffn {Directive} %default-prec
|
|
Assign a precedence to rules lacking an explicit @code{%prec} modifier
|
|
(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
|
|
@end deffn
|
|
@end ifset
|
|
|
|
@deffn {Directive} %type
|
|
Declare the type of semantic values for a nonterminal symbol
|
|
(@pxref{Type Decl, ,Nonterminal Symbols}).
|
|
@end deffn
|
|
|
|
@deffn {Directive} %start
|
|
Specify the grammar's start symbol (@pxref{Start Decl, ,The
|
|
Start-Symbol}).
|
|
@end deffn
|
|
|
|
@deffn {Directive} %expect
|
|
Declare the expected number of shift-reduce conflicts
|
|
(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
|
|
@end deffn
|
|
|
|
|
|
@sp 1
|
|
@noindent
|
|
In order to change the behavior of @command{bison}, use the following
|
|
directives:
|
|
|
|
@deffn {Directive} %debug
|
|
In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
|
|
already defined, so that the debugging facilities are compiled.
|
|
@end deffn
|
|
@xref{Tracing, ,Tracing Your Parser}.
|
|
|
|
@deffn {Directive} %defines
|
|
Write a header file containing macro definitions for the token type
|
|
names defined in the grammar as well as a few other declarations.
|
|
If the parser output file is named @file{@var{name}.c} then this file
|
|
is named @file{@var{name}.h}.
|
|
|
|
Unless @code{YYSTYPE} is already defined as a macro, the output header
|
|
declares @code{YYSTYPE}. Therefore, if you are using a @code{%union}
|
|
(@pxref{Multiple Types, ,More Than One Value Type}) with components
|
|
that require other definitions, or if you have defined a
|
|
@code{YYSTYPE} macro (@pxref{Value Type, ,Data Types of Semantic
|
|
Values}), you need to arrange for these definitions to be propagated to
|
|
all modules, e.g., by putting them in a
|
|
prerequisite header that is included both by your parser and by any
|
|
other module that needs @code{YYSTYPE}.
|
|
|
|
Unless your parser is pure, the output header declares @code{yylval}
|
|
as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
|
|
Parser}.
|
|
|
|
If you have also used locations, the output header declares
|
|
@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
|
|
@code{YYSTYPE} and @code{yylval}. @xref{Locations, ,Tracking
|
|
Locations}.
|
|
|
|
This output file is normally essential if you wish to put the
|
|
definition of @code{yylex} in a separate source file, because
|
|
@code{yylex} typically needs to be able to refer to the
|
|
above-mentioned declarations and to the token type codes.
|
|
@xref{Token Values, ,Semantic Values of Tokens}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %destructor
|
|
Specifying how the parser should reclaim the memory associated to
|
|
discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %file-prefix="@var{prefix}"
|
|
Specify a prefix to use for all Bison output file names. The names are
|
|
chosen as if the input file were named @file{@var{prefix}.y}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %locations
|
|
Generate the code processing the locations (@pxref{Action Features,
|
|
,Special Features for Use in Actions}). This mode is enabled as soon as
|
|
the grammar uses the special @samp{@@@var{n}} tokens, but if your
|
|
grammar does not use it, using @samp{%locations} allows for more
|
|
accurate syntax error messages.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %name-prefix="@var{prefix}"
|
|
Rename the external symbols used in the parser so that they start with
|
|
@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
|
|
is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
|
|
@code{yylval}, @code{yylloc}, @code{yychar}, @code{yydebug}, and
|
|
possible @code{yylloc}. For example, if you use
|
|
@samp{%name-prefix="c_"}, the names become @code{c_parse}, @code{c_lex},
|
|
and so on. @xref{Multiple Parsers, ,Multiple Parsers in the Same
|
|
Program}.
|
|
@end deffn
|
|
|
|
@ifset defaultprec
|
|
@deffn {Directive} %no-default-prec
|
|
Do not assign a precedence to rules lacking an explicit @code{%prec}
|
|
modifier (@pxref{Contextual Precedence, ,Context-Dependent
|
|
Precedence}).
|
|
@end deffn
|
|
@end ifset
|
|
|
|
@deffn {Directive} %no-parser
|
|
Do not include any C code in the parser file; generate tables only. The
|
|
parser file contains just @code{#define} directives and static variable
|
|
declarations.
|
|
|
|
This option also tells Bison to write the C code for the grammar actions
|
|
into a file named @file{@var{filename}.act}, in the form of a
|
|
brace-surrounded body fit for a @code{switch} statement.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %no-lines
|
|
Don't generate any @code{#line} preprocessor commands in the parser
|
|
file. Ordinarily Bison writes these commands in the parser file so that
|
|
the C compiler and debuggers will associate errors and object code with
|
|
your source file (the grammar file). This directive causes them to
|
|
associate errors with the parser file, treating it an independent source
|
|
file in its own right.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %output="@var{filename}"
|
|
Specify the @var{filename} for the parser file.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %pure-parser
|
|
Request a pure (reentrant) parser program (@pxref{Pure Decl, ,A Pure
|
|
(Reentrant) Parser}).
|
|
@end deffn
|
|
|
|
@deffn {Directive} %token-table
|
|
Generate an array of token names in the parser file. The name of the
|
|
array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
|
|
token whose internal Bison token code number is @var{i}. The first
|
|
three elements of @code{yytname} correspond to the predefined tokens
|
|
@code{"$end"},
|
|
@code{"error"}, and @code{"$undefined"}; after these come the symbols
|
|
defined in the grammar file.
|
|
|
|
For single-character literal tokens and literal string tokens, the name
|
|
in the table includes the single-quote or double-quote characters: for
|
|
example, @code{"'+'"} is a single-character literal and @code{"\"<=\""}
|
|
is a literal string token. All the characters of the literal string
|
|
token appear verbatim in the string found in the table; even
|
|
double-quote characters are not escaped. For example, if the token
|
|
consists of three characters @samp{*"*}, its string in @code{yytname}
|
|
contains @samp{"*"*"}. (In C, that would be written as
|
|
@code{"\"*\"*\""}).
|
|
|
|
When you specify @code{%token-table}, Bison also generates macro
|
|
definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
|
|
@code{YYNRULES}, and @code{YYNSTATES}:
|
|
|
|
@table @code
|
|
@item YYNTOKENS
|
|
The highest token number, plus one.
|
|
@item YYNNTS
|
|
The number of nonterminal symbols.
|
|
@item YYNRULES
|
|
The number of grammar rules,
|
|
@item YYNSTATES
|
|
The number of parser states (@pxref{Parser States}).
|
|
@end table
|
|
@end deffn
|
|
|
|
@deffn {Directive} %verbose
|
|
Write an extra output file containing verbose descriptions of the
|
|
parser states and what is done for each type of look-ahead token in
|
|
that state. @xref{Understanding, , Understanding Your Parser}, for more
|
|
information.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %yacc
|
|
Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
|
|
including its naming conventions. @xref{Bison Options}, for more.
|
|
@end deffn
|
|
|
|
|
|
@node Multiple Parsers
|
|
@section Multiple Parsers in the Same Program
|
|
|
|
Most programs that use Bison parse only one language and therefore contain
|
|
only one Bison parser. But what if you want to parse more than one
|
|
language with the same program? Then you need to avoid a name conflict
|
|
between different definitions of @code{yyparse}, @code{yylval}, and so on.
|
|
|
|
The easy way to do this is to use the option @samp{-p @var{prefix}}
|
|
(@pxref{Invocation, ,Invoking Bison}). This renames the interface
|
|
functions and variables of the Bison parser to start with @var{prefix}
|
|
instead of @samp{yy}. You can use this to give each parser distinct
|
|
names that do not conflict.
|
|
|
|
The precise list of symbols renamed is @code{yyparse}, @code{yylex},
|
|
@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
|
|
@code{yychar} and @code{yydebug}. For example, if you use @samp{-p c},
|
|
the names become @code{cparse}, @code{clex}, and so on.
|
|
|
|
@strong{All the other variables and macros associated with Bison are not
|
|
renamed.} These others are not global; there is no conflict if the same
|
|
name is used in different parsers. For example, @code{YYSTYPE} is not
|
|
renamed, but defining this in different ways in different parsers causes
|
|
no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
|
|
|
|
The @samp{-p} option works by adding macro definitions to the beginning
|
|
of the parser source file, defining @code{yyparse} as
|
|
@code{@var{prefix}parse}, and so on. This effectively substitutes one
|
|
name for the other in the entire parser file.
|
|
|
|
@node Interface
|
|
@chapter Parser C-Language Interface
|
|
@cindex C-language interface
|
|
@cindex interface
|
|
|
|
The Bison parser is actually a C function named @code{yyparse}. Here we
|
|
describe the interface conventions of @code{yyparse} and the other
|
|
functions that it needs to use.
|
|
|
|
Keep in mind that the parser uses many C identifiers starting with
|
|
@samp{yy} and @samp{YY} for internal purposes. If you use such an
|
|
identifier (aside from those in this manual) in an action or in epilogue
|
|
in the grammar file, you are likely to run into trouble.
|
|
|
|
@menu
|
|
* Parser Function:: How to call @code{yyparse} and what it returns.
|
|
* Lexical:: You must supply a function @code{yylex}
|
|
which reads tokens.
|
|
* Error Reporting:: You must supply a function @code{yyerror}.
|
|
* Action Features:: Special features for use in actions.
|
|
@end menu
|
|
|
|
@node Parser Function
|
|
@section The Parser Function @code{yyparse}
|
|
@findex yyparse
|
|
|
|
You call the function @code{yyparse} to cause parsing to occur. This
|
|
function reads tokens, executes actions, and ultimately returns when it
|
|
encounters end-of-input or an unrecoverable syntax error. You can also
|
|
write an action which directs @code{yyparse} to return immediately
|
|
without reading further.
|
|
|
|
|
|
@deftypefun int yyparse (void)
|
|
The value returned by @code{yyparse} is 0 if parsing was successful (return
|
|
is due to end-of-input).
|
|
|
|
The value is 1 if parsing failed (return is due to a syntax error).
|
|
@end deftypefun
|
|
|
|
In an action, you can cause immediate return from @code{yyparse} by using
|
|
these macros:
|
|
|
|
@defmac YYACCEPT
|
|
@findex YYACCEPT
|
|
Return immediately with value 0 (to report success).
|
|
@end defmac
|
|
|
|
@defmac YYABORT
|
|
@findex YYABORT
|
|
Return immediately with value 1 (to report failure).
|
|
@end defmac
|
|
|
|
If you use a reentrant parser, you can optionally pass additional
|
|
parameter information to it in a reentrant way. To do so, use the
|
|
declaration @code{%parse-param}:
|
|
|
|
@deffn {Directive} %parse-param @{@var{argument-declaration}@}
|
|
@findex %parse-param
|
|
Declare that an argument declared by @code{argument-declaration} is an
|
|
additional @code{yyparse} argument.
|
|
The @var{argument-declaration} is used when declaring
|
|
functions or prototypes. The last identifier in
|
|
@var{argument-declaration} must be the argument name.
|
|
@end deffn
|
|
|
|
Here's an example. Write this in the parser:
|
|
|
|
@example
|
|
%parse-param @{int *nastiness@}
|
|
%parse-param @{int *randomness@}
|
|
@end example
|
|
|
|
@noindent
|
|
Then call the parser like this:
|
|
|
|
@example
|
|
@{
|
|
int nastiness, randomness;
|
|
@dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
|
|
value = yyparse (&nastiness, &randomness);
|
|
@dots{}
|
|
@}
|
|
@end example
|
|
|
|
@noindent
|
|
In the grammar actions, use expressions like this to refer to the data:
|
|
|
|
@example
|
|
exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
|
|
@end example
|
|
|
|
|
|
@node Lexical
|
|
@section The Lexical Analyzer Function @code{yylex}
|
|
@findex yylex
|
|
@cindex lexical analyzer
|
|
|
|
The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
|
|
the input stream and returns them to the parser. Bison does not create
|
|
this function automatically; you must write it so that @code{yyparse} can
|
|
call it. The function is sometimes referred to as a lexical scanner.
|
|
|
|
In simple programs, @code{yylex} is often defined at the end of the Bison
|
|
grammar file. If @code{yylex} is defined in a separate source file, you
|
|
need to arrange for the token-type macro definitions to be available there.
|
|
To do this, use the @samp{-d} option when you run Bison, so that it will
|
|
write these macro definitions into a separate header file
|
|
@file{@var{name}.tab.h} which you can include in the other source files
|
|
that need it. @xref{Invocation, ,Invoking Bison}.
|
|
|
|
@menu
|
|
* Calling Convention:: How @code{yyparse} calls @code{yylex}.
|
|
* Token Values:: How @code{yylex} must return the semantic value
|
|
of the token it has read.
|
|
* Token Locations:: How @code{yylex} must return the text location
|
|
(line number, etc.) of the token, if the
|
|
actions want that.
|
|
* Pure Calling:: How the calling convention differs
|
|
in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
|
|
@end menu
|
|
|
|
@node Calling Convention
|
|
@subsection Calling Convention for @code{yylex}
|
|
|
|
The value that @code{yylex} returns must be the positive numeric code
|
|
for the type of token it has just found; a zero or negative value
|
|
signifies end-of-input.
|
|
|
|
When a token is referred to in the grammar rules by a name, that name
|
|
in the parser file becomes a C macro whose definition is the proper
|
|
numeric code for that token type. So @code{yylex} can use the name
|
|
to indicate that type. @xref{Symbols}.
|
|
|
|
When a token is referred to in the grammar rules by a character literal,
|
|
the numeric code for that character is also the code for the token type.
|
|
So @code{yylex} can simply return that character code, possibly converted
|
|
to @code{unsigned char} to avoid sign-extension. The null character
|
|
must not be used this way, because its code is zero and that
|
|
signifies end-of-input.
|
|
|
|
Here is an example showing these things:
|
|
|
|
@example
|
|
int
|
|
yylex (void)
|
|
@{
|
|
@dots{}
|
|
if (c == EOF) /* Detect end-of-input. */
|
|
return 0;
|
|
@dots{}
|
|
if (c == '+' || c == '-')
|
|
return c; /* Assume token type for `+' is '+'. */
|
|
@dots{}
|
|
return INT; /* Return the type of the token. */
|
|
@dots{}
|
|
@}
|
|
@end example
|
|
|
|
@noindent
|
|
This interface has been designed so that the output from the @code{lex}
|
|
utility can be used without change as the definition of @code{yylex}.
|
|
|
|
If the grammar uses literal string tokens, there are two ways that
|
|
@code{yylex} can determine the token type codes for them:
|
|
|
|
@itemize @bullet
|
|
@item
|
|
If the grammar defines symbolic token names as aliases for the
|
|
literal string tokens, @code{yylex} can use these symbolic names like
|
|
all others. In this case, the use of the literal string tokens in
|
|
the grammar file has no effect on @code{yylex}.
|
|
|
|
@item
|
|
@code{yylex} can find the multicharacter token in the @code{yytname}
|
|
table. The index of the token in the table is the token type's code.
|
|
The name of a multicharacter token is recorded in @code{yytname} with a
|
|
double-quote, the token's characters, and another double-quote. The
|
|
token's characters are not escaped in any way; they appear verbatim in
|
|
the contents of the string in the table.
|
|
|
|
Here's code for looking up a token in @code{yytname}, assuming that the
|
|
characters of the token are stored in @code{token_buffer}.
|
|
|
|
@smallexample
|
|
for (i = 0; i < YYNTOKENS; i++)
|
|
@{
|
|
if (yytname[i] != 0
|
|
&& yytname[i][0] == '"'
|
|
&& ! strncmp (yytname[i] + 1, token_buffer,
|
|
strlen (token_buffer))
|
|
&& yytname[i][strlen (token_buffer) + 1] == '"'
|
|
&& yytname[i][strlen (token_buffer) + 2] == 0)
|
|
break;
|
|
@}
|
|
@end smallexample
|
|
|
|
The @code{yytname} table is generated only if you use the
|
|
@code{%token-table} declaration. @xref{Decl Summary}.
|
|
@end itemize
|
|
|
|
@node Token Values
|
|
@subsection Semantic Values of Tokens
|
|
|
|
@vindex yylval
|
|
In an ordinary (non-reentrant) parser, the semantic value of the token must
|
|
be stored into the global variable @code{yylval}. When you are using
|
|
just one data type for semantic values, @code{yylval} has that type.
|
|
Thus, if the type is @code{int} (the default), you might write this in
|
|
@code{yylex}:
|
|
|
|
@example
|
|
@group
|
|
@dots{}
|
|
yylval = value; /* Put value onto Bison stack. */
|
|
return INT; /* Return the type of the token. */
|
|
@dots{}
|
|
@end group
|
|
@end example
|
|
|
|
When you are using multiple data types, @code{yylval}'s type is a union
|
|
made from the @code{%union} declaration (@pxref{Union Decl, ,The
|
|
Collection of Value Types}). So when you store a token's value, you
|
|
must use the proper member of the union. If the @code{%union}
|
|
declaration looks like this:
|
|
|
|
@example
|
|
@group
|
|
%union @{
|
|
int intval;
|
|
double val;
|
|
symrec *tptr;
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
then the code in @code{yylex} might look like this:
|
|
|
|
@example
|
|
@group
|
|
@dots{}
|
|
yylval.intval = value; /* Put value onto Bison stack. */
|
|
return INT; /* Return the type of the token. */
|
|
@dots{}
|
|
@end group
|
|
@end example
|
|
|
|
@node Token Locations
|
|
@subsection Textual Locations of Tokens
|
|
|
|
@vindex yylloc
|
|
If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
|
|
Tracking Locations}) in actions to keep track of the
|
|
textual locations of tokens and groupings, then you must provide this
|
|
information in @code{yylex}. The function @code{yyparse} expects to
|
|
find the textual location of a token just parsed in the global variable
|
|
@code{yylloc}. So @code{yylex} must store the proper data in that
|
|
variable.
|
|
|
|
By default, the value of @code{yylloc} is a structure and you need only
|
|
initialize the members that are going to be used by the actions. The
|
|
four members are called @code{first_line}, @code{first_column},
|
|
@code{last_line} and @code{last_column}. Note that the use of this
|
|
feature makes the parser noticeably slower.
|
|
|
|
@tindex YYLTYPE
|
|
The data type of @code{yylloc} has the name @code{YYLTYPE}.
|
|
|
|
@node Pure Calling
|
|
@subsection Calling Conventions for Pure Parsers
|
|
|
|
When you use the Bison declaration @code{%pure-parser} to request a
|
|
pure, reentrant parser, the global communication variables @code{yylval}
|
|
and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
|
|
Parser}.) In such parsers the two global variables are replaced by
|
|
pointers passed as arguments to @code{yylex}. You must declare them as
|
|
shown here, and pass the information back by storing it through those
|
|
pointers.
|
|
|
|
@example
|
|
int
|
|
yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
|
|
@{
|
|
@dots{}
|
|
*lvalp = value; /* Put value onto Bison stack. */
|
|
return INT; /* Return the type of the token. */
|
|
@dots{}
|
|
@}
|
|
@end example
|
|
|
|
If the grammar file does not use the @samp{@@} constructs to refer to
|
|
textual locations, then the type @code{YYLTYPE} will not be defined. In
|
|
this case, omit the second argument; @code{yylex} will be called with
|
|
only one argument.
|
|
|
|
|
|
If you wish to pass the additional parameter data to @code{yylex}, use
|
|
@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
|
|
Function}).
|
|
|
|
@deffn {Directive} lex-param @{@var{argument-declaration}@}
|
|
@findex %lex-param
|
|
Declare that @code{argument-declaration} is an additional @code{yylex}
|
|
argument declaration.
|
|
@end deffn
|
|
|
|
For instance:
|
|
|
|
@example
|
|
%parse-param @{int *nastiness@}
|
|
%lex-param @{int *nastiness@}
|
|
%parse-param @{int *randomness@}
|
|
@end example
|
|
|
|
@noindent
|
|
results in the following signature:
|
|
|
|
@example
|
|
int yylex (int *nastiness);
|
|
int yyparse (int *nastiness, int *randomness);
|
|
@end example
|
|
|
|
If @code{%pure-parser} is added:
|
|
|
|
@example
|
|
int yylex (YYSTYPE *lvalp, int *nastiness);
|
|
int yyparse (int *nastiness, int *randomness);
|
|
@end example
|
|
|
|
@noindent
|
|
and finally, if both @code{%pure-parser} and @code{%locations} are used:
|
|
|
|
@example
|
|
int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
|
|
int yyparse (int *nastiness, int *randomness);
|
|
@end example
|
|
|
|
@node Error Reporting
|
|
@section The Error Reporting Function @code{yyerror}
|
|
@cindex error reporting function
|
|
@findex yyerror
|
|
@cindex parse error
|
|
@cindex syntax error
|
|
|
|
The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
|
|
whenever it reads a token which cannot satisfy any syntax rule. An
|
|
action in the grammar can also explicitly proclaim an error, using the
|
|
macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
|
|
in Actions}).
|
|
|
|
The Bison parser expects to report the error by calling an error
|
|
reporting function named @code{yyerror}, which you must supply. It is
|
|
called by @code{yyparse} whenever a syntax error is found, and it
|
|
receives one argument. For a syntax error, the string is normally
|
|
@w{@code{"syntax error"}}.
|
|
|
|
@findex %error-verbose
|
|
If you invoke the directive @code{%error-verbose} in the Bison
|
|
declarations section (@pxref{Bison Declarations, ,The Bison Declarations
|
|
Section}), then Bison provides a more verbose and specific error message
|
|
string instead of just plain @w{@code{"syntax error"}}.
|
|
|
|
The parser can detect one other kind of error: stack overflow. This
|
|
happens when the input contains constructions that are very deeply
|
|
nested. It isn't likely you will encounter this, since the Bison
|
|
parser extends its stack automatically up to a very large limit. But
|
|
if overflow happens, @code{yyparse} calls @code{yyerror} in the usual
|
|
fashion, except that the argument string is @w{@code{"parser stack
|
|
overflow"}}.
|
|
|
|
The following definition suffices in simple programs:
|
|
|
|
@example
|
|
@group
|
|
void
|
|
yyerror (char const *s)
|
|
@{
|
|
@end group
|
|
@group
|
|
fprintf (stderr, "%s\n", s);
|
|
@}
|
|
@end group
|
|
@end example
|
|
|
|
After @code{yyerror} returns to @code{yyparse}, the latter will attempt
|
|
error recovery if you have written suitable error recovery grammar rules
|
|
(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
|
|
immediately return 1.
|
|
|
|
Obviously, in location tracking pure parsers, @code{yyerror} should have
|
|
an access to the current location. This is indeed the case for the GLR
|
|
parsers, but not for the Yacc parser, for historical reasons. I.e., if
|
|
@samp{%locations %pure-parser} is passed then the prototypes for
|
|
@code{yyerror} are:
|
|
|
|
@example
|
|
void yyerror (char const *msg); /* Yacc parsers. */
|
|
void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
|
|
@end example
|
|
|
|
If @samp{%parse-param @{int *nastiness@}} is used, then:
|
|
|
|
@example
|
|
void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
|
|
void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
|
|
@end example
|
|
|
|
Finally, GLR and Yacc parsers share the same @code{yyerror} calling
|
|
convention for absolutely pure parsers, i.e., when the calling
|
|
convention of @code{yylex} @emph{and} the calling convention of
|
|
@code{%pure-parser} are pure. I.e.:
|
|
|
|
@example
|
|
/* Location tracking. */
|
|
%locations
|
|
/* Pure yylex. */
|
|
%pure-parser
|
|
%lex-param @{int *nastiness@}
|
|
/* Pure yyparse. */
|
|
%parse-param @{int *nastiness@}
|
|
%parse-param @{int *randomness@}
|
|
@end example
|
|
|
|
@noindent
|
|
results in the following signatures for all the parser kinds:
|
|
|
|
@example
|
|
int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
|
|
int yyparse (int *nastiness, int *randomness);
|
|
void yyerror (YYLTYPE *locp,
|
|
int *nastiness, int *randomness,
|
|
char const *msg);
|
|
@end example
|
|
|
|
@noindent
|
|
The prototypes are only indications of how the code produced by Bison
|
|
uses @code{yyerror}. Bison-generated code always ignores the returned
|
|
value, so @code{yyerror} can return any type, including @code{void}.
|
|
Also, @code{yyerror} can be a variadic function; that is why the
|
|
message is always passed last.
|
|
|
|
Traditionally @code{yyerror} returns an @code{int} that is always
|
|
ignored, but this is purely for historical reasons, and @code{void} is
|
|
preferable since it more accurately describes the return type for
|
|
@code{yyerror}.
|
|
|
|
@vindex yynerrs
|
|
The variable @code{yynerrs} contains the number of syntax errors
|
|
encountered so far. Normally this variable is global; but if you
|
|
request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
|
|
then it is a local variable which only the actions can access.
|
|
|
|
@node Action Features
|
|
@section Special Features for Use in Actions
|
|
@cindex summary, action features
|
|
@cindex action features summary
|
|
|
|
Here is a table of Bison constructs, variables and macros that
|
|
are useful in actions.
|
|
|
|
@deffn {Variable} $$
|
|
Acts like a variable that contains the semantic value for the
|
|
grouping made by the current rule. @xref{Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} $@var{n}
|
|
Acts like a variable that contains the semantic value for the
|
|
@var{n}th component of the current rule. @xref{Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} $<@var{typealt}>$
|
|
Like @code{$$} but specifies alternative @var{typealt} in the union
|
|
specified by the @code{%union} declaration. @xref{Action Types, ,Data
|
|
Types of Values in Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} $<@var{typealt}>@var{n}
|
|
Like @code{$@var{n}} but specifies alternative @var{typealt} in the
|
|
union specified by the @code{%union} declaration.
|
|
@xref{Action Types, ,Data Types of Values in Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYABORT;
|
|
Return immediately from @code{yyparse}, indicating failure.
|
|
@xref{Parser Function, ,The Parser Function @code{yyparse}}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYACCEPT;
|
|
Return immediately from @code{yyparse}, indicating success.
|
|
@xref{Parser Function, ,The Parser Function @code{yyparse}}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYBACKUP (@var{token}, @var{value});
|
|
@findex YYBACKUP
|
|
Unshift a token. This macro is allowed only for rules that reduce
|
|
a single value, and only when there is no look-ahead token.
|
|
It is also disallowed in @acronym{GLR} parsers.
|
|
It installs a look-ahead token with token type @var{token} and
|
|
semantic value @var{value}; then it discards the value that was
|
|
going to be reduced by this rule.
|
|
|
|
If the macro is used when it is not valid, such as when there is
|
|
a look-ahead token already, then it reports a syntax error with
|
|
a message @samp{cannot back up} and performs ordinary error
|
|
recovery.
|
|
|
|
In either case, the rest of the action is not executed.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYEMPTY
|
|
@vindex YYEMPTY
|
|
Value stored in @code{yychar} when there is no look-ahead token.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYERROR;
|
|
@findex YYERROR
|
|
Cause an immediate syntax error. This statement initiates error
|
|
recovery just as if the parser itself had detected an error; however, it
|
|
does not call @code{yyerror}, and does not print any message. If you
|
|
want to print an error message, call @code{yyerror} explicitly before
|
|
the @samp{YYERROR;} statement. @xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYRECOVERING
|
|
This macro stands for an expression that has the value 1 when the parser
|
|
is recovering from a syntax error, and 0 the rest of the time.
|
|
@xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} yychar
|
|
Variable containing the current look-ahead token. (In a pure parser,
|
|
this is actually a local variable within @code{yyparse}.) When there is
|
|
no look-ahead token, the value @code{YYEMPTY} is stored in the variable.
|
|
@xref{Look-Ahead, ,Look-Ahead Tokens}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} yyclearin;
|
|
Discard the current look-ahead token. This is useful primarily in
|
|
error rules. @xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} yyerrok;
|
|
Resume generating error messages immediately for subsequent syntax
|
|
errors. This is useful primarily in error rules.
|
|
@xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Value} @@$
|
|
@findex @@$
|
|
Acts like a structure variable containing information on the textual location
|
|
of the grouping made by the current rule. @xref{Locations, ,
|
|
Tracking Locations}.
|
|
|
|
@c Check if those paragraphs are still useful or not.
|
|
|
|
@c @example
|
|
@c struct @{
|
|
@c int first_line, last_line;
|
|
@c int first_column, last_column;
|
|
@c @};
|
|
@c @end example
|
|
|
|
@c Thus, to get the starting line number of the third component, you would
|
|
@c use @samp{@@3.first_line}.
|
|
|
|
@c In order for the members of this structure to contain valid information,
|
|
@c you must make @code{yylex} supply this information about each token.
|
|
@c If you need only certain members, then @code{yylex} need only fill in
|
|
@c those members.
|
|
|
|
@c The use of this feature makes the parser noticeably slower.
|
|
@end deffn
|
|
|
|
@deffn {Value} @@@var{n}
|
|
@findex @@@var{n}
|
|
Acts like a structure variable containing information on the textual location
|
|
of the @var{n}th component of the current rule. @xref{Locations, ,
|
|
Tracking Locations}.
|
|
@end deffn
|
|
|
|
|
|
@node Algorithm
|
|
@chapter The Bison Parser Algorithm
|
|
@cindex Bison parser algorithm
|
|
@cindex algorithm of parser
|
|
@cindex shifting
|
|
@cindex reduction
|
|
@cindex parser stack
|
|
@cindex stack, parser
|
|
|
|
As Bison reads tokens, it pushes them onto a stack along with their
|
|
semantic values. The stack is called the @dfn{parser stack}. Pushing a
|
|
token is traditionally called @dfn{shifting}.
|
|
|
|
For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
|
|
@samp{3} to come. The stack will have four elements, one for each token
|
|
that was shifted.
|
|
|
|
But the stack does not always have an element for each token read. When
|
|
the last @var{n} tokens and groupings shifted match the components of a
|
|
grammar rule, they can be combined according to that rule. This is called
|
|
@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
|
|
single grouping whose symbol is the result (left hand side) of that rule.
|
|
Running the rule's action is part of the process of reduction, because this
|
|
is what computes the semantic value of the resulting grouping.
|
|
|
|
For example, if the infix calculator's parser stack contains this:
|
|
|
|
@example
|
|
1 + 5 * 3
|
|
@end example
|
|
|
|
@noindent
|
|
and the next input token is a newline character, then the last three
|
|
elements can be reduced to 15 via the rule:
|
|
|
|
@example
|
|
expr: expr '*' expr;
|
|
@end example
|
|
|
|
@noindent
|
|
Then the stack contains just these three elements:
|
|
|
|
@example
|
|
1 + 15
|
|
@end example
|
|
|
|
@noindent
|
|
At this point, another reduction can be made, resulting in the single value
|
|
16. Then the newline token can be shifted.
|
|
|
|
The parser tries, by shifts and reductions, to reduce the entire input down
|
|
to a single grouping whose symbol is the grammar's start-symbol
|
|
(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
|
|
|
|
This kind of parser is known in the literature as a bottom-up parser.
|
|
|
|
@menu
|
|
* Look-Ahead:: Parser looks one token ahead when deciding what to do.
|
|
* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
|
|
* Precedence:: Operator precedence works by resolving conflicts.
|
|
* Contextual Precedence:: When an operator's precedence depends on context.
|
|
* Parser States:: The parser is a finite-state-machine with stack.
|
|
* Reduce/Reduce:: When two rules are applicable in the same situation.
|
|
* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
|
|
* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
|
|
* Stack Overflow:: What happens when stack gets full. How to avoid it.
|
|
@end menu
|
|
|
|
@node Look-Ahead
|
|
@section Look-Ahead Tokens
|
|
@cindex look-ahead token
|
|
|
|
The Bison parser does @emph{not} always reduce immediately as soon as the
|
|
last @var{n} tokens and groupings match a rule. This is because such a
|
|
simple strategy is inadequate to handle most languages. Instead, when a
|
|
reduction is possible, the parser sometimes ``looks ahead'' at the next
|
|
token in order to decide what to do.
|
|
|
|
When a token is read, it is not immediately shifted; first it becomes the
|
|
@dfn{look-ahead token}, which is not on the stack. Now the parser can
|
|
perform one or more reductions of tokens and groupings on the stack, while
|
|
the look-ahead token remains off to the side. When no more reductions
|
|
should take place, the look-ahead token is shifted onto the stack. This
|
|
does not mean that all possible reductions have been done; depending on the
|
|
token type of the look-ahead token, some rules may choose to delay their
|
|
application.
|
|
|
|
Here is a simple case where look-ahead is needed. These three rules define
|
|
expressions which contain binary addition operators and postfix unary
|
|
factorial operators (@samp{!}), and allow parentheses for grouping.
|
|
|
|
@example
|
|
@group
|
|
expr: term '+' expr
|
|
| term
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
term: '(' expr ')'
|
|
| term '!'
|
|
| NUMBER
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
|
|
should be done? If the following token is @samp{)}, then the first three
|
|
tokens must be reduced to form an @code{expr}. This is the only valid
|
|
course, because shifting the @samp{)} would produce a sequence of symbols
|
|
@w{@code{term ')'}}, and no rule allows this.
|
|
|
|
If the following token is @samp{!}, then it must be shifted immediately so
|
|
that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
|
|
parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
|
|
@code{expr}. It would then be impossible to shift the @samp{!} because
|
|
doing so would produce on the stack the sequence of symbols @code{expr
|
|
'!'}. No rule allows that sequence.
|
|
|
|
@vindex yychar
|
|
The current look-ahead token is stored in the variable @code{yychar}.
|
|
@xref{Action Features, ,Special Features for Use in Actions}.
|
|
|
|
@node Shift/Reduce
|
|
@section Shift/Reduce Conflicts
|
|
@cindex conflicts
|
|
@cindex shift/reduce conflicts
|
|
@cindex dangling @code{else}
|
|
@cindex @code{else}, dangling
|
|
|
|
Suppose we are parsing a language which has if-then and if-then-else
|
|
statements, with a pair of rules like this:
|
|
|
|
@example
|
|
@group
|
|
if_stmt:
|
|
IF expr THEN stmt
|
|
| IF expr THEN stmt ELSE stmt
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
|
|
terminal symbols for specific keyword tokens.
|
|
|
|
When the @code{ELSE} token is read and becomes the look-ahead token, the
|
|
contents of the stack (assuming the input is valid) are just right for
|
|
reduction by the first rule. But it is also legitimate to shift the
|
|
@code{ELSE}, because that would lead to eventual reduction by the second
|
|
rule.
|
|
|
|
This situation, where either a shift or a reduction would be valid, is
|
|
called a @dfn{shift/reduce conflict}. Bison is designed to resolve
|
|
these conflicts by choosing to shift, unless otherwise directed by
|
|
operator precedence declarations. To see the reason for this, let's
|
|
contrast it with the other alternative.
|
|
|
|
Since the parser prefers to shift the @code{ELSE}, the result is to attach
|
|
the else-clause to the innermost if-statement, making these two inputs
|
|
equivalent:
|
|
|
|
@example
|
|
if x then if y then win (); else lose;
|
|
|
|
if x then do; if y then win (); else lose; end;
|
|
@end example
|
|
|
|
But if the parser chose to reduce when possible rather than shift, the
|
|
result would be to attach the else-clause to the outermost if-statement,
|
|
making these two inputs equivalent:
|
|
|
|
@example
|
|
if x then if y then win (); else lose;
|
|
|
|
if x then do; if y then win (); end; else lose;
|
|
@end example
|
|
|
|
The conflict exists because the grammar as written is ambiguous: either
|
|
parsing of the simple nested if-statement is legitimate. The established
|
|
convention is that these ambiguities are resolved by attaching the
|
|
else-clause to the innermost if-statement; this is what Bison accomplishes
|
|
by choosing to shift rather than reduce. (It would ideally be cleaner to
|
|
write an unambiguous grammar, but that is very hard to do in this case.)
|
|
This particular ambiguity was first encountered in the specifications of
|
|
Algol 60 and is called the ``dangling @code{else}'' ambiguity.
|
|
|
|
To avoid warnings from Bison about predictable, legitimate shift/reduce
|
|
conflicts, use the @code{%expect @var{n}} declaration. There will be no
|
|
warning as long as the number of shift/reduce conflicts is exactly @var{n}.
|
|
@xref{Expect Decl, ,Suppressing Conflict Warnings}.
|
|
|
|
The definition of @code{if_stmt} above is solely to blame for the
|
|
conflict, but the conflict does not actually appear without additional
|
|
rules. Here is a complete Bison input file that actually manifests the
|
|
conflict:
|
|
|
|
@example
|
|
@group
|
|
%token IF THEN ELSE variable
|
|
%%
|
|
@end group
|
|
@group
|
|
stmt: expr
|
|
| if_stmt
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
if_stmt:
|
|
IF expr THEN stmt
|
|
| IF expr THEN stmt ELSE stmt
|
|
;
|
|
@end group
|
|
|
|
expr: variable
|
|
;
|
|
@end example
|
|
|
|
@node Precedence
|
|
@section Operator Precedence
|
|
@cindex operator precedence
|
|
@cindex precedence of operators
|
|
|
|
Another situation where shift/reduce conflicts appear is in arithmetic
|
|
expressions. Here shifting is not always the preferred resolution; the
|
|
Bison declarations for operator precedence allow you to specify when to
|
|
shift and when to reduce.
|
|
|
|
@menu
|
|
* Why Precedence:: An example showing why precedence is needed.
|
|
* Using Precedence:: How to specify precedence in Bison grammars.
|
|
* Precedence Examples:: How these features are used in the previous example.
|
|
* How Precedence:: How they work.
|
|
@end menu
|
|
|
|
@node Why Precedence
|
|
@subsection When Precedence is Needed
|
|
|
|
Consider the following ambiguous grammar fragment (ambiguous because the
|
|
input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
|
|
|
|
@example
|
|
@group
|
|
expr: expr '-' expr
|
|
| expr '*' expr
|
|
| expr '<' expr
|
|
| '(' expr ')'
|
|
@dots{}
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
|
|
should it reduce them via the rule for the subtraction operator? It
|
|
depends on the next token. Of course, if the next token is @samp{)}, we
|
|
must reduce; shifting is invalid because no single rule can reduce the
|
|
token sequence @w{@samp{- 2 )}} or anything starting with that. But if
|
|
the next token is @samp{*} or @samp{<}, we have a choice: either
|
|
shifting or reduction would allow the parse to complete, but with
|
|
different results.
|
|
|
|
To decide which one Bison should do, we must consider the results. If
|
|
the next operator token @var{op} is shifted, then it must be reduced
|
|
first in order to permit another opportunity to reduce the difference.
|
|
The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
|
|
hand, if the subtraction is reduced before shifting @var{op}, the result
|
|
is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
|
|
reduce should depend on the relative precedence of the operators
|
|
@samp{-} and @var{op}: @samp{*} should be shifted first, but not
|
|
@samp{<}.
|
|
|
|
@cindex associativity
|
|
What about input such as @w{@samp{1 - 2 - 5}}; should this be
|
|
@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
|
|
operators we prefer the former, which is called @dfn{left association}.
|
|
The latter alternative, @dfn{right association}, is desirable for
|
|
assignment operators. The choice of left or right association is a
|
|
matter of whether the parser chooses to shift or reduce when the stack
|
|
contains @w{@samp{1 - 2}} and the look-ahead token is @samp{-}: shifting
|
|
makes right-associativity.
|
|
|
|
@node Using Precedence
|
|
@subsection Specifying Operator Precedence
|
|
@findex %left
|
|
@findex %right
|
|
@findex %nonassoc
|
|
|
|
Bison allows you to specify these choices with the operator precedence
|
|
declarations @code{%left} and @code{%right}. Each such declaration
|
|
contains a list of tokens, which are operators whose precedence and
|
|
associativity is being declared. The @code{%left} declaration makes all
|
|
those operators left-associative and the @code{%right} declaration makes
|
|
them right-associative. A third alternative is @code{%nonassoc}, which
|
|
declares that it is a syntax error to find the same operator twice ``in a
|
|
row''.
|
|
|
|
The relative precedence of different operators is controlled by the
|
|
order in which they are declared. The first @code{%left} or
|
|
@code{%right} declaration in the file declares the operators whose
|
|
precedence is lowest, the next such declaration declares the operators
|
|
whose precedence is a little higher, and so on.
|
|
|
|
@node Precedence Examples
|
|
@subsection Precedence Examples
|
|
|
|
In our example, we would want the following declarations:
|
|
|
|
@example
|
|
%left '<'
|
|
%left '-'
|
|
%left '*'
|
|
@end example
|
|
|
|
In a more complete example, which supports other operators as well, we
|
|
would declare them in groups of equal precedence. For example, @code{'+'} is
|
|
declared with @code{'-'}:
|
|
|
|
@example
|
|
%left '<' '>' '=' NE LE GE
|
|
%left '+' '-'
|
|
%left '*' '/'
|
|
@end example
|
|
|
|
@noindent
|
|
(Here @code{NE} and so on stand for the operators for ``not equal''
|
|
and so on. We assume that these tokens are more than one character long
|
|
and therefore are represented by names, not character literals.)
|
|
|
|
@node How Precedence
|
|
@subsection How Precedence Works
|
|
|
|
The first effect of the precedence declarations is to assign precedence
|
|
levels to the terminal symbols declared. The second effect is to assign
|
|
precedence levels to certain rules: each rule gets its precedence from
|
|
the last terminal symbol mentioned in the components. (You can also
|
|
specify explicitly the precedence of a rule. @xref{Contextual
|
|
Precedence, ,Context-Dependent Precedence}.)
|
|
|
|
Finally, the resolution of conflicts works by comparing the precedence
|
|
of the rule being considered with that of the look-ahead token. If the
|
|
token's precedence is higher, the choice is to shift. If the rule's
|
|
precedence is higher, the choice is to reduce. If they have equal
|
|
precedence, the choice is made based on the associativity of that
|
|
precedence level. The verbose output file made by @samp{-v}
|
|
(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
|
|
resolved.
|
|
|
|
Not all rules and not all tokens have precedence. If either the rule or
|
|
the look-ahead token has no precedence, then the default is to shift.
|
|
|
|
@node Contextual Precedence
|
|
@section Context-Dependent Precedence
|
|
@cindex context-dependent precedence
|
|
@cindex unary operator precedence
|
|
@cindex precedence, context-dependent
|
|
@cindex precedence, unary operator
|
|
@findex %prec
|
|
|
|
Often the precedence of an operator depends on the context. This sounds
|
|
outlandish at first, but it is really very common. For example, a minus
|
|
sign typically has a very high precedence as a unary operator, and a
|
|
somewhat lower precedence (lower than multiplication) as a binary operator.
|
|
|
|
The Bison precedence declarations, @code{%left}, @code{%right} and
|
|
@code{%nonassoc}, can only be used once for a given token; so a token has
|
|
only one precedence declared in this way. For context-dependent
|
|
precedence, you need to use an additional mechanism: the @code{%prec}
|
|
modifier for rules.
|
|
|
|
The @code{%prec} modifier declares the precedence of a particular rule by
|
|
specifying a terminal symbol whose precedence should be used for that rule.
|
|
It's not necessary for that symbol to appear otherwise in the rule. The
|
|
modifier's syntax is:
|
|
|
|
@example
|
|
%prec @var{terminal-symbol}
|
|
@end example
|
|
|
|
@noindent
|
|
and it is written after the components of the rule. Its effect is to
|
|
assign the rule the precedence of @var{terminal-symbol}, overriding
|
|
the precedence that would be deduced for it in the ordinary way. The
|
|
altered rule precedence then affects how conflicts involving that rule
|
|
are resolved (@pxref{Precedence, ,Operator Precedence}).
|
|
|
|
Here is how @code{%prec} solves the problem of unary minus. First, declare
|
|
a precedence for a fictitious terminal symbol named @code{UMINUS}. There
|
|
are no tokens of this type, but the symbol serves to stand for its
|
|
precedence:
|
|
|
|
@example
|
|
@dots{}
|
|
%left '+' '-'
|
|
%left '*'
|
|
%left UMINUS
|
|
@end example
|
|
|
|
Now the precedence of @code{UMINUS} can be used in specific rules:
|
|
|
|
@example
|
|
@group
|
|
exp: @dots{}
|
|
| exp '-' exp
|
|
@dots{}
|
|
| '-' exp %prec UMINUS
|
|
@end group
|
|
@end example
|
|
|
|
@ifset defaultprec
|
|
If you forget to append @code{%prec UMINUS} to the rule for unary
|
|
minus, Bison silently assumes that minus has its usual precedence.
|
|
This kind of problem can be tricky to debug, since one typically
|
|
discovers the mistake only by testing the code.
|
|
|
|
The @code{%no-default-prec;} declaration makes it easier to discover
|
|
this kind of problem systematically. It causes rules that lack a
|
|
@code{%prec} modifier to have no precedence, even if the last terminal
|
|
symbol mentioned in their components has a declared precedence.
|
|
|
|
If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
|
|
for all rules that participate in precedence conflict resolution.
|
|
Then you will see any shift/reduce conflict until you tell Bison how
|
|
to resolve it, either by changing your grammar or by adding an
|
|
explicit precedence. This will probably add declarations to the
|
|
grammar, but it helps to protect against incorrect rule precedences.
|
|
|
|
The effect of @code{%no-default-prec;} can be reversed by giving
|
|
@code{%default-prec;}, which is the default.
|
|
@end ifset
|
|
|
|
@node Parser States
|
|
@section Parser States
|
|
@cindex finite-state machine
|
|
@cindex parser state
|
|
@cindex state (of parser)
|
|
|
|
The function @code{yyparse} is implemented using a finite-state machine.
|
|
The values pushed on the parser stack are not simply token type codes; they
|
|
represent the entire sequence of terminal and nonterminal symbols at or
|
|
near the top of the stack. The current state collects all the information
|
|
about previous input which is relevant to deciding what to do next.
|
|
|
|
Each time a look-ahead token is read, the current parser state together
|
|
with the type of look-ahead token are looked up in a table. This table
|
|
entry can say, ``Shift the look-ahead token.'' In this case, it also
|
|
specifies the new parser state, which is pushed onto the top of the
|
|
parser stack. Or it can say, ``Reduce using rule number @var{n}.''
|
|
This means that a certain number of tokens or groupings are taken off
|
|
the top of the stack, and replaced by one grouping. In other words,
|
|
that number of states are popped from the stack, and one new state is
|
|
pushed.
|
|
|
|
There is one other alternative: the table can say that the look-ahead token
|
|
is erroneous in the current state. This causes error processing to begin
|
|
(@pxref{Error Recovery}).
|
|
|
|
@node Reduce/Reduce
|
|
@section Reduce/Reduce Conflicts
|
|
@cindex reduce/reduce conflict
|
|
@cindex conflicts, reduce/reduce
|
|
|
|
A reduce/reduce conflict occurs if there are two or more rules that apply
|
|
to the same sequence of input. This usually indicates a serious error
|
|
in the grammar.
|
|
|
|
For example, here is an erroneous attempt to define a sequence
|
|
of zero or more @code{word} groupings.
|
|
|
|
@example
|
|
sequence: /* empty */
|
|
@{ printf ("empty sequence\n"); @}
|
|
| maybeword
|
|
| sequence word
|
|
@{ printf ("added word %s\n", $2); @}
|
|
;
|
|
|
|
maybeword: /* empty */
|
|
@{ printf ("empty maybeword\n"); @}
|
|
| word
|
|
@{ printf ("single word %s\n", $1); @}
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
The error is an ambiguity: there is more than one way to parse a single
|
|
@code{word} into a @code{sequence}. It could be reduced to a
|
|
@code{maybeword} and then into a @code{sequence} via the second rule.
|
|
Alternatively, nothing-at-all could be reduced into a @code{sequence}
|
|
via the first rule, and this could be combined with the @code{word}
|
|
using the third rule for @code{sequence}.
|
|
|
|
There is also more than one way to reduce nothing-at-all into a
|
|
@code{sequence}. This can be done directly via the first rule,
|
|
or indirectly via @code{maybeword} and then the second rule.
|
|
|
|
You might think that this is a distinction without a difference, because it
|
|
does not change whether any particular input is valid or not. But it does
|
|
affect which actions are run. One parsing order runs the second rule's
|
|
action; the other runs the first rule's action and the third rule's action.
|
|
In this example, the output of the program changes.
|
|
|
|
Bison resolves a reduce/reduce conflict by choosing to use the rule that
|
|
appears first in the grammar, but it is very risky to rely on this. Every
|
|
reduce/reduce conflict must be studied and usually eliminated. Here is the
|
|
proper way to define @code{sequence}:
|
|
|
|
@example
|
|
sequence: /* empty */
|
|
@{ printf ("empty sequence\n"); @}
|
|
| sequence word
|
|
@{ printf ("added word %s\n", $2); @}
|
|
;
|
|
@end example
|
|
|
|
Here is another common error that yields a reduce/reduce conflict:
|
|
|
|
@example
|
|
sequence: /* empty */
|
|
| sequence words
|
|
| sequence redirects
|
|
;
|
|
|
|
words: /* empty */
|
|
| words word
|
|
;
|
|
|
|
redirects:/* empty */
|
|
| redirects redirect
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
The intention here is to define a sequence which can contain either
|
|
@code{word} or @code{redirect} groupings. The individual definitions of
|
|
@code{sequence}, @code{words} and @code{redirects} are error-free, but the
|
|
three together make a subtle ambiguity: even an empty input can be parsed
|
|
in infinitely many ways!
|
|
|
|
Consider: nothing-at-all could be a @code{words}. Or it could be two
|
|
@code{words} in a row, or three, or any number. It could equally well be a
|
|
@code{redirects}, or two, or any number. Or it could be a @code{words}
|
|
followed by three @code{redirects} and another @code{words}. And so on.
|
|
|
|
Here are two ways to correct these rules. First, to make it a single level
|
|
of sequence:
|
|
|
|
@example
|
|
sequence: /* empty */
|
|
| sequence word
|
|
| sequence redirect
|
|
;
|
|
@end example
|
|
|
|
Second, to prevent either a @code{words} or a @code{redirects}
|
|
from being empty:
|
|
|
|
@example
|
|
sequence: /* empty */
|
|
| sequence words
|
|
| sequence redirects
|
|
;
|
|
|
|
words: word
|
|
| words word
|
|
;
|
|
|
|
redirects:redirect
|
|
| redirects redirect
|
|
;
|
|
@end example
|
|
|
|
@node Mystery Conflicts
|
|
@section Mysterious Reduce/Reduce Conflicts
|
|
|
|
Sometimes reduce/reduce conflicts can occur that don't look warranted.
|
|
Here is an example:
|
|
|
|
@example
|
|
@group
|
|
%token ID
|
|
|
|
%%
|
|
def: param_spec return_spec ','
|
|
;
|
|
param_spec:
|
|
type
|
|
| name_list ':' type
|
|
;
|
|
@end group
|
|
@group
|
|
return_spec:
|
|
type
|
|
| name ':' type
|
|
;
|
|
@end group
|
|
@group
|
|
type: ID
|
|
;
|
|
@end group
|
|
@group
|
|
name: ID
|
|
;
|
|
name_list:
|
|
name
|
|
| name ',' name_list
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
It would seem that this grammar can be parsed with only a single token
|
|
of look-ahead: when a @code{param_spec} is being read, an @code{ID} is
|
|
a @code{name} if a comma or colon follows, or a @code{type} if another
|
|
@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
|
|
|
|
@cindex @acronym{LR}(1)
|
|
@cindex @acronym{LALR}(1)
|
|
However, Bison, like most parser generators, cannot actually handle all
|
|
@acronym{LR}(1) grammars. In this grammar, two contexts, that after
|
|
an @code{ID}
|
|
at the beginning of a @code{param_spec} and likewise at the beginning of
|
|
a @code{return_spec}, are similar enough that Bison assumes they are the
|
|
same. They appear similar because the same set of rules would be
|
|
active---the rule for reducing to a @code{name} and that for reducing to
|
|
a @code{type}. Bison is unable to determine at that stage of processing
|
|
that the rules would require different look-ahead tokens in the two
|
|
contexts, so it makes a single parser state for them both. Combining
|
|
the two contexts causes a conflict later. In parser terminology, this
|
|
occurrence means that the grammar is not @acronym{LALR}(1).
|
|
|
|
In general, it is better to fix deficiencies than to document them. But
|
|
this particular deficiency is intrinsically hard to fix; parser
|
|
generators that can handle @acronym{LR}(1) grammars are hard to write
|
|
and tend to
|
|
produce parsers that are very large. In practice, Bison is more useful
|
|
as it is now.
|
|
|
|
When the problem arises, you can often fix it by identifying the two
|
|
parser states that are being confused, and adding something to make them
|
|
look distinct. In the above example, adding one rule to
|
|
@code{return_spec} as follows makes the problem go away:
|
|
|
|
@example
|
|
@group
|
|
%token BOGUS
|
|
@dots{}
|
|
%%
|
|
@dots{}
|
|
return_spec:
|
|
type
|
|
| name ':' type
|
|
/* This rule is never used. */
|
|
| ID BOGUS
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
This corrects the problem because it introduces the possibility of an
|
|
additional active rule in the context after the @code{ID} at the beginning of
|
|
@code{return_spec}. This rule is not active in the corresponding context
|
|
in a @code{param_spec}, so the two contexts receive distinct parser states.
|
|
As long as the token @code{BOGUS} is never generated by @code{yylex},
|
|
the added rule cannot alter the way actual input is parsed.
|
|
|
|
In this particular example, there is another way to solve the problem:
|
|
rewrite the rule for @code{return_spec} to use @code{ID} directly
|
|
instead of via @code{name}. This also causes the two confusing
|
|
contexts to have different sets of active rules, because the one for
|
|
@code{return_spec} activates the altered rule for @code{return_spec}
|
|
rather than the one for @code{name}.
|
|
|
|
@example
|
|
param_spec:
|
|
type
|
|
| name_list ':' type
|
|
;
|
|
return_spec:
|
|
type
|
|
| ID ':' type
|
|
;
|
|
@end example
|
|
|
|
@node Generalized LR Parsing
|
|
@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
|
|
@cindex @acronym{GLR} parsing
|
|
@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
|
|
@cindex ambiguous grammars
|
|
@cindex non-deterministic parsing
|
|
|
|
Bison produces @emph{deterministic} parsers that choose uniquely
|
|
when to reduce and which reduction to apply
|
|
based on a summary of the preceding input and on one extra token of look-ahead.
|
|
As a result, normal Bison handles a proper subset of the family of
|
|
context-free languages.
|
|
Ambiguous grammars, since they have strings with more than one possible
|
|
sequence of reductions cannot have deterministic parsers in this sense.
|
|
The same is true of languages that require more than one symbol of
|
|
look-ahead, since the parser lacks the information necessary to make a
|
|
decision at the point it must be made in a shift-reduce parser.
|
|
Finally, as previously mentioned (@pxref{Mystery Conflicts}),
|
|
there are languages where Bison's particular choice of how to
|
|
summarize the input seen so far loses necessary information.
|
|
|
|
When you use the @samp{%glr-parser} declaration in your grammar file,
|
|
Bison generates a parser that uses a different algorithm, called
|
|
Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
|
|
parser uses the same basic
|
|
algorithm for parsing as an ordinary Bison parser, but behaves
|
|
differently in cases where there is a shift-reduce conflict that has not
|
|
been resolved by precedence rules (@pxref{Precedence}) or a
|
|
reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
|
|
situation, it
|
|
effectively @emph{splits} into a several parsers, one for each possible
|
|
shift or reduction. These parsers then proceed as usual, consuming
|
|
tokens in lock-step. Some of the stacks may encounter other conflicts
|
|
and split further, with the result that instead of a sequence of states,
|
|
a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
|
|
|
|
In effect, each stack represents a guess as to what the proper parse
|
|
is. Additional input may indicate that a guess was wrong, in which case
|
|
the appropriate stack silently disappears. Otherwise, the semantics
|
|
actions generated in each stack are saved, rather than being executed
|
|
immediately. When a stack disappears, its saved semantic actions never
|
|
get executed. When a reduction causes two stacks to become equivalent,
|
|
their sets of semantic actions are both saved with the state that
|
|
results from the reduction. We say that two stacks are equivalent
|
|
when they both represent the same sequence of states,
|
|
and each pair of corresponding states represents a
|
|
grammar symbol that produces the same segment of the input token
|
|
stream.
|
|
|
|
Whenever the parser makes a transition from having multiple
|
|
states to having one, it reverts to the normal @acronym{LALR}(1) parsing
|
|
algorithm, after resolving and executing the saved-up actions.
|
|
At this transition, some of the states on the stack will have semantic
|
|
values that are sets (actually multisets) of possible actions. The
|
|
parser tries to pick one of the actions by first finding one whose rule
|
|
has the highest dynamic precedence, as set by the @samp{%dprec}
|
|
declaration. Otherwise, if the alternative actions are not ordered by
|
|
precedence, but there the same merging function is declared for both
|
|
rules by the @samp{%merge} declaration,
|
|
Bison resolves and evaluates both and then calls the merge function on
|
|
the result. Otherwise, it reports an ambiguity.
|
|
|
|
It is possible to use a data structure for the @acronym{GLR} parsing tree that
|
|
permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
|
|
size of the input), any unambiguous (not necessarily
|
|
@acronym{LALR}(1)) grammar in
|
|
quadratic worst-case time, and any general (possibly ambiguous)
|
|
context-free grammar in cubic worst-case time. However, Bison currently
|
|
uses a simpler data structure that requires time proportional to the
|
|
length of the input times the maximum number of stacks required for any
|
|
prefix of the input. Thus, really ambiguous or non-deterministic
|
|
grammars can require exponential time and space to process. Such badly
|
|
behaving examples, however, are not generally of practical interest.
|
|
Usually, non-determinism in a grammar is local---the parser is ``in
|
|
doubt'' only for a few tokens at a time. Therefore, the current data
|
|
structure should generally be adequate. On @acronym{LALR}(1) portions of a
|
|
grammar, in particular, it is only slightly slower than with the default
|
|
Bison parser.
|
|
|
|
For a more detailed exposition of GLR parsers, please see: Elizabeth
|
|
Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
|
|
Generalised @acronym{LR} Parsers, Royal Holloway, University of
|
|
London, Department of Computer Science, TR-00-12,
|
|
@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
|
|
(2000-12-24).
|
|
|
|
@node Stack Overflow
|
|
@section Stack Overflow, and How to Avoid It
|
|
@cindex stack overflow
|
|
@cindex parser stack overflow
|
|
@cindex overflow of parser stack
|
|
|
|
The Bison parser stack can overflow if too many tokens are shifted and
|
|
not reduced. When this happens, the parser function @code{yyparse}
|
|
returns a nonzero value, pausing only to call @code{yyerror} to report
|
|
the overflow.
|
|
|
|
Because Bison parsers have growing stacks, hitting the upper limit
|
|
usually results from using a right recursion instead of a left
|
|
recursion, @xref{Recursion, ,Recursive Rules}.
|
|
|
|
@vindex YYMAXDEPTH
|
|
By defining the macro @code{YYMAXDEPTH}, you can control how deep the
|
|
parser stack can become before a stack overflow occurs. Define the
|
|
macro with a value that is an integer. This value is the maximum number
|
|
of tokens that can be shifted (and not reduced) before overflow.
|
|
It must be a constant expression whose value is known at compile time.
|
|
|
|
The stack space allowed is not necessarily allocated. If you specify a
|
|
large value for @code{YYMAXDEPTH}, the parser actually allocates a small
|
|
stack at first, and then makes it bigger by stages as needed. This
|
|
increasing allocation happens automatically and silently. Therefore,
|
|
you do not need to make @code{YYMAXDEPTH} painfully small merely to save
|
|
space for ordinary inputs that do not need much stack.
|
|
|
|
@cindex default stack limit
|
|
The default value of @code{YYMAXDEPTH}, if you do not define it, is
|
|
10000.
|
|
|
|
@vindex YYINITDEPTH
|
|
You can control how much stack is allocated initially by defining the
|
|
macro @code{YYINITDEPTH}. This value too must be a compile-time
|
|
constant integer. The default is 200.
|
|
|
|
@c FIXME: C++ output.
|
|
Because of semantical differences between C and C++, the
|
|
@acronym{LALR}(1) parsers
|
|
in C produced by Bison by compiled as C++ cannot grow. In this precise
|
|
case (compiling a C parser as C++) you are suggested to grow
|
|
@code{YYINITDEPTH}. In the near future, a C++ output output will be
|
|
provided which addresses this issue.
|
|
|
|
@node Error Recovery
|
|
@chapter Error Recovery
|
|
@cindex error recovery
|
|
@cindex recovery from errors
|
|
|
|
It is not usually acceptable to have a program terminate on a syntax
|
|
error. For example, a compiler should recover sufficiently to parse the
|
|
rest of the input file and check it for errors; a calculator should accept
|
|
another expression.
|
|
|
|
In a simple interactive command parser where each input is one line, it may
|
|
be sufficient to allow @code{yyparse} to return 1 on error and have the
|
|
caller ignore the rest of the input line when that happens (and then call
|
|
@code{yyparse} again). But this is inadequate for a compiler, because it
|
|
forgets all the syntactic context leading up to the error. A syntax error
|
|
deep within a function in the compiler input should not cause the compiler
|
|
to treat the following line like the beginning of a source file.
|
|
|
|
@findex error
|
|
You can define how to recover from a syntax error by writing rules to
|
|
recognize the special token @code{error}. This is a terminal symbol that
|
|
is always defined (you need not declare it) and reserved for error
|
|
handling. The Bison parser generates an @code{error} token whenever a
|
|
syntax error happens; if you have provided a rule to recognize this token
|
|
in the current context, the parse can continue.
|
|
|
|
For example:
|
|
|
|
@example
|
|
stmnts: /* empty string */
|
|
| stmnts '\n'
|
|
| stmnts exp '\n'
|
|
| stmnts error '\n'
|
|
@end example
|
|
|
|
The fourth rule in this example says that an error followed by a newline
|
|
makes a valid addition to any @code{stmnts}.
|
|
|
|
What happens if a syntax error occurs in the middle of an @code{exp}? The
|
|
error recovery rule, interpreted strictly, applies to the precise sequence
|
|
of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
|
|
the middle of an @code{exp}, there will probably be some additional tokens
|
|
and subexpressions on the stack after the last @code{stmnts}, and there
|
|
will be tokens to read before the next newline. So the rule is not
|
|
applicable in the ordinary way.
|
|
|
|
But Bison can force the situation to fit the rule, by discarding part of
|
|
the semantic context and part of the input. First it discards states
|
|
and objects from the stack until it gets back to a state in which the
|
|
@code{error} token is acceptable. (This means that the subexpressions
|
|
already parsed are discarded, back to the last complete @code{stmnts}.)
|
|
At this point the @code{error} token can be shifted. Then, if the old
|
|
look-ahead token is not acceptable to be shifted next, the parser reads
|
|
tokens and discards them until it finds a token which is acceptable. In
|
|
this example, Bison reads and discards input until the next newline so
|
|
that the fourth rule can apply. Note that discarded symbols are
|
|
possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
|
|
Discarded Symbols}, for a means to reclaim this memory.
|
|
|
|
The choice of error rules in the grammar is a choice of strategies for
|
|
error recovery. A simple and useful strategy is simply to skip the rest of
|
|
the current input line or current statement if an error is detected:
|
|
|
|
@example
|
|
stmnt: error ';' /* On error, skip until ';' is read. */
|
|
@end example
|
|
|
|
It is also useful to recover to the matching close-delimiter of an
|
|
opening-delimiter that has already been parsed. Otherwise the
|
|
close-delimiter will probably appear to be unmatched, and generate another,
|
|
spurious error message:
|
|
|
|
@example
|
|
primary: '(' expr ')'
|
|
| '(' error ')'
|
|
@dots{}
|
|
;
|
|
@end example
|
|
|
|
Error recovery strategies are necessarily guesses. When they guess wrong,
|
|
one syntax error often leads to another. In the above example, the error
|
|
recovery rule guesses that an error is due to bad input within one
|
|
@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
|
|
middle of a valid @code{stmnt}. After the error recovery rule recovers
|
|
from the first error, another syntax error will be found straightaway,
|
|
since the text following the spurious semicolon is also an invalid
|
|
@code{stmnt}.
|
|
|
|
To prevent an outpouring of error messages, the parser will output no error
|
|
message for another syntax error that happens shortly after the first; only
|
|
after three consecutive input tokens have been successfully shifted will
|
|
error messages resume.
|
|
|
|
Note that rules which accept the @code{error} token may have actions, just
|
|
as any other rules can.
|
|
|
|
@findex yyerrok
|
|
You can make error messages resume immediately by using the macro
|
|
@code{yyerrok} in an action. If you do this in the error rule's action, no
|
|
error messages will be suppressed. This macro requires no arguments;
|
|
@samp{yyerrok;} is a valid C statement.
|
|
|
|
@findex yyclearin
|
|
The previous look-ahead token is reanalyzed immediately after an error. If
|
|
this is unacceptable, then the macro @code{yyclearin} may be used to clear
|
|
this token. Write the statement @samp{yyclearin;} in the error rule's
|
|
action.
|
|
|
|
For example, suppose that on a syntax error, an error handling routine is
|
|
called that advances the input stream to some point where parsing should
|
|
once again commence. The next symbol returned by the lexical scanner is
|
|
probably correct. The previous look-ahead token ought to be discarded
|
|
with @samp{yyclearin;}.
|
|
|
|
@vindex YYRECOVERING
|
|
The macro @code{YYRECOVERING} stands for an expression that has the
|
|
value 1 when the parser is recovering from a syntax error, and 0 the
|
|
rest of the time. A value of 1 indicates that error messages are
|
|
currently suppressed for new syntax errors.
|
|
|
|
@node Context Dependency
|
|
@chapter Handling Context Dependencies
|
|
|
|
The Bison paradigm is to parse tokens first, then group them into larger
|
|
syntactic units. In many languages, the meaning of a token is affected by
|
|
its context. Although this violates the Bison paradigm, certain techniques
|
|
(known as @dfn{kludges}) may enable you to write Bison parsers for such
|
|
languages.
|
|
|
|
@menu
|
|
* Semantic Tokens:: Token parsing can depend on the semantic context.
|
|
* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
|
|
* Tie-in Recovery:: Lexical tie-ins have implications for how
|
|
error recovery rules must be written.
|
|
@end menu
|
|
|
|
(Actually, ``kludge'' means any technique that gets its job done but is
|
|
neither clean nor robust.)
|
|
|
|
@node Semantic Tokens
|
|
@section Semantic Info in Token Types
|
|
|
|
The C language has a context dependency: the way an identifier is used
|
|
depends on what its current meaning is. For example, consider this:
|
|
|
|
@example
|
|
foo (x);
|
|
@end example
|
|
|
|
This looks like a function call statement, but if @code{foo} is a typedef
|
|
name, then this is actually a declaration of @code{x}. How can a Bison
|
|
parser for C decide how to parse this input?
|
|
|
|
The method used in @acronym{GNU} C is to have two different token types,
|
|
@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
|
|
identifier, it looks up the current declaration of the identifier in order
|
|
to decide which token type to return: @code{TYPENAME} if the identifier is
|
|
declared as a typedef, @code{IDENTIFIER} otherwise.
|
|
|
|
The grammar rules can then express the context dependency by the choice of
|
|
token type to recognize. @code{IDENTIFIER} is accepted as an expression,
|
|
but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
|
|
@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
|
|
is @emph{not} significant, such as in declarations that can shadow a
|
|
typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
|
|
accepted---there is one rule for each of the two token types.
|
|
|
|
This technique is simple to use if the decision of which kinds of
|
|
identifiers to allow is made at a place close to where the identifier is
|
|
parsed. But in C this is not always so: C allows a declaration to
|
|
redeclare a typedef name provided an explicit type has been specified
|
|
earlier:
|
|
|
|
@example
|
|
typedef int foo, bar, lose;
|
|
static foo (bar); /* @r{redeclare @code{bar} as static variable} */
|
|
static int foo (lose); /* @r{redeclare @code{foo} as function} */
|
|
@end example
|
|
|
|
Unfortunately, the name being declared is separated from the declaration
|
|
construct itself by a complicated syntactic structure---the ``declarator''.
|
|
|
|
As a result, part of the Bison parser for C needs to be duplicated, with
|
|
all the nonterminal names changed: once for parsing a declaration in
|
|
which a typedef name can be redefined, and once for parsing a
|
|
declaration in which that can't be done. Here is a part of the
|
|
duplication, with actions omitted for brevity:
|
|
|
|
@example
|
|
initdcl:
|
|
declarator maybeasm '='
|
|
init
|
|
| declarator maybeasm
|
|
;
|
|
|
|
notype_initdcl:
|
|
notype_declarator maybeasm '='
|
|
init
|
|
| notype_declarator maybeasm
|
|
;
|
|
@end example
|
|
|
|
@noindent
|
|
Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
|
|
cannot. The distinction between @code{declarator} and
|
|
@code{notype_declarator} is the same sort of thing.
|
|
|
|
There is some similarity between this technique and a lexical tie-in
|
|
(described next), in that information which alters the lexical analysis is
|
|
changed during parsing by other parts of the program. The difference is
|
|
here the information is global, and is used for other purposes in the
|
|
program. A true lexical tie-in has a special-purpose flag controlled by
|
|
the syntactic context.
|
|
|
|
@node Lexical Tie-ins
|
|
@section Lexical Tie-ins
|
|
@cindex lexical tie-in
|
|
|
|
One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
|
|
which is set by Bison actions, whose purpose is to alter the way tokens are
|
|
parsed.
|
|
|
|
For example, suppose we have a language vaguely like C, but with a special
|
|
construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
|
|
an expression in parentheses in which all integers are hexadecimal. In
|
|
particular, the token @samp{a1b} must be treated as an integer rather than
|
|
as an identifier if it appears in that context. Here is how you can do it:
|
|
|
|
@example
|
|
@group
|
|
%@{
|
|
int hexflag;
|
|
int yylex (void);
|
|
void yyerror (char const *);
|
|
%@}
|
|
%%
|
|
@dots{}
|
|
@end group
|
|
@group
|
|
expr: IDENTIFIER
|
|
| constant
|
|
| HEX '('
|
|
@{ hexflag = 1; @}
|
|
expr ')'
|
|
@{ hexflag = 0;
|
|
$$ = $4; @}
|
|
| expr '+' expr
|
|
@{ $$ = make_sum ($1, $3); @}
|
|
@dots{}
|
|
;
|
|
@end group
|
|
|
|
@group
|
|
constant:
|
|
INTEGER
|
|
| STRING
|
|
;
|
|
@end group
|
|
@end example
|
|
|
|
@noindent
|
|
Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
|
|
it is nonzero, all integers are parsed in hexadecimal, and tokens starting
|
|
with letters are parsed as integers if possible.
|
|
|
|
The declaration of @code{hexflag} shown in the prologue of the parser file
|
|
is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
|
|
You must also write the code in @code{yylex} to obey the flag.
|
|
|
|
@node Tie-in Recovery
|
|
@section Lexical Tie-ins and Error Recovery
|
|
|
|
Lexical tie-ins make strict demands on any error recovery rules you have.
|
|
@xref{Error Recovery}.
|
|
|
|
The reason for this is that the purpose of an error recovery rule is to
|
|
abort the parsing of one construct and resume in some larger construct.
|
|
For example, in C-like languages, a typical error recovery rule is to skip
|
|
tokens until the next semicolon, and then start a new statement, like this:
|
|
|
|
@example
|
|
stmt: expr ';'
|
|
| IF '(' expr ')' stmt @{ @dots{} @}
|
|
@dots{}
|
|
error ';'
|
|
@{ hexflag = 0; @}
|
|
;
|
|
@end example
|
|
|
|
If there is a syntax error in the middle of a @samp{hex (@var{expr})}
|
|
construct, this error rule will apply, and then the action for the
|
|
completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
|
|
remain set for the entire rest of the input, or until the next @code{hex}
|
|
keyword, causing identifiers to be misinterpreted as integers.
|
|
|
|
To avoid this problem the error recovery rule itself clears @code{hexflag}.
|
|
|
|
There may also be an error recovery rule that works within expressions.
|
|
For example, there could be a rule which applies within parentheses
|
|
and skips to the close-parenthesis:
|
|
|
|
@example
|
|
@group
|
|
expr: @dots{}
|
|
| '(' expr ')'
|
|
@{ $$ = $2; @}
|
|
| '(' error ')'
|
|
@dots{}
|
|
@end group
|
|
@end example
|
|
|
|
If this rule acts within the @code{hex} construct, it is not going to abort
|
|
that construct (since it applies to an inner level of parentheses within
|
|
the construct). Therefore, it should not clear the flag: the rest of
|
|
the @code{hex} construct should be parsed with the flag still in effect.
|
|
|
|
What if there is an error recovery rule which might abort out of the
|
|
@code{hex} construct or might not, depending on circumstances? There is no
|
|
way you can write the action to determine whether a @code{hex} construct is
|
|
being aborted or not. So if you are using a lexical tie-in, you had better
|
|
make sure your error recovery rules are not of this kind. Each rule must
|
|
be such that you can be sure that it always will, or always won't, have to
|
|
clear the flag.
|
|
|
|
@c ================================================== Debugging Your Parser
|
|
|
|
@node Debugging
|
|
@chapter Debugging Your Parser
|
|
|
|
Developing a parser can be a challenge, especially if you don't
|
|
understand the algorithm (@pxref{Algorithm, ,The Bison Parser
|
|
Algorithm}). Even so, sometimes a detailed description of the automaton
|
|
can help (@pxref{Understanding, , Understanding Your Parser}), or
|
|
tracing the execution of the parser can give some insight on why it
|
|
behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
|
|
|
|
@menu
|
|
* Understanding:: Understanding the structure of your parser.
|
|
* Tracing:: Tracing the execution of your parser.
|
|
@end menu
|
|
|
|
@node Understanding
|
|
@section Understanding Your Parser
|
|
|
|
As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
|
|
Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
|
|
frequent than one would hope), looking at this automaton is required to
|
|
tune or simply fix a parser. Bison provides two different
|
|
representation of it, either textually or graphically (as a @acronym{VCG}
|
|
file).
|
|
|
|
The textual file is generated when the options @option{--report} or
|
|
@option{--verbose} are specified, see @xref{Invocation, , Invoking
|
|
Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
|
|
the parser output file name, and adding @samp{.output} instead.
|
|
Therefore, if the input file is @file{foo.y}, then the parser file is
|
|
called @file{foo.tab.c} by default. As a consequence, the verbose
|
|
output file is called @file{foo.output}.
|
|
|
|
The following grammar file, @file{calc.y}, will be used in the sequel:
|
|
|
|
@example
|
|
%token NUM STR
|
|
%left '+' '-'
|
|
%left '*'
|
|
%%
|
|
exp: exp '+' exp
|
|
| exp '-' exp
|
|
| exp '*' exp
|
|
| exp '/' exp
|
|
| NUM
|
|
;
|
|
useless: STR;
|
|
%%
|
|
@end example
|
|
|
|
@command{bison} reports:
|
|
|
|
@example
|
|
calc.y: warning: 1 useless nonterminal and 1 useless rule
|
|
calc.y:11.1-7: warning: useless nonterminal: useless
|
|
calc.y:11.10-12: warning: useless rule: useless: STR
|
|
calc.y: conflicts: 7 shift/reduce
|
|
@end example
|
|
|
|
When given @option{--report=state}, in addition to @file{calc.tab.c}, it
|
|
creates a file @file{calc.output} with contents detailed below. The
|
|
order of the output and the exact presentation might vary, but the
|
|
interpretation is the same.
|
|
|
|
The first section includes details on conflicts that were solved thanks
|
|
to precedence and/or associativity:
|
|
|
|
@example
|
|
Conflict in state 8 between rule 2 and token '+' resolved as reduce.
|
|
Conflict in state 8 between rule 2 and token '-' resolved as reduce.
|
|
Conflict in state 8 between rule 2 and token '*' resolved as shift.
|
|
@exdent @dots{}
|
|
@end example
|
|
|
|
@noindent
|
|
The next section lists states that still have conflicts.
|
|
|
|
@example
|
|
State 8 conflicts: 1 shift/reduce
|
|
State 9 conflicts: 1 shift/reduce
|
|
State 10 conflicts: 1 shift/reduce
|
|
State 11 conflicts: 4 shift/reduce
|
|
@end example
|
|
|
|
@noindent
|
|
@cindex token, useless
|
|
@cindex useless token
|
|
@cindex nonterminal, useless
|
|
@cindex useless nonterminal
|
|
@cindex rule, useless
|
|
@cindex useless rule
|
|
The next section reports useless tokens, nonterminal and rules. Useless
|
|
nonterminals and rules are removed in order to produce a smaller parser,
|
|
but useless tokens are preserved, since they might be used by the
|
|
scanner (note the difference between ``useless'' and ``not used''
|
|
below):
|
|
|
|
@example
|
|
Useless nonterminals:
|
|
useless
|
|
|
|
Terminals which are not used:
|
|
STR
|
|
|
|
Useless rules:
|
|
#6 useless: STR;
|
|
@end example
|
|
|
|
@noindent
|
|
The next section reproduces the exact grammar that Bison used:
|
|
|
|
@example
|
|
Grammar
|
|
|
|
Number, Line, Rule
|
|
0 5 $accept -> exp $end
|
|
1 5 exp -> exp '+' exp
|
|
2 6 exp -> exp '-' exp
|
|
3 7 exp -> exp '*' exp
|
|
4 8 exp -> exp '/' exp
|
|
5 9 exp -> NUM
|
|
@end example
|
|
|
|
@noindent
|
|
and reports the uses of the symbols:
|
|
|
|
@example
|
|
Terminals, with rules where they appear
|
|
|
|
$end (0) 0
|
|
'*' (42) 3
|
|
'+' (43) 1
|
|
'-' (45) 2
|
|
'/' (47) 4
|
|
error (256)
|
|
NUM (258) 5
|
|
|
|
Nonterminals, with rules where they appear
|
|
|
|
$accept (8)
|
|
on left: 0
|
|
exp (9)
|
|
on left: 1 2 3 4 5, on right: 0 1 2 3 4
|
|
@end example
|
|
|
|
@noindent
|
|
@cindex item
|
|
@cindex pointed rule
|
|
@cindex rule, pointed
|
|
Bison then proceeds onto the automaton itself, describing each state
|
|
with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
|
|
item is a production rule together with a point (marked by @samp{.})
|
|
that the input cursor.
|
|
|
|
@example
|
|
state 0
|
|
|
|
$accept -> . exp $ (rule 0)
|
|
|
|
NUM shift, and go to state 1
|
|
|
|
exp go to state 2
|
|
@end example
|
|
|
|
This reads as follows: ``state 0 corresponds to being at the very
|
|
beginning of the parsing, in the initial rule, right before the start
|
|
symbol (here, @code{exp}). When the parser returns to this state right
|
|
after having reduced a rule that produced an @code{exp}, the control
|
|
flow jumps to state 2. If there is no such transition on a nonterminal
|
|
symbol, and the look-ahead is a @code{NUM}, then this token is shifted on
|
|
the parse stack, and the control flow jumps to state 1. Any other
|
|
look-ahead triggers a syntax error.''
|
|
|
|
@cindex core, item set
|
|
@cindex item set core
|
|
@cindex kernel, item set
|
|
@cindex item set core
|
|
Even though the only active rule in state 0 seems to be rule 0, the
|
|
report lists @code{NUM} as a look-ahead token because @code{NUM} can be
|
|
at the beginning of any rule deriving an @code{exp}. By default Bison
|
|
reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
|
|
you want to see more detail you can invoke @command{bison} with
|
|
@option{--report=itemset} to list all the items, include those that can
|
|
be derived:
|
|
|
|
@example
|
|
state 0
|
|
|
|
$accept -> . exp $ (rule 0)
|
|
exp -> . exp '+' exp (rule 1)
|
|
exp -> . exp '-' exp (rule 2)
|
|
exp -> . exp '*' exp (rule 3)
|
|
exp -> . exp '/' exp (rule 4)
|
|
exp -> . NUM (rule 5)
|
|
|
|
NUM shift, and go to state 1
|
|
|
|
exp go to state 2
|
|
@end example
|
|
|
|
@noindent
|
|
In the state 1...
|
|
|
|
@example
|
|
state 1
|
|
|
|
exp -> NUM . (rule 5)
|
|
|
|
$default reduce using rule 5 (exp)
|
|
@end example
|
|
|
|
@noindent
|
|
the rule 5, @samp{exp: NUM;}, is completed. Whatever the look-ahead token
|
|
(@samp{$default}), the parser will reduce it. If it was coming from
|
|
state 0, then, after this reduction it will return to state 0, and will
|
|
jump to state 2 (@samp{exp: go to state 2}).
|
|
|
|
@example
|
|
state 2
|
|
|
|
$accept -> exp . $ (rule 0)
|
|
exp -> exp . '+' exp (rule 1)
|
|
exp -> exp . '-' exp (rule 2)
|
|
exp -> exp . '*' exp (rule 3)
|
|
exp -> exp . '/' exp (rule 4)
|
|
|
|
$ shift, and go to state 3
|
|
'+' shift, and go to state 4
|
|
'-' shift, and go to state 5
|
|
'*' shift, and go to state 6
|
|
'/' shift, and go to state 7
|
|
@end example
|
|
|
|
@noindent
|
|
In state 2, the automaton can only shift a symbol. For instance,
|
|
because of the item @samp{exp -> exp . '+' exp}, if the look-ahead if
|
|
@samp{+}, it will be shifted on the parse stack, and the automaton
|
|
control will jump to state 4, corresponding to the item @samp{exp -> exp
|
|
'+' . exp}. Since there is no default action, any other token than
|
|
those listed above will trigger a syntax error.
|
|
|
|
The state 3 is named the @dfn{final state}, or the @dfn{accepting
|
|
state}:
|
|
|
|
@example
|
|
state 3
|
|
|
|
$accept -> exp $ . (rule 0)
|
|
|
|
$default accept
|
|
@end example
|
|
|
|
@noindent
|
|
the initial rule is completed (the start symbol and the end
|
|
of input were read), the parsing exits successfully.
|
|
|
|
The interpretation of states 4 to 7 is straightforward, and is left to
|
|
the reader.
|
|
|
|
@example
|
|
state 4
|
|
|
|
exp -> exp '+' . exp (rule 1)
|
|
|
|
NUM shift, and go to state 1
|
|
|
|
exp go to state 8
|
|
|
|
state 5
|
|
|
|
exp -> exp '-' . exp (rule 2)
|
|
|
|
NUM shift, and go to state 1
|
|
|
|
exp go to state 9
|
|
|
|
state 6
|
|
|
|
exp -> exp '*' . exp (rule 3)
|
|
|
|
NUM shift, and go to state 1
|
|
|
|
exp go to state 10
|
|
|
|
state 7
|
|
|
|
exp -> exp '/' . exp (rule 4)
|
|
|
|
NUM shift, and go to state 1
|
|
|
|
exp go to state 11
|
|
@end example
|
|
|
|
As was announced in beginning of the report, @samp{State 8 conflicts:
|
|
1 shift/reduce}:
|
|
|
|
@example
|
|
state 8
|
|
|
|
exp -> exp . '+' exp (rule 1)
|
|
exp -> exp '+' exp . (rule 1)
|
|
exp -> exp . '-' exp (rule 2)
|
|
exp -> exp . '*' exp (rule 3)
|
|
exp -> exp . '/' exp (rule 4)
|
|
|
|
'*' shift, and go to state 6
|
|
'/' shift, and go to state 7
|
|
|
|
'/' [reduce using rule 1 (exp)]
|
|
$default reduce using rule 1 (exp)
|
|
@end example
|
|
|
|
Indeed, there are two actions associated to the look-ahead @samp{/}:
|
|
either shifting (and going to state 7), or reducing rule 1. The
|
|
conflict means that either the grammar is ambiguous, or the parser lacks
|
|
information to make the right decision. Indeed the grammar is
|
|
ambiguous, as, since we did not specify the precedence of @samp{/}, the
|
|
sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
|
|
NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
|
|
NUM}, which corresponds to reducing rule 1.
|
|
|
|
Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
|
|
arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
|
|
Shift/Reduce Conflicts}. Discarded actions are reported in between
|
|
square brackets.
|
|
|
|
Note that all the previous states had a single possible action: either
|
|
shifting the next token and going to the corresponding state, or
|
|
reducing a single rule. In the other cases, i.e., when shifting
|
|
@emph{and} reducing is possible or when @emph{several} reductions are
|
|
possible, the look-ahead is required to select the action. State 8 is
|
|
one such state: if the look-ahead is @samp{*} or @samp{/} then the action
|
|
is shifting, otherwise the action is reducing rule 1. In other words,
|
|
the first two items, corresponding to rule 1, are not eligible when the
|
|
look-ahead token is @samp{*}, since we specified that @samp{*} has higher
|
|
precedence than @samp{+}. More generally, some items are eligible only
|
|
with some set of possible look-ahead tokens. When run with
|
|
@option{--report=look-ahead}, Bison specifies these look-ahead tokens:
|
|
|
|
@example
|
|
state 8
|
|
|
|
exp -> exp . '+' exp [$, '+', '-', '/'] (rule 1)
|
|
exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
|
|
exp -> exp . '-' exp (rule 2)
|
|
exp -> exp . '*' exp (rule 3)
|
|
exp -> exp . '/' exp (rule 4)
|
|
|
|
'*' shift, and go to state 6
|
|
'/' shift, and go to state 7
|
|
|
|
'/' [reduce using rule 1 (exp)]
|
|
$default reduce using rule 1 (exp)
|
|
@end example
|
|
|
|
The remaining states are similar:
|
|
|
|
@example
|
|
state 9
|
|
|
|
exp -> exp . '+' exp (rule 1)
|
|
exp -> exp . '-' exp (rule 2)
|
|
exp -> exp '-' exp . (rule 2)
|
|
exp -> exp . '*' exp (rule 3)
|
|
exp -> exp . '/' exp (rule 4)
|
|
|
|
'*' shift, and go to state 6
|
|
'/' shift, and go to state 7
|
|
|
|
'/' [reduce using rule 2 (exp)]
|
|
$default reduce using rule 2 (exp)
|
|
|
|
state 10
|
|
|
|
exp -> exp . '+' exp (rule 1)
|
|
exp -> exp . '-' exp (rule 2)
|
|
exp -> exp . '*' exp (rule 3)
|
|
exp -> exp '*' exp . (rule 3)
|
|
exp -> exp . '/' exp (rule 4)
|
|
|
|
'/' shift, and go to state 7
|
|
|
|
'/' [reduce using rule 3 (exp)]
|
|
$default reduce using rule 3 (exp)
|
|
|
|
state 11
|
|
|
|
exp -> exp . '+' exp (rule 1)
|
|
exp -> exp . '-' exp (rule 2)
|
|
exp -> exp . '*' exp (rule 3)
|
|
exp -> exp . '/' exp (rule 4)
|
|
exp -> exp '/' exp . (rule 4)
|
|
|
|
'+' shift, and go to state 4
|
|
'-' shift, and go to state 5
|
|
'*' shift, and go to state 6
|
|
'/' shift, and go to state 7
|
|
|
|
'+' [reduce using rule 4 (exp)]
|
|
'-' [reduce using rule 4 (exp)]
|
|
'*' [reduce using rule 4 (exp)]
|
|
'/' [reduce using rule 4 (exp)]
|
|
$default reduce using rule 4 (exp)
|
|
@end example
|
|
|
|
@noindent
|
|
Observe that state 11 contains conflicts due to the lack of precedence
|
|
of @samp{/} wrt @samp{+}, @samp{-}, and @samp{*}, but also because the
|
|
associativity of @samp{/} is not specified.
|
|
|
|
|
|
@node Tracing
|
|
@section Tracing Your Parser
|
|
@findex yydebug
|
|
@cindex debugging
|
|
@cindex tracing the parser
|
|
|
|
If a Bison grammar compiles properly but doesn't do what you want when it
|
|
runs, the @code{yydebug} parser-trace feature can help you figure out why.
|
|
|
|
There are several means to enable compilation of trace facilities:
|
|
|
|
@table @asis
|
|
@item the macro @code{YYDEBUG}
|
|
@findex YYDEBUG
|
|
Define the macro @code{YYDEBUG} to a nonzero value when you compile the
|
|
parser. This is compliant with @acronym{POSIX} Yacc. You could use
|
|
@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
|
|
YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
|
|
Prologue}).
|
|
|
|
@item the option @option{-t}, @option{--debug}
|
|
Use the @samp{-t} option when you run Bison (@pxref{Invocation,
|
|
,Invoking Bison}). This is @acronym{POSIX} compliant too.
|
|
|
|
@item the directive @samp{%debug}
|
|
@findex %debug
|
|
Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
|
|
Declaration Summary}). This is a Bison extension, which will prove
|
|
useful when Bison will output parsers for languages that don't use a
|
|
preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
|
|
you, this is
|
|
the preferred solution.
|
|
@end table
|
|
|
|
We suggest that you always enable the debug option so that debugging is
|
|
always possible.
|
|
|
|
The trace facility outputs messages with macro calls of the form
|
|
@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
|
|
@var{format} and @var{args} are the usual @code{printf} format and
|
|
arguments. If you define @code{YYDEBUG} to a nonzero value but do not
|
|
define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
|
|
and @code{YYPRINTF} is defined to @code{fprintf}.
|
|
|
|
Once you have compiled the program with trace facilities, the way to
|
|
request a trace is to store a nonzero value in the variable @code{yydebug}.
|
|
You can do this by making the C code do it (in @code{main}, perhaps), or
|
|
you can alter the value with a C debugger.
|
|
|
|
Each step taken by the parser when @code{yydebug} is nonzero produces a
|
|
line or two of trace information, written on @code{stderr}. The trace
|
|
messages tell you these things:
|
|
|
|
@itemize @bullet
|
|
@item
|
|
Each time the parser calls @code{yylex}, what kind of token was read.
|
|
|
|
@item
|
|
Each time a token is shifted, the depth and complete contents of the
|
|
state stack (@pxref{Parser States}).
|
|
|
|
@item
|
|
Each time a rule is reduced, which rule it is, and the complete contents
|
|
of the state stack afterward.
|
|
@end itemize
|
|
|
|
To make sense of this information, it helps to refer to the listing file
|
|
produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
|
|
Bison}). This file shows the meaning of each state in terms of
|
|
positions in various rules, and also what each state will do with each
|
|
possible input token. As you read the successive trace messages, you
|
|
can see that the parser is functioning according to its specification in
|
|
the listing file. Eventually you will arrive at the place where
|
|
something undesirable happens, and you will see which parts of the
|
|
grammar are to blame.
|
|
|
|
The parser file is a C program and you can use C debuggers on it, but it's
|
|
not easy to interpret what it is doing. The parser function is a
|
|
finite-state machine interpreter, and aside from the actions it executes
|
|
the same code over and over. Only the values of variables show where in
|
|
the grammar it is working.
|
|
|
|
@findex YYPRINT
|
|
The debugging information normally gives the token type of each token
|
|
read, but not its semantic value. You can optionally define a macro
|
|
named @code{YYPRINT} to provide a way to print the value. If you define
|
|
@code{YYPRINT}, it should take three arguments. The parser will pass a
|
|
standard I/O stream, the numeric code for the token type, and the token
|
|
value (from @code{yylval}).
|
|
|
|
Here is an example of @code{YYPRINT} suitable for the multi-function
|
|
calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}):
|
|
|
|
@smallexample
|
|
%@{
|
|
static void print_token_value (FILE *, int, YYSTYPE);
|
|
#define YYPRINT(file, type, value) print_token_value (file, type, value)
|
|
%@}
|
|
|
|
@dots{} %% @dots{} %% @dots{}
|
|
|
|
static void
|
|
print_token_value (FILE *file, int type, YYSTYPE value)
|
|
@{
|
|
if (type == VAR)
|
|
fprintf (file, "%s", value.tptr->name);
|
|
else if (type == NUM)
|
|
fprintf (file, "%d", value.val);
|
|
@}
|
|
@end smallexample
|
|
|
|
@c ================================================= Invoking Bison
|
|
|
|
@node Invocation
|
|
@chapter Invoking Bison
|
|
@cindex invoking Bison
|
|
@cindex Bison invocation
|
|
@cindex options for invoking Bison
|
|
|
|
The usual way to invoke Bison is as follows:
|
|
|
|
@example
|
|
bison @var{infile}
|
|
@end example
|
|
|
|
Here @var{infile} is the grammar file name, which usually ends in
|
|
@samp{.y}. The parser file's name is made by replacing the @samp{.y}
|
|
with @samp{.tab.c}. Thus, the @samp{bison foo.y} filename yields
|
|
@file{foo.tab.c}, and the @samp{bison hack/foo.y} filename yields
|
|
@file{hack/foo.tab.c}. It's also possible, in case you are writing
|
|
C++ code instead of C in your grammar file, to name it @file{foo.ypp}
|
|
or @file{foo.y++}. Then, the output files will take an extension like
|
|
the given one as input (respectively @file{foo.tab.cpp} and
|
|
@file{foo.tab.c++}).
|
|
This feature takes effect with all options that manipulate filenames like
|
|
@samp{-o} or @samp{-d}.
|
|
|
|
For example :
|
|
|
|
@example
|
|
bison -d @var{infile.yxx}
|
|
@end example
|
|
@noindent
|
|
will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
|
|
|
|
@example
|
|
bison -d -o @var{output.c++} @var{infile.y}
|
|
@end example
|
|
@noindent
|
|
will produce @file{output.c++} and @file{outfile.h++}.
|
|
|
|
For compatibility with @acronym{POSIX}, the standard Bison
|
|
distribution also contains a shell script called @command{yacc} that
|
|
invokes Bison with the @option{-y} option.
|
|
|
|
@menu
|
|
* Bison Options:: All the options described in detail,
|
|
in alphabetical order by short options.
|
|
* Option Cross Key:: Alphabetical list of long options.
|
|
* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
|
|
@end menu
|
|
|
|
@node Bison Options
|
|
@section Bison Options
|
|
|
|
Bison supports both traditional single-letter options and mnemonic long
|
|
option names. Long option names are indicated with @samp{--} instead of
|
|
@samp{-}. Abbreviations for option names are allowed as long as they
|
|
are unique. When a long option takes an argument, like
|
|
@samp{--file-prefix}, connect the option name and the argument with
|
|
@samp{=}.
|
|
|
|
Here is a list of options that can be used with Bison, alphabetized by
|
|
short option. It is followed by a cross key alphabetized by long
|
|
option.
|
|
|
|
@c Please, keep this ordered as in `bison --help'.
|
|
@noindent
|
|
Operations modes:
|
|
@table @option
|
|
@item -h
|
|
@itemx --help
|
|
Print a summary of the command-line options to Bison and exit.
|
|
|
|
@item -V
|
|
@itemx --version
|
|
Print the version number of Bison and exit.
|
|
|
|
@need 1750
|
|
@item -y
|
|
@itemx --yacc
|
|
Equivalent to @samp{-o y.tab.c}; the parser output file is called
|
|
@file{y.tab.c}, and the other outputs are called @file{y.output} and
|
|
@file{y.tab.h}. The purpose of this option is to imitate Yacc's output
|
|
file name conventions. Thus, the following shell script can substitute
|
|
for Yacc, and the Bison distribution contains such a script for
|
|
compatibility with @acronym{POSIX}:
|
|
|
|
@example
|
|
#! /bin/sh
|
|
bison -y "$@@"
|
|
@end example
|
|
@end table
|
|
|
|
@noindent
|
|
Tuning the parser:
|
|
|
|
@table @option
|
|
@item -S @var{file}
|
|
@itemx --skeleton=@var{file}
|
|
Specify the skeleton to use. You probably don't need this option unless
|
|
you are developing Bison.
|
|
|
|
@item -t
|
|
@itemx --debug
|
|
In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
|
|
already defined, so that the debugging facilities are compiled.
|
|
@xref{Tracing, ,Tracing Your Parser}.
|
|
|
|
@item --locations
|
|
Pretend that @code{%locations} was specified. @xref{Decl Summary}.
|
|
|
|
@item -p @var{prefix}
|
|
@itemx --name-prefix=@var{prefix}
|
|
Pretend that @code{%name-prefix="@var{prefix}"} was specified.
|
|
@xref{Decl Summary}.
|
|
|
|
@item -l
|
|
@itemx --no-lines
|
|
Don't put any @code{#line} preprocessor commands in the parser file.
|
|
Ordinarily Bison puts them in the parser file so that the C compiler
|
|
and debuggers will associate errors with your source file, the
|
|
grammar file. This option causes them to associate errors with the
|
|
parser file, treating it as an independent source file in its own right.
|
|
|
|
@item -n
|
|
@itemx --no-parser
|
|
Pretend that @code{%no-parser} was specified. @xref{Decl Summary}.
|
|
|
|
@item -k
|
|
@itemx --token-table
|
|
Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
|
|
@end table
|
|
|
|
@noindent
|
|
Adjust the output:
|
|
|
|
@table @option
|
|
@item -d
|
|
@itemx --defines
|
|
Pretend that @code{%defines} was specified, i.e., write an extra output
|
|
file containing macro definitions for the token type names defined in
|
|
the grammar, as well as a few other declarations. @xref{Decl Summary}.
|
|
|
|
@item --defines=@var{defines-file}
|
|
Same as above, but save in the file @var{defines-file}.
|
|
|
|
@item -b @var{file-prefix}
|
|
@itemx --file-prefix=@var{prefix}
|
|
Pretend that @code{%verbose} was specified, i.e, specify prefix to use
|
|
for all Bison output file names. @xref{Decl Summary}.
|
|
|
|
@item -r @var{things}
|
|
@itemx --report=@var{things}
|
|
Write an extra output file containing verbose description of the comma
|
|
separated list of @var{things} among:
|
|
|
|
@table @code
|
|
@item state
|
|
Description of the grammar, conflicts (resolved and unresolved), and
|
|
@acronym{LALR} automaton.
|
|
|
|
@item look-ahead
|
|
Implies @code{state} and augments the description of the automaton with
|
|
each rule's look-ahead set.
|
|
|
|
@item itemset
|
|
Implies @code{state} and augments the description of the automaton with
|
|
the full set of items for each state, instead of its core only.
|
|
@end table
|
|
|
|
For instance, on the following grammar
|
|
|
|
@item -v
|
|
@itemx --verbose
|
|
Pretend that @code{%verbose} was specified, i.e, write an extra output
|
|
file containing verbose descriptions of the grammar and
|
|
parser. @xref{Decl Summary}.
|
|
|
|
@item -o @var{filename}
|
|
@itemx --output=@var{filename}
|
|
Specify the @var{filename} for the parser file.
|
|
|
|
The other output files' names are constructed from @var{filename} as
|
|
described under the @samp{-v} and @samp{-d} options.
|
|
|
|
@item -g
|
|
Output a @acronym{VCG} definition of the @acronym{LALR}(1) grammar
|
|
automaton computed by Bison. If the grammar file is @file{foo.y}, the
|
|
@acronym{VCG} output file will
|
|
be @file{foo.vcg}.
|
|
|
|
@item --graph=@var{graph-file}
|
|
The behavior of @var{--graph} is the same than @samp{-g}. The only
|
|
difference is that it has an optional argument which is the name of
|
|
the output graph filename.
|
|
@end table
|
|
|
|
@node Option Cross Key
|
|
@section Option Cross Key
|
|
|
|
Here is a list of options, alphabetized by long option, to help you find
|
|
the corresponding short option.
|
|
|
|
@tex
|
|
\def\leaderfill{\leaders\hbox to 1em{\hss.\hss}\hfill}
|
|
|
|
{\tt
|
|
\line{ --debug \leaderfill -t}
|
|
\line{ --defines \leaderfill -d}
|
|
\line{ --file-prefix \leaderfill -b}
|
|
\line{ --graph \leaderfill -g}
|
|
\line{ --help \leaderfill -h}
|
|
\line{ --name-prefix \leaderfill -p}
|
|
\line{ --no-lines \leaderfill -l}
|
|
\line{ --no-parser \leaderfill -n}
|
|
\line{ --output \leaderfill -o}
|
|
\line{ --token-table \leaderfill -k}
|
|
\line{ --verbose \leaderfill -v}
|
|
\line{ --version \leaderfill -V}
|
|
\line{ --yacc \leaderfill -y}
|
|
}
|
|
@end tex
|
|
|
|
@ifinfo
|
|
@example
|
|
--debug -t
|
|
--defines=@var{defines-file} -d
|
|
--file-prefix=@var{prefix} -b @var{file-prefix}
|
|
--graph=@var{graph-file} -d
|
|
--help -h
|
|
--name-prefix=@var{prefix} -p @var{name-prefix}
|
|
--no-lines -l
|
|
--no-parser -n
|
|
--output=@var{outfile} -o @var{outfile}
|
|
--token-table -k
|
|
--verbose -v
|
|
--version -V
|
|
--yacc -y
|
|
@end example
|
|
@end ifinfo
|
|
|
|
@node Yacc Library
|
|
@section Yacc Library
|
|
|
|
The Yacc library contains default implementations of the
|
|
@code{yyerror} and @code{main} functions. These default
|
|
implementations are normally not useful, but @acronym{POSIX} requires
|
|
them. To use the Yacc library, link your program with the
|
|
@option{-ly} option. Note that Bison's implementation of the Yacc
|
|
library is distributed under the terms of the @acronym{GNU} General
|
|
Public License (@pxref{Copying}).
|
|
|
|
If you use the Yacc library's @code{yyerror} function, you should
|
|
declare @code{yyerror} as follows:
|
|
|
|
@example
|
|
int yyerror (char const *);
|
|
@end example
|
|
|
|
Bison ignores the @code{int} value returned by this @code{yyerror}.
|
|
If you use the Yacc library's @code{main} function, your
|
|
@code{yyparse} function should have the following type signature:
|
|
|
|
@example
|
|
int yyparse (void);
|
|
@end example
|
|
|
|
@c ================================================= Invoking Bison
|
|
|
|
@node FAQ
|
|
@chapter Frequently Asked Questions
|
|
@cindex frequently asked questions
|
|
@cindex questions
|
|
|
|
Several questions about Bison come up occasionally. Here some of them
|
|
are addressed.
|
|
|
|
@menu
|
|
* Parser Stack Overflow:: Breaking the Stack Limits
|
|
* How Can I Reset the Parser:: @code{yyparse} Keeps some State
|
|
* Strings are Destroyed:: @code{yylval} Loses Track of Strings
|
|
* C++ Parsers:: Compiling Parsers with C++ Compilers
|
|
* Implementing Loops:: Control Flow in the Calculator
|
|
@end menu
|
|
|
|
@node Parser Stack Overflow
|
|
@section Parser Stack Overflow
|
|
|
|
@display
|
|
My parser returns with error with a @samp{parser stack overflow}
|
|
message. What can I do?
|
|
@end display
|
|
|
|
This question is already addressed elsewhere, @xref{Recursion,
|
|
,Recursive Rules}.
|
|
|
|
@node How Can I Reset the Parser
|
|
@section How Can I Reset the Parser
|
|
|
|
The following phenomenon has several symptoms, resulting in the
|
|
following typical questions:
|
|
|
|
@display
|
|
I invoke @code{yyparse} several times, and on correct input it works
|
|
properly; but when a parse error is found, all the other calls fail
|
|
too. How can I reset the error flag of @code{yyparse}?
|
|
@end display
|
|
|
|
@noindent
|
|
or
|
|
|
|
@display
|
|
My parser includes support for an @samp{#include}-like feature, in
|
|
which case I run @code{yyparse} from @code{yyparse}. This fails
|
|
although I did specify I needed a @code{%pure-parser}.
|
|
@end display
|
|
|
|
These problems typically come not from Bison itself, but from
|
|
Lex-generated scanners. Because these scanners use large buffers for
|
|
speed, they might not notice a change of input file. As a
|
|
demonstration, consider the following source file,
|
|
@file{first-line.l}:
|
|
|
|
@verbatim
|
|
%{
|
|
#include <stdio.h>
|
|
#include <stdlib.h>
|
|
%}
|
|
%%
|
|
.*\n ECHO; return 1;
|
|
%%
|
|
int
|
|
yyparse (char const *file)
|
|
{
|
|
yyin = fopen (file, "r");
|
|
if (!yyin)
|
|
exit (2);
|
|
/* One token only. */
|
|
yylex ();
|
|
if (fclose (yyin) != 0)
|
|
exit (3);
|
|
return 0;
|
|
}
|
|
|
|
int
|
|
main (void)
|
|
{
|
|
yyparse ("input");
|
|
yyparse ("input");
|
|
return 0;
|
|
}
|
|
@end verbatim
|
|
|
|
@noindent
|
|
If the file @file{input} contains
|
|
|
|
@verbatim
|
|
input:1: Hello,
|
|
input:2: World!
|
|
@end verbatim
|
|
|
|
@noindent
|
|
then instead of getting the first line twice, you get:
|
|
|
|
@example
|
|
$ @kbd{flex -ofirst-line.c first-line.l}
|
|
$ @kbd{gcc -ofirst-line first-line.c -ll}
|
|
$ @kbd{./first-line}
|
|
input:1: Hello,
|
|
input:2: World!
|
|
@end example
|
|
|
|
Therefore, whenever you change @code{yyin}, you must tell the
|
|
Lex-generated scanner to discard its current buffer and switch to the
|
|
new one. This depends upon your implementation of Lex; see its
|
|
documentation for more. For Flex, it suffices to call
|
|
@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
|
|
Flex-generated scanner needs to read from several input streams to
|
|
handle features like include files, you might consider using Flex
|
|
functions like @samp{yy_switch_to_buffer} that manipulate multiple
|
|
input buffers.
|
|
|
|
If your Flex-generated scanner uses start conditions (@pxref{Start
|
|
conditions, , Start conditions, flex, The Flex Manual}), you might
|
|
also want to reset the scanner's state, i.e., go back to the initial
|
|
start condition, through a call to @samp{BEGIN (0)}.
|
|
|
|
@node Strings are Destroyed
|
|
@section Strings are Destroyed
|
|
|
|
@display
|
|
My parser seems to destroy old strings, or maybe it loses track of
|
|
them. Instead of reporting @samp{"foo", "bar"}, it reports
|
|
@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
|
|
@end display
|
|
|
|
This error is probably the single most frequent ``bug report'' sent to
|
|
Bison lists, but is only concerned with a misunderstanding of the role
|
|
of scanner. Consider the following Lex code:
|
|
|
|
@verbatim
|
|
%{
|
|
#include <stdio.h>
|
|
char *yylval = NULL;
|
|
%}
|
|
%%
|
|
.* yylval = yytext; return 1;
|
|
\n /* IGNORE */
|
|
%%
|
|
int
|
|
main ()
|
|
{
|
|
/* Similar to using $1, $2 in a Bison action. */
|
|
char *fst = (yylex (), yylval);
|
|
char *snd = (yylex (), yylval);
|
|
printf ("\"%s\", \"%s\"\n", fst, snd);
|
|
return 0;
|
|
}
|
|
@end verbatim
|
|
|
|
If you compile and run this code, you get:
|
|
|
|
@example
|
|
$ @kbd{flex -osplit-lines.c split-lines.l}
|
|
$ @kbd{gcc -osplit-lines split-lines.c -ll}
|
|
$ @kbd{printf 'one\ntwo\n' | ./split-lines}
|
|
"one
|
|
two", "two"
|
|
@end example
|
|
|
|
@noindent
|
|
this is because @code{yytext} is a buffer provided for @emph{reading}
|
|
in the action, but if you want to keep it, you have to duplicate it
|
|
(e.g., using @code{strdup}). Note that the output may depend on how
|
|
your implementation of Lex handles @code{yytext}. For instance, when
|
|
given the Lex compatibility option @option{-l} (which triggers the
|
|
option @samp{%array}) Flex generates a different behavior:
|
|
|
|
@example
|
|
$ @kbd{flex -l -osplit-lines.c split-lines.l}
|
|
$ @kbd{gcc -osplit-lines split-lines.c -ll}
|
|
$ @kbd{printf 'one\ntwo\n' | ./split-lines}
|
|
"two", "two"
|
|
@end example
|
|
|
|
|
|
@node C++ Parsers
|
|
@section C++ Parsers
|
|
|
|
@display
|
|
How can I generate parsers in C++?
|
|
@end display
|
|
|
|
We are working on a C++ output for Bison, but unfortunately, for lack
|
|
of time, the skeleton is not finished. It is functional, but in
|
|
numerous respects, it will require additional work which @emph{might}
|
|
break backward compatibility. Since the skeleton for C++ is not
|
|
documented, we do not consider ourselves bound to this interface,
|
|
nevertheless, as much as possible we will try to keep compatibility.
|
|
|
|
Another possibility is to use the regular C parsers, and to compile
|
|
them with a C++ compiler. This works properly, provided that you bear
|
|
some simple C++ rules in mind, such as not including ``real classes''
|
|
(i.e., structure with constructors) in unions. Therefore, in the
|
|
@code{%union}, use pointers to classes, or better yet, a single
|
|
pointer type to the root of your lexical/syntactic hierarchy.
|
|
|
|
|
|
@node Implementing Loops
|
|
@section Implementing Loops
|
|
|
|
@display
|
|
My simple calculator supports variables, assignments, and functions,
|
|
but how can I implement loops?
|
|
@end display
|
|
|
|
Although very pedagogical, the examples included in the document blur
|
|
the distinction to make between the parser---whose job is to recover
|
|
the structure of a text and to transmit it to subsequent modules of
|
|
the program---and the processing (such as the execution) of this
|
|
structure. This works well with so called straight line programs,
|
|
i.e., precisely those that have a straightforward execution model:
|
|
execute simple instructions one after the others.
|
|
|
|
@cindex abstract syntax tree
|
|
@cindex @acronym{AST}
|
|
If you want a richer model, you will probably need to use the parser
|
|
to construct a tree that does represent the structure it has
|
|
recovered; this tree is usually called the @dfn{abstract syntax tree},
|
|
or @dfn{@acronym{AST}} for short. Then, walking through this tree,
|
|
traversing it in various ways, will enable treatments such as its
|
|
execution or its translation, which will result in an interpreter or a
|
|
compiler.
|
|
|
|
This topic is way beyond the scope of this manual, and the reader is
|
|
invited to consult the dedicated literature.
|
|
|
|
|
|
|
|
@c ================================================= Table of Symbols
|
|
|
|
@node Table of Symbols
|
|
@appendix Bison Symbols
|
|
@cindex Bison symbols, table of
|
|
@cindex symbols in Bison, table of
|
|
|
|
@deffn {Variable} @@$
|
|
In an action, the location of the left-hand side of the rule.
|
|
@xref{Locations, , Locations Overview}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} @@@var{n}
|
|
In an action, the location of the @var{n}-th symbol of the right-hand
|
|
side of the rule. @xref{Locations, , Locations Overview}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} $$
|
|
In an action, the semantic value of the left-hand side of the rule.
|
|
@xref{Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} $@var{n}
|
|
In an action, the semantic value of the @var{n}-th symbol of the
|
|
right-hand side of the rule. @xref{Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Symbol} $accept
|
|
The predefined nonterminal whose only rule is @samp{$accept: @var{start}
|
|
$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
|
|
Start-Symbol}. It cannot be used in the grammar.
|
|
@end deffn
|
|
|
|
@deffn {Symbol} $end
|
|
The predefined token marking the end of the token stream. It cannot be
|
|
used in the grammar.
|
|
@end deffn
|
|
|
|
@deffn {Symbol} $undefined
|
|
The predefined token onto which all undefined values returned by
|
|
@code{yylex} are mapped. It cannot be used in the grammar, rather, use
|
|
@code{error}.
|
|
@end deffn
|
|
|
|
@deffn {Symbol} error
|
|
A token name reserved for error recovery. This token may be used in
|
|
grammar rules so as to allow the Bison parser to recognize an error in
|
|
the grammar without halting the process. In effect, a sentence
|
|
containing an error may be recognized as valid. On a syntax error, the
|
|
token @code{error} becomes the current look-ahead token. Actions
|
|
corresponding to @code{error} are then executed, and the look-ahead
|
|
token is reset to the token that originally caused the violation.
|
|
@xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYABORT
|
|
Macro to pretend that an unrecoverable syntax error has occurred, by
|
|
making @code{yyparse} return 1 immediately. The error reporting
|
|
function @code{yyerror} is not called. @xref{Parser Function, ,The
|
|
Parser Function @code{yyparse}}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYACCEPT
|
|
Macro to pretend that a complete utterance of the language has been
|
|
read, by making @code{yyparse} return 0 immediately.
|
|
@xref{Parser Function, ,The Parser Function @code{yyparse}}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYBACKUP
|
|
Macro to discard a value from the parser stack and fake a look-ahead
|
|
token. @xref{Action Features, ,Special Features for Use in Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYDEBUG
|
|
Macro to define to equip the parser with tracing code. @xref{Tracing,
|
|
,Tracing Your Parser}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYERROR
|
|
Macro to pretend that a syntax error has just been detected: call
|
|
@code{yyerror} and then perform normal error recovery if possible
|
|
(@pxref{Error Recovery}), or (if recovery is impossible) make
|
|
@code{yyparse} return 1. @xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYERROR_VERBOSE
|
|
An obsolete macro that you define with @code{#define} in the prologue
|
|
to request verbose, specific error message strings
|
|
when @code{yyerror} is called. It doesn't matter what definition you
|
|
use for @code{YYERROR_VERBOSE}, just whether you define it. Using
|
|
@code{%error-verbose} is preferred.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYINITDEPTH
|
|
Macro for specifying the initial size of the parser stack.
|
|
@xref{Stack Overflow}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYLEX_PARAM
|
|
An obsolete macro for specifying an extra argument (or list of extra
|
|
arguments) for @code{yyparse} to pass to @code{yylex}. he use of this
|
|
macro is deprecated, and is supported only for Yacc like parsers.
|
|
@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
|
|
@end deffn
|
|
|
|
@deffn {Type} YYLTYPE
|
|
Data type of @code{yylloc}; by default, a structure with four
|
|
members. @xref{Location Type, , Data Types of Locations}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYMAXDEPTH
|
|
Macro for specifying the maximum size of the parser stack. @xref{Stack
|
|
Overflow}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYPARSE_PARAM
|
|
An obsolete macro for specifying the name of a parameter that
|
|
@code{yyparse} should accept. The use of this macro is deprecated, and
|
|
is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
|
|
Conventions for Pure Parsers}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYRECOVERING
|
|
Macro whose value indicates whether the parser is recovering from a
|
|
syntax error. @xref{Action Features, ,Special Features for Use in Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} YYSTACK_USE_ALLOCA
|
|
Macro used to control the use of @code{alloca}. If defined to @samp{0},
|
|
the parser will not use @code{alloca} but @code{malloc} when trying to
|
|
grow its internal stacks. Do @emph{not} define @code{YYSTACK_USE_ALLOCA}
|
|
to anything else.
|
|
@end deffn
|
|
|
|
@deffn {Type} YYSTYPE
|
|
Data type of semantic values; @code{int} by default.
|
|
@xref{Value Type, ,Data Types of Semantic Values}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} yychar
|
|
External integer variable that contains the integer value of the current
|
|
look-ahead token. (In a pure parser, it is a local variable within
|
|
@code{yyparse}.) Error-recovery rule actions may examine this variable.
|
|
@xref{Action Features, ,Special Features for Use in Actions}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} yyclearin
|
|
Macro used in error-recovery rule actions. It clears the previous
|
|
look-ahead token. @xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} yydebug
|
|
External integer variable set to zero by default. If @code{yydebug}
|
|
is given a nonzero value, the parser will output information on input
|
|
symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
|
|
@end deffn
|
|
|
|
@deffn {Macro} yyerrok
|
|
Macro to cause parser to recover immediately to its normal mode
|
|
after a syntax error. @xref{Error Recovery}.
|
|
@end deffn
|
|
|
|
@deffn {Function} yyerror
|
|
User-supplied function to be called by @code{yyparse} on error.
|
|
@xref{Error Reporting, ,The Error
|
|
Reporting Function @code{yyerror}}.
|
|
@end deffn
|
|
|
|
@deffn {Function} yylex
|
|
User-supplied lexical analyzer function, called with no arguments to get
|
|
the next token. @xref{Lexical, ,The Lexical Analyzer Function
|
|
@code{yylex}}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} yylval
|
|
External variable in which @code{yylex} should place the semantic
|
|
value associated with a token. (In a pure parser, it is a local
|
|
variable within @code{yyparse}, and its address is passed to
|
|
@code{yylex}.) @xref{Token Values, ,Semantic Values of Tokens}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} yylloc
|
|
External variable in which @code{yylex} should place the line and column
|
|
numbers associated with a token. (In a pure parser, it is a local
|
|
variable within @code{yyparse}, and its address is passed to
|
|
@code{yylex}.) You can ignore this variable if you don't use the
|
|
@samp{@@} feature in the grammar actions. @xref{Token Locations,
|
|
,Textual Locations of Tokens}.
|
|
@end deffn
|
|
|
|
@deffn {Variable} yynerrs
|
|
Global variable which Bison increments each time there is a syntax error.
|
|
(In a pure parser, it is a local variable within @code{yyparse}.)
|
|
@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
|
|
@end deffn
|
|
|
|
@deffn {Function} yyparse
|
|
The parser function produced by Bison; call this function to start
|
|
parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %debug
|
|
Equip the parser for debugging. @xref{Decl Summary}.
|
|
@end deffn
|
|
|
|
@ifset defaultprec
|
|
@deffn {Directive} %default-prec
|
|
Assign a precedence to rules that lack an explicit @samp{%prec}
|
|
modifier. @xref{Contextual Precedence, ,Context-Dependent
|
|
Precedence}.
|
|
@end deffn
|
|
@end ifset
|
|
|
|
@deffn {Directive} %defines
|
|
Bison declaration to create a header file meant for the scanner.
|
|
@xref{Decl Summary}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %destructor
|
|
Specifying how the parser should reclaim the memory associated to
|
|
discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %dprec
|
|
Bison declaration to assign a precedence to a rule that is used at parse
|
|
time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
|
|
@acronym{GLR} Parsers}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %error-verbose
|
|
Bison declaration to request verbose, specific error message strings
|
|
when @code{yyerror} is called.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %file-prefix="@var{prefix}"
|
|
Bison declaration to set the prefix of the output files. @xref{Decl
|
|
Summary}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %glr-parser
|
|
Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
|
|
Parsers, ,Writing @acronym{GLR} Parsers}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %left
|
|
Bison declaration to assign left associativity to token(s).
|
|
@xref{Precedence Decl, ,Operator Precedence}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %lex-param @{@var{argument-declaration}@}
|
|
Bison declaration to specifying an additional parameter that
|
|
@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
|
|
for Pure Parsers}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %merge
|
|
Bison declaration to assign a merging function to a rule. If there is a
|
|
reduce/reduce conflict with a rule having the same merging function, the
|
|
function is applied to the two semantic values to get a single result.
|
|
@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %name-prefix="@var{prefix}"
|
|
Bison declaration to rename the external symbols. @xref{Decl Summary}.
|
|
@end deffn
|
|
|
|
@ifset defaultprec
|
|
@deffn {Directive} %no-default-prec
|
|
Do not assign a precedence to rules that lack an explicit @samp{%prec}
|
|
modifier. @xref{Contextual Precedence, ,Context-Dependent
|
|
Precedence}.
|
|
@end deffn
|
|
@end ifset
|
|
|
|
@deffn {Directive} %no-lines
|
|
Bison declaration to avoid generating @code{#line} directives in the
|
|
parser file. @xref{Decl Summary}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %nonassoc
|
|
Bison declaration to assign non-associativity to token(s).
|
|
@xref{Precedence Decl, ,Operator Precedence}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %output="@var{filename}"
|
|
Bison declaration to set the name of the parser file. @xref{Decl
|
|
Summary}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %parse-param @{@var{argument-declaration}@}
|
|
Bison declaration to specifying an additional parameter that
|
|
@code{yyparse} should accept. @xref{Parser Function,, The Parser
|
|
Function @code{yyparse}}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %prec
|
|
Bison declaration to assign a precedence to a specific rule.
|
|
@xref{Contextual Precedence, ,Context-Dependent Precedence}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %pure-parser
|
|
Bison declaration to request a pure (reentrant) parser.
|
|
@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %right
|
|
Bison declaration to assign right associativity to token(s).
|
|
@xref{Precedence Decl, ,Operator Precedence}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %start
|
|
Bison declaration to specify the start symbol. @xref{Start Decl, ,The
|
|
Start-Symbol}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %token
|
|
Bison declaration to declare token(s) without specifying precedence.
|
|
@xref{Token Decl, ,Token Type Names}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %token-table
|
|
Bison declaration to include a token name table in the parser file.
|
|
@xref{Decl Summary}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %type
|
|
Bison declaration to declare nonterminals. @xref{Type Decl,
|
|
,Nonterminal Symbols}.
|
|
@end deffn
|
|
|
|
@deffn {Directive} %union
|
|
Bison declaration to specify several possible data types for semantic
|
|
values. @xref{Union Decl, ,The Collection of Value Types}.
|
|
@end deffn
|
|
|
|
@sp 1
|
|
|
|
These are the punctuation and delimiters used in Bison input:
|
|
|
|
@deffn {Delimiter} %%
|
|
Delimiter used to separate the grammar rule section from the
|
|
Bison declarations section or the epilogue.
|
|
@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
|
|
@end deffn
|
|
|
|
@c Don't insert spaces, or check the DVI output.
|
|
@deffn {Delimiter} %@{@var{code}%@}
|
|
All code listed between @samp{%@{} and @samp{%@}} is copied directly to
|
|
the output file uninterpreted. Such code forms the prologue of the input
|
|
file. @xref{Grammar Outline, ,Outline of a Bison
|
|
Grammar}.
|
|
@end deffn
|
|
|
|
@deffn {Construct} /*@dots{}*/
|
|
Comment delimiters, as in C.
|
|
@end deffn
|
|
|
|
@deffn {Delimiter} :
|
|
Separates a rule's result from its components. @xref{Rules, ,Syntax of
|
|
Grammar Rules}.
|
|
@end deffn
|
|
|
|
@deffn {Delimiter} ;
|
|
Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
|
|
@end deffn
|
|
|
|
@deffn {Delimiter} |
|
|
Separates alternate rules for the same result nonterminal.
|
|
@xref{Rules, ,Syntax of Grammar Rules}.
|
|
@end deffn
|
|
|
|
@node Glossary
|
|
@appendix Glossary
|
|
@cindex glossary
|
|
|
|
@table @asis
|
|
@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
|
|
Formal method of specifying context-free grammars originally proposed
|
|
by John Backus, and slightly improved by Peter Naur in his 1960-01-02
|
|
committee document contributing to what became the Algol 60 report.
|
|
@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
|
|
|
|
@item Context-free grammars
|
|
Grammars specified as rules that can be applied regardless of context.
|
|
Thus, if there is a rule which says that an integer can be used as an
|
|
expression, integers are allowed @emph{anywhere} an expression is
|
|
permitted. @xref{Language and Grammar, ,Languages and Context-Free
|
|
Grammars}.
|
|
|
|
@item Dynamic allocation
|
|
Allocation of memory that occurs during execution, rather than at
|
|
compile time or on entry to a function.
|
|
|
|
@item Empty string
|
|
Analogous to the empty set in set theory, the empty string is a
|
|
character string of length zero.
|
|
|
|
@item Finite-state stack machine
|
|
A ``machine'' that has discrete states in which it is said to exist at
|
|
each instant in time. As input to the machine is processed, the
|
|
machine moves from state to state as specified by the logic of the
|
|
machine. In the case of the parser, the input is the language being
|
|
parsed, and the states correspond to various stages in the grammar
|
|
rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
|
|
|
|
@item Generalized @acronym{LR} (@acronym{GLR})
|
|
A parsing algorithm that can handle all context-free grammars, including those
|
|
that are not @acronym{LALR}(1). It resolves situations that Bison's
|
|
usual @acronym{LALR}(1)
|
|
algorithm cannot by effectively splitting off multiple parsers, trying all
|
|
possible parsers, and discarding those that fail in the light of additional
|
|
right context. @xref{Generalized LR Parsing, ,Generalized
|
|
@acronym{LR} Parsing}.
|
|
|
|
@item Grouping
|
|
A language construct that is (in general) grammatically divisible;
|
|
for example, `expression' or `declaration' in C@.
|
|
@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
|
|
|
|
@item Infix operator
|
|
An arithmetic operator that is placed between the operands on which it
|
|
performs some operation.
|
|
|
|
@item Input stream
|
|
A continuous flow of data between devices or programs.
|
|
|
|
@item Language construct
|
|
One of the typical usage schemas of the language. For example, one of
|
|
the constructs of the C language is the @code{if} statement.
|
|
@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
|
|
|
|
@item Left associativity
|
|
Operators having left associativity are analyzed from left to right:
|
|
@samp{a+b+c} first computes @samp{a+b} and then combines with
|
|
@samp{c}. @xref{Precedence, ,Operator Precedence}.
|
|
|
|
@item Left recursion
|
|
A rule whose result symbol is also its first component symbol; for
|
|
example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
|
|
Rules}.
|
|
|
|
@item Left-to-right parsing
|
|
Parsing a sentence of a language by analyzing it token by token from
|
|
left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
|
|
|
|
@item Lexical analyzer (scanner)
|
|
A function that reads an input stream and returns tokens one by one.
|
|
@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
|
|
|
|
@item Lexical tie-in
|
|
A flag, set by actions in the grammar rules, which alters the way
|
|
tokens are parsed. @xref{Lexical Tie-ins}.
|
|
|
|
@item Literal string token
|
|
A token which consists of two or more fixed characters. @xref{Symbols}.
|
|
|
|
@item Look-ahead token
|
|
A token already read but not yet shifted. @xref{Look-Ahead, ,Look-Ahead
|
|
Tokens}.
|
|
|
|
@item @acronym{LALR}(1)
|
|
The class of context-free grammars that Bison (like most other parser
|
|
generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
|
|
Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
|
|
|
|
@item @acronym{LR}(1)
|
|
The class of context-free grammars in which at most one token of
|
|
look-ahead is needed to disambiguate the parsing of any piece of input.
|
|
|
|
@item Nonterminal symbol
|
|
A grammar symbol standing for a grammatical construct that can
|
|
be expressed through rules in terms of smaller constructs; in other
|
|
words, a construct that is not a token. @xref{Symbols}.
|
|
|
|
@item Parser
|
|
A function that recognizes valid sentences of a language by analyzing
|
|
the syntax structure of a set of tokens passed to it from a lexical
|
|
analyzer.
|
|
|
|
@item Postfix operator
|
|
An arithmetic operator that is placed after the operands upon which it
|
|
performs some operation.
|
|
|
|
@item Reduction
|
|
Replacing a string of nonterminals and/or terminals with a single
|
|
nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
|
|
Parser Algorithm}.
|
|
|
|
@item Reentrant
|
|
A reentrant subprogram is a subprogram which can be in invoked any
|
|
number of times in parallel, without interference between the various
|
|
invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
|
|
|
|
@item Reverse polish notation
|
|
A language in which all operators are postfix operators.
|
|
|
|
@item Right recursion
|
|
A rule whose result symbol is also its last component symbol; for
|
|
example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
|
|
Rules}.
|
|
|
|
@item Semantics
|
|
In computer languages, the semantics are specified by the actions
|
|
taken for each instance of the language, i.e., the meaning of
|
|
each statement. @xref{Semantics, ,Defining Language Semantics}.
|
|
|
|
@item Shift
|
|
A parser is said to shift when it makes the choice of analyzing
|
|
further input from the stream rather than reducing immediately some
|
|
already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
|
|
|
|
@item Single-character literal
|
|
A single character that is recognized and interpreted as is.
|
|
@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
|
|
|
|
@item Start symbol
|
|
The nonterminal symbol that stands for a complete valid utterance in
|
|
the language being parsed. The start symbol is usually listed as the
|
|
first nonterminal symbol in a language specification.
|
|
@xref{Start Decl, ,The Start-Symbol}.
|
|
|
|
@item Symbol table
|
|
A data structure where symbol names and associated data are stored
|
|
during parsing to allow for recognition and use of existing
|
|
information in repeated uses of a symbol. @xref{Multi-function Calc}.
|
|
|
|
@item Syntax error
|
|
An error encountered during parsing of an input stream due to invalid
|
|
syntax. @xref{Error Recovery}.
|
|
|
|
@item Token
|
|
A basic, grammatically indivisible unit of a language. The symbol
|
|
that describes a token in the grammar is a terminal symbol.
|
|
The input of the Bison parser is a stream of tokens which comes from
|
|
the lexical analyzer. @xref{Symbols}.
|
|
|
|
@item Terminal symbol
|
|
A grammar symbol that has no rules in the grammar and therefore is
|
|
grammatically indivisible. The piece of text it represents is a token.
|
|
@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
|
|
@end table
|
|
|
|
@node Copying This Manual
|
|
@appendix Copying This Manual
|
|
|
|
@menu
|
|
* GNU Free Documentation License:: License for copying this manual.
|
|
@end menu
|
|
|
|
@include fdl.texi
|
|
|
|
@node Index
|
|
@unnumbered Index
|
|
|
|
@printindex cp
|
|
|
|
@bye
|
|
|
|
@c LocalWords: texinfo setfilename settitle setchapternewpage finalout
|
|
@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
|
|
@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
|
|
@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
|
|
@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
|
|
@c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex
|
|
@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
|
|
@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
|
|
@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
|
|
@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
|
|
@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
|
|
@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
|
|
@c LocalWords: longjmp fprintf stderr preg yylloc YYLTYPE cos ln
|
|
@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
|
|
@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
|
|
@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
|
|
@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
|
|
@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless typefull yynerrs
|
|
@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
|
|
@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
|
|
@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
|
|
@c LocalWords: YYEMPTY YYRECOVERING yyclearin GE def UMINUS maybeword
|
|
@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
|
|
@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm VCG notype
|
|
@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
|
|
@c LocalWords: YYPRINTF infile ypp yxx outfile itemx vcg tex leaderfill
|
|
@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
|
|
@c LocalWords: yyrestart nbar yytext fst snd osplit ntwo strdup AST
|
|
@c LocalWords: YYSTACK DVI fdl printindex
|