Future/bison
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/bison.git synced 2026-03-09 04:13:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

tests: provide check-examples and check-tests

Browse Source 
* examples/local.mk (check-examples): New.
* tests/local.mk (check-tests): New.
* README-hacking.md: Document them.
This commit is contained in:
Akim Demaille
2021-01-26 07:11:32 +01:00
parent b27bba5ced
commit 77a8b84fc8
4 changed files with 29 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
README-hacking.md
View File

@@ -388,6 +388,11 @@ In examples/, there is a number of ready-to-use examples (see
suites run by `make check`. The test results are in local `*.log` files suites run by `make check`. The test results are in local `*.log` files
(e.g., `$build/examples/c/calc/calc.log`). (e.g., `$build/examples/c/calc/calc.log`).
To check only the examples, run `make check-examples`. To check just one
example,
make check-examples TESTS='examples/c/bistromathic/bistromathic.test'
### The Main Test Suite ### The Main Test Suite
The main test suite, in tests/, is written on top of GNU Autotest, which is The main test suite, in tests/, is written on top of GNU Autotest, which is
part of Autoconf. Run `info autoconf 'Using Autotest'` to read the part of Autoconf. Run `info autoconf 'Using Autotest'` to read the
@@ -401,28 +406,23 @@ contains lots of details that should help diagnosing issues, including build
issues. The per-test logs are more convenient when working locally. issues. The per-test logs are more convenient when working locally.
#### TESTSUITEFLAGS #### TESTSUITEFLAGS
To run just the main test suite, run `make check-local`. To run just the main test suite, run `make check-tests`.
The default is for make check-local to run all tests sequentially. This can The default is for `make check-tests` to run all tests sequentially. This
be very time consuming when checking repeatedly or on slower setups. This can be very time consuming when checking repeatedly or on slower setups.
can be sped up in two ways: This can be sped up in two ways.
Using -j, in a make-like fashion, for example: 1. Using -j, in a make-like fashion, for example:
$ make check-local TESTSUITEFLAGS='-j8' $ make check-tests TESTSUITEFLAGS='-j8'
Actually, when using GNU Make, TESTSUITEFLAGS defaults to the -jN passed to When using GNU Make, TESTSUITEFLAGS defaults to the -jN passed to it, so you
it, so you may simply run may simply run
$ make check-local -j8 $ make check-tests -j8
Running only the tests of a certain category, as specified in the AT files 2. Running only the tests of a certain category. See
with AT_KEYWORDS([[category]]). Categories include: [tests/README.md](tests/README.md#keywords) for the list of categories.
- c++, for c++ parsers
- deprec, for tests concerning deprecated constructs.
- glr, for glr parsers
- java, for java parsers
- report, for automaton dumps
To get a list of all the tests (and their keywords for -k), run To get a list of all the tests (and their keywords for -k), run
@@ -430,11 +430,11 @@ To get a list of all the tests (and their keywords for -k), run
To run a specific set of tests, use -k (for "keyword"). For example: To run a specific set of tests, use -k (for "keyword"). For example:
$ make check-local TESTSUITEFLAGS='-k c++' $ make check-tests TESTSUITEFLAGS='-k c++'
Both can be combined. Both can be combined.
$ make check-local TESTSUITEFLAGS='-j8 -k c++' $ make check-tests TESTSUITEFLAGS='-j8 -k c++'
To rerun the tests that failed: To rerun the tests that failed:
@@ -488,7 +488,7 @@ useful. Here's one way to set them up with GCC 10 on Mac Ports
4. Run the tests with leak detection enabled 4. Run the tests with leak detection enabled
(`ASAN_OPTIONS=detect_leaks=1`). E.g. for counterexamples: (`ASAN_OPTIONS=detect_leaks=1`). E.g. for counterexamples:
``` ```
$ make check-local TESTSUITEFLAGS='-j5 -k cex' ASAN_OPTIONS=detect_leaks=1 $ make check-tests TESTSUITEFLAGS='-j5 -k cex' ASAN_OPTIONS=detect_leaks=1
``` ```
5. You might need a suppression file. See 5. You might need a suppression file. See

3
examples/local.mk
View File

@@ -93,6 +93,9 @@ dist_examples_DATA = %D%/README.md
CLEANDIRS += %D%/*.dSYM CLEANDIRS += %D%/*.dSYM
.PHONY: check-examples
check-examples: check-TESTS
include %D%/c/local.mk include %D%/c/local.mk
include %D%/c++/local.mk include %D%/c++/local.mk
include %D%/d/local.mk include %D%/d/local.mk

9
tests/README.md
View File

@@ -7,11 +7,12 @@ define a number of other macros to:
- what skeleton is used - what skeleton is used
# Keywords # Keywords
The following keywords are used to label tests. Please follow them The following keywords are used to label tests (with
and avoid creating synonyms. AT_KEYWORDS([[category]])). Please follow them and avoid creating
synonyms.
- action - action
- api.value.type - api.value.type
- c++ - c++: c++ language
- cex: counterexamples. - cex: counterexamples.
- d: d language - d: d language
- deprec: deprecated features. - deprec: deprecated features.
@@ -23,7 +24,7 @@ and avoid creating synonyms.
- %merge - %merge
- multistart - multistart
- push - push
- report - report: for automaton dumps
- %union - %union
- variant - variant

3
tests/local.mk
View File

@@ -114,7 +114,8 @@ recheck: $(RUN_TESTSUITE_deps)
-e 'eof && /^(\d+).*: FAILED/ && print "$$1 "' \ -e 'eof && /^(\d+).*: FAILED/ && print "$$1 "' \
%D%/testsuite.dir/*/testsuite.log) %D%/testsuite.dir/*/testsuite.log)
check-local: $(RUN_TESTSUITE_deps) .PHONY: check-tests
check-local check-tests: $(RUN_TESTSUITE_deps)
$(RUN_TESTSUITE) $(RUN_TESTSUITE)
# Run the test suite on the *installed* tree. # Run the test suite on the *installed* tree.