RGBASM does this itself, so should not output sections with 16-bit
alignment, but an invalid object file could otherwise cause
`getPlacement` to infinitely loop.
Since SRAM sections start from 0, we had been using a signed
`int8_t curScrambleSRAM` counter to allow 0 as a scrambled bank,
but this actually just placed all floating SRAM sections in bank 0.
Commit 21eaa2d2cc missed some
intentional unsigned underflow that is needed to set `nbTilesInBank`.
Without it, some of the randomly-seeded RGBGFX tests, such as
`./rgbgfx_test seed8.bin -b 207 -N 54,256`, can fail with "Color
mismatch after round-tripping" errors.
- If both banks 0 and 1 were completely filled, then
`assume(bank == 0)` would false and potentially crash.
- If the input image had fully-background tiles, they would be
incorrectly counted towards `nbTiles` and could cause the
"Image contains N tiles, exceeding the limit" check to fail.
`tileData[realY + 1 % options.bitDepth]` relied on order of
operations and the fact that `1 % 1 == 0` but `1 % 2 == 1` for
correct indexing. However, it was not obvious at first glance,
and confusable with the more common `(base + offset) % size`
pattern. (And would need changing anyway if we ever support 4bpp.)
Negative macro arguments count from the end, i.e. `\<-1>`
is equivalent to `\<_NARG>`, even after `shift`ing them.
Negative arguments cannot be used to access shifted values.
Creating a `Symbol` with `type = SYM_LABEL` but `section = nullptr`
is inconsistent and dangerous. I was not able to cause any buggy
behavior so far, but it's safer and reasonable to not create such
a symbol in the first place.
The main consequence is that `DEF(LabelOutsideSection)` will now
evaluate as 0.
This value "has no impact unless `INSTALL_PARALLEL` is enabled",
and since our install procedure is so simple, I don't think we need
to add that. (The `make install` commands don't use `-j` either.)
- Use CTest labels to filter tests ("internal"/"external", "free"/"nonfree",
and individual tool+project names) instead of defining `TESTS_RUN_NONFREE`
- Allow each external test to run independently in CTest
- Remove the unused no-op `fetch-test-deps.sh --only-internal` option
We shouldn't assume how the user wants to install things.
In particular, `--break-system-packages` is egregious.
---------
Co-authored-by: Rangi <[email protected]>
We only use `$<INSTALL_PREFIX>`` when cross-compiling for MinGW on Linux,
so the relevant block is now wrapped in `if(WIN32)`
so a user can more easily lower the `cmake_minimum_required` version.
- Fix logic for color set comparison (which affects sorting them)
- Prune color sets which are proper subsets of newly-encountered ones
(a comment implied we were already doing this, but we weren't)
- Add more verbose logging to debug this behavior
Instead of reading 64 characters at a time into a rolling buffer
as `shiftChar()` is called, we read 8192 characters at a time into
a complete buffer before any `peek()`/`shiftChar()` operations.
This will let us use C++20 features that GCC 9's experimental
C++20 support did not yet cover, such as "concepts".
This reverts some commits:
- 6bcd79b997
- d5ce5329ea
- 728d14879b
Commit 63a911e657 switched
to `-std=c++20` since at the time we required GCC 10 or higher.
Commit d5ce5329ea partially reverted
to `-std=c++2a` since we once again support GCC 9, which did not
yet handle `std=c++20`.
The latter is not a Bash built-in,
and some environments turn out to produce non-conforming
return codes.
`type` is provided by Bash itself,
so we can rely on it better.
- Add more `assume()` checks in the lexer
- Replace double negative "`disable* = false`" with "`enable* = true`"
- Naming convention of "`std::deque<> *Stack`" when relying on
`.push_front()` and `.pop_back()`
MSBuild is slow, and its output is hard to parse visually
(it's all interwoven, like Make without `--output-sync`).
Since we use CMake anyway, we don't care about the build system.
We can, so let's not ask if we should!
More realistically, we can right now so it's nice to mark that we do, but this is subject to be re-evaluated if any later changes conflicts with this.
Note also that this only affects the binaries we distribute with our releases!
It may very well be possible to compile RGBDS on even older versions of Mac OS X if you have the appropriate compiler setup;
we're just not testing it nor offering such binaries ourselves, but third-party packagers are welcome to do so.
(Looking at you, TigerBrew! ;)
- Do not error about local labels following keywords in skips or captures (fixes#1955)
- Do not incompletely attempt to handle line continuations in skips (fixes#1956)
- Rename `skipToLeadingIdentifier` to `skipToLeadingKeyword`, refactor to merge
`skipToEOL` into it, and use it for both skips and captures
Automatically grabs and compiles libpng, and avoids having hacky lines in our Makefile
(the compiler specification *should* be orthogonal to the build target!)
Also move the MinGW package install to the `install_deps.sh` script,
to move logic off of the YAML.
- Turns out the variable wasn't being set, and thus the flags weren't being passed
- Passing the flags via those variables is basically a bad idea, and it
turns out to break libpng's config script
- The flags need to be passed during link as well, otherwise the binary
isn't linked as a fat one
Fixes#1916
This claim was true in v0.4.2, but intentionally changed in v0.5.0.
rgblink(1) already explains that "all visible labels and exported numeric constants"
will appear, which is a more appropriate location than rgbasm(5).
`uint32_t(param)` is actually functional notation of explicit casting,
not a direct constructor call. `uint32_t{param}` would be uniform
initialization syntax for a constructor call, but would not allow
narrowing from `uint64_t`. See issue #1904 for discussion.
- No fractional digits are necessary after the decimal point,
e.g. `42.` is valid instead of `42.0`
- Error messages refer to "fixed-point" not "integer" constants
- Test more carefully for lexing unrelated to underscores
- Updates `actions/checkout` from 4 to 6
- Updates `docker/login-action` from 3 to 4
- Updates `Chizkiyahu/delete-untagged-ghcr-action` from 5 to 6
- Updates `actions/upload-artifact` from 4 to 7
- Updates `actions/download-artifact` from 4 to 8
- Updates `cygwin/cygwin-install-action` from 4 to 6
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Turns out those get new major releases somewhat often, and we get deprecation warnings and brownouts and all that.
Automatic bumps should help us avoid that, though I've set a *very* large scan interval to shield us from churn.
Besides the immediate performance improvement, reducing
the frequency of downloads should make spurious
failures (network, rate-limiting...) less bothersome.
`FetchContent` respects existing installs, and downloads and compiles the libs
if they aren't found.
This is admittedly a little finicky, since this ignores the usual `Find*`
modules provided by CMake, requiring a bit of glue on our side.
But, one upside is that this moves that logic from our CI into the build system,
which can thus benefit other downstream users.
This also opens the door to some improvements in upcoming commits.
Doing this with Bison turned out to be much more painful, however, due to
`FindBISON` providing the specific `bison_target` command; thus, it remains
installed externally, so that it can be picked up by `FindBISON`.
This also bumps our CMake version requirement slightly, though it's
possible that older versions keep working, or could be supported with small patches;
however, our CI doesn't provide anything below 3.31, so we can't check.
Set the flags in the build itself instead of trying to override the cache
(this was an artifact of when I didn't understand the CMake variable model),
which avoids them being duplicated on each re-configuration.
Also remove `-g`, since it's defined by the build type by default
(we leave it up to the user to choose Debug or RelWithDebInfo).
Make the common files into an object library, which lets them
be compiled only once (saving 41 build steps)
This also lends itself well to removing the per-program loop,
which simplifies the code somewhat.
Note that the DLL name created by CMake with `BUILD_SHARED_LIBS`
has changed from `zlib1.dll` to `z.dll`. This was caused by the
"Continued rewrite of CMake build [Vollstrecker]" from its
changelog <https://zlib.net/ChangeLog.txt>, and has been reported
in <https://github.com/madler/zlib/issues/1181>.
The MinGW package `libz-mingw-w64-dev` still calls it `libz1.dll`.
`bison -Wall` enables all diagnostics except `-Wcounterexamples`
(added in 3.7; we already pass it), `-Wyacc` (which we do not need),
and `-Wdangling-alias` (added in 3.5).
These were observed in https://udd.debian.org/lintian/?packages=rgbds
and can be reproduced by running:
for f in man/*; do
echo $f
LC_ALL=C.UTF-8 MANROFFSEQ='' MANWIDTH=120 \
man --warnings -E UTF-8 -l -Tutf8 -Z "$f" >/dev/null
done
- ID numbers (for fstack nodes, sections, symbols, patches, etc)
might be too large for their associated collection
- Enum values might be invalid
- Bank values might be out of range for their section types
gcc 15.2.1 20250813 complains "address of automatic variable can
escape to `musttail` call" from `-Wmaybe-musttail-local-addr`,
and guaranteeing tail-call optimization cross-platform is more
trouble than it's worth.
RGBASM tests now use `rgblink -x` instead of `dd` to get trimmed
test output. RGBLINK tests cannot do so because some of them
(e.g. bank-numbers.asm and sizeof-startof.asm) rely on ROMX
sections above 1, and `-x` implies `-t` which breaks that.
* Use clang-tidy `misc-include-cleaner` for IWYU `#include` cleanup
* Use `std::optional<size_t>` instead of `ssize_t`
* Rename some functions in linkdefs.hpp
* Fix header order
- "Double quotes" for strings (filenames, section names, CLI option arguments, etc)
- 'Single quotes' for characters and CLI option flags
- `Backticks` for keywords and identifiers (symbol names, charmap names, etc)
CLI option flags also have their leading dashes
* Move the folder structure to ARCHITECTURE.md (which can be
expanded as per #707)
* Move the acknowledgements to CONTRIBUTORS.md
* Remove the history (it's a copy of man/rgbds.7)
This is an edge case of how the lexer changes behavior when
reading string literals versus reading unquoted source code.
Three double quotes would begin a multi-line string literal,
but just two are still ambiguous, so it has to `peek()` to
see whether it's inside a multi-line string literal or
outside an empty regular string.
Profiling RGBDS on building pret/pokecrystal reveals commit
cfe1f60e47 as a clear outlier
among the past few hundred commits for reducing performance
(an 8% increase on my machine from ~5 to ~5.4 seconds).
* Implement warning diagnostic flags for RGBFIX
* `-m/--mbc-type help` prints to stdout
* Support `-m list` as well as `-m help`
* Make invalid `rgbfix -l` characters a fatal error, like other invalid CLI arguments
* Refactor fix/main.cpp into multiple files
- Single unified routine for erroring out or handling missing dependencies
- Single three-state enum instead of two Booleans for missing dependencies
(this causes `-MC` to imply `-MG` instead of needing `-MG -MC`)
- Functions than can miss included files return a Boolean for whether the
parser should `YYACCEPT` and exit
Windows 2025 no longer has a D:/ drive
<https://github.com/actions/runner-images/issues/12416>
which affects where the cached files go.
This can result in one of two errors in the "Install zlib" step
after restoring from cache:
- file cannot create directory: D:/a/rgbds/rgbds/install_dir/lib.
Maybe need administrative privileges.
- file INSTALL cannot find "C:/a/rgbds/rgbds/zbuild/Release/zlib1.dll":
File exists.
Separate caches per OS should solve this problem.
* No need to use `locale`s
* Implement octal numbers, `_` digit separators, and `0x/0b/0o` prefixes in linkerscripts
* Refactor some functions out of `yylex`
* Support `\0` in linkerscripts
This feature is referred to as "code/data literals" in ASMotor,
and simply as "literals" in some older assemblers like MIDAS
for the PDP-10. RGBASM already had the "section fragments"
feature for keeping disparate contents together when linked,
so these worked naturally as "fragment literals".
* Remove `err` and `warn`, keep `errx` and `warnx`, using them in RGBGFX too
* Separate RGBGFX and RGBLINK warnings/errors from main options
* Separate `report` function into `error` and `fatal` messages
* Implicit newlines for most RGBASM errors
The original algorithm makes a simplifying assumption that one
proto-palette does not fully contain another, and we have no
particular reason to violate this condition.
ACT palette files support a transparent color index, but RGBGFX
cannot apply *one* such transparent color; it would need every
palette's first color to be transparent. Also ACT files tend to
say the first color is transparent anyway, which the user may
not have intended.
ACO palette files can specify version 2 color data, but it's
required to come after version 1 data, and the colors themselves
already exist in the earlier v1 data; v2 just adds UTF-16 names.
Thus, we do not need to be handling these data in those formats.
These fall into a few categories:
- `_unreachable()`
- Verbose print messages
- Errors that should never practically occur (alloc/read/write failure,
more than UINT32_MAX anonymous labels, etc)
`Expansion::advance()` can increase its offset beyond the size,
so I don't think this assumption was valid in the first place;
`BufferedContent::advance()` should be able to as well.
- `-fsanitize=undefined` encompasses multiple checks we were specifying
- "detect_leaks=1" for `__asan_default_options` checks for memory leaks
(except for with macOS clang++, which does not support LSan)
- `-fsanitize=float-divide-by-zero` is an extra UBSan check
(and reveals a UB bug to fix with fixed-point `DIV` and `LOG`)
This is primarily a correctness change, *not* a performance one.
The expected performance impact is minimal anyway.
The goal is to eliminate the use of platform-inconsistent floating-point operations
for this load-bearing task.
* Remove namespaces
* Prefer `bool operator==`, not `friend auto operator==`
* Prefer not to use `using`
* Use a `constexpr` function instead of a template for `flipTable`
Identify files by (device, inode), not by path, so that symlinks,
relative paths, case-insensitive paths, or other edge cases
do not result in double includes.
Part of that condition's purpose is to ensure that we read the correct
lexer state; but it's possible now for the fstack to be non-empty
*before* the lexer state is registered, i.e. if there is an error
in the function that registers it.
This causes a NULL pointer deref.
- The '#' component for type 's' now escapes the string characters
- The '#' component for type 'f' now prints a precision suffix
- The new 'q' component specifies a precision value
* Implement a '#' prefix for raw identifiers that may alias keywords
* Review comments
* Disallow hashless raw identifiers in interpolations
* Run clang-format
* Implement custom generic tagged union `Either`
This should be more efficient than `std::variant`, while still
keeping runtime safety as it `assert`s when `get`ting values.
* Use `Either` for RPN expressions
* Use `Either` for file stack node data
* Use `Either` for `File` buffer
* Use `Either` for `STRFMT` args
* Use `Either` for RGBLINK symbol values
* Support an equivalent of `std::monostate` for `Either`
* Use `Either` for lexer tokens
* Use `Either` for symbol values
* Use `Either` for lexer mmap/buffer state
This line was missing; in its absence, as soon as the array reallocs,
all registered symbol pointers are invalidated, leading to a crash
(e.g. when generating a sym file).
Made the reallocation check into a hard error, too.
Since fileSymbols gets registered with each section, we would have to
change that, and then it cascades throughout all of RGBLINK.
This will be handled correctly in the Rust port.
Passing a count parameter to the vector constructor does not reserve
count elements but instead fills the vector with count elements.
This caused the endianness check to fail first because the first 256
characters were null bytes.
The function only stopped at the end of the *arrays*,
not of *their used portions*!
This could cause false negatives one way or the other.
This appears not to affect the palette packing, since the packing algorithm
deals with them efficiently; but it should speed up processing slightly,
and as the test changes show, it also improves the UX of palette packing
error messages!
* Clarify the operator relationship of ordinary and fixed-point numbers
* Attempt to clarify description of fixed-point numbers
* Note that RGBASM does not check fixed-point precisions
* Simplify sine table example a bit
* Remove misleading equations describing `DIV`, `MUL`, and `FMOD`
* Various minor style and formatting fixups
The generated `result.pal` can change between platforms, it turns out
(due to "unstable sort" differences), and creates additional churn if
changing anything related to palette generation.
tl;dr this makes the test less flaky in the long run.
- FIX: `Label & const` was not actually doing the `& const` masking
(fixes#1446)
- ADD: `LOW(Label)` can be constant if `Label` is aligned to 8 or more bits
(resolves#1444)
- ADD: `!expr` can be constant 0 if `expr` has any non-zero bits
(resolves#1447)
- `LOW()` and `HIGH()` have their own RPN operator values
(resolves#1445)
The change to RPN values means that the object file version was incremented.
This also refactors unary operators and functions, combining their
evaluation similarly to binary ones.
Large sizes are more efficient when it's actually buffered,
but most of the time `mmap` is used instead, and the extra size
just slows down allocation of lexer states.
- Fixed-point formulas are implemented using IEEE-754 floating-point
internally, which could give infinity or NaN values whose conversion
to fixed-point integer was platform-dependent.
- Formatting fixed-point $8000_0000 (INT32_MIN, -2147483648) was
not putting the negative sign in front.
The Python Pillow image library gives different output for
`Image.getcolors()` in version 9 than version 10. This causes
libbet's SGB border to be built differently, giving a different
ROM hash and technically failing the test script.
This patches libbet to only keep the known-good colors. It also
adapts the coverage script to run all the tests, including
building the third-party projects.
This way, if a child context initializes `\@`, the parent won't
reset it. And if the child context did not initialize `\@`,
then resetting it would be redundant.
`std::visit` is (arguably) cleaner code, but older versions of gcc
and clang (not very old; the ones packaged with Ubuntu 22.04 LTS)
compile them as tables of function pointers, instead of efficient
jump tables.
This is a work in progress: its performance is unacceptably slow,
and it is obviously not a complete refactoring:
- The parser's semantic functions are still written for C-style
strings, taking `.c_str()` pointers instead of `std::string`
references (and using their methods, `<algorithm>`s, etc).
- Quoted string literals from the lexer still use our `String`
struct, which wraps around a fixed-size char array.
- Symbol values, macro arguments, and so forth are still pointers
to C-style strings with unclear ownership semantics (i.e. we
still have "leaks as a feature").
This allows us to control the order in which sections are iterated,
instead of it depending on the internals of `std::map`. (This order
is arbitrary, but should be deterministic regardless.)
It complains about *those* calls returning a dangling reference...
Unfortunately, GCC does not provide more information about what the
reference *is*. (It only mentions the temporary being destroyed at
the end of this *huge* expression.)
I am normally not one to just commit a thing that gets rid of a
warning if I can't explain why, but this eludes me.
Stubbing out the calls to only return a captured variable still
complains, which led me to test that the error wasn't stemming
from the `Visitor` itself... which it seems to?
But I don't see why a reference to the *visitor* should be kept...
Anyway, here is the obligatory part where I state my yearning for Rust :)
This still doesn't allow identical exported label constants.
That can be addressed when or if it's requested for a real use case.
Symbols only store one source filename + line number, so this
arbitrarily keeps the last read symbol as the reported one.
Fixes `make develop` with Clang 17.
The warning only triggers on the auto-generated `yynerrs_` in `src/asm/parser.cpp`,
so it's not very useful to us right now.
- Since we have style rules to include foo.hpp at the top of its
corresponding foo.cpp, this takes any headers included by foo.hpp
as being also guaranteed for foo.cpp.
- Use C-style <foo.h> instead of <cfoo>, since the latter only
guarantees putting symbols in the `std` namespace, which we are
not using for C functions (e.g. `printf` not `std::printf`).
- Remove now-unused `__PRETTY_FUNCTION__` reporting
We were not initializing some expressions, and they were using
the values of the previous expressions instead. This just so
happened to not crash the tests, and to sometimes even give valid
results (although `BANK()` of a non-label symbol being $4B4E4142,
the ASCII balue of "BANK", was something we missed).
* Avoid redirecting error messages in RGBLINK tests
We check that RGBLINK prints nothing to stdout, so all that
would be captured on stdout is a putative failure message.
* Require each RGBLINK test to check error output
Otherwise, you can have a test that just... checks nothing!
* Document how to add tests
Fixes#1300's last remaining item.
Also add `checkdiff` rules to remind us to update that doc if
any of the test driver scripts are updated.
This removes the last use of `strdup`
This required making a lot of related pointers be `const`.
That in turn conflicted with the need to `munmap()` a pointer
eventually, which was similar to the need to eventually `free()`
an `Expansion`'s contents, so I used the same solution of a
`union`. That lets us normally use the `const` pointer for
`const` correctness, and the non-`const` one for the not-really-
mutating destruction cases.
Split the declarations into those required for the `%union` and
those only required for the `code`.
Only declare functions on top; define them at the bottom.
* Replace FAMs with `std::vector`s (or one `std::string`) in four `struct`s
* Anonymous types declared in an anonymous union are also non-standard
Only Clang complains about this (-Wnested-anon-types)
* Fixes from temporarily re-enabling more compiler warnings
* More edits suggested by cppcheck
* Fix hanging on append_yylval_string
* Fix FOR loop increment
macOS 10.15 introduced full `std::filesystem::path` support.
Before that our use of it would cause the build to fail.
This was not caught because "-mmacosx-version-min=10.9" was only
being passed to clang++ for release builds.
This passes that flag in a new static CI test build, and introduces
a hack developed by @LIJI32 to silence the availability errors,
since we use features already available in macOS 10.9.
This means we are testing both "vanilla" building,
and building static binaries using the same configuration
as during release, which should help avoiding last-minute
surprises.
* Allow for optional sections in linker scripts
These are more useful for frameworks/toolchains.
* Check for an active mem region everywhere
Do you like segfaults? Too bad!
* Allow the address to be floating in linker scripts
Try and make the life of SDCC interop easier.
* Also validate alignment when floating
* Overhaul the linker script manual page
Documenting the new features, but also restructuring the
existing documentation to make the manual page (hopefully)
easier to understand.
Ensure consistency in linker script parser interface
Make the declaration and definition visible when compiling
so that the compiler can warn about inconsistencies.
Co-authored-by: ISSOtm <[email protected]>
Notable side effects:
* Use the standard-conformant MSVC preproc
* Add test for linker script INCLUDE
* Improve wording of placement conflict errors
* Fix errors from not newline-terminated files
* Teach checkdiff about the linker script doc
* Call linker script "commands" "directives" instead
---------
Co-authored-by: Rangi42 <[email protected]>
As reported by the following command:
mandoc -T lint man/* | grep -v "input text line longer than 80 bytes"
(One of them also showed up as a warning while setting up Debian packaging.)
Based on SameBoy's "accurate modern" colour mapping.
Special thanks to @Rangi42 for her patience throughout development, @LIJI32 for researching the colour curve and helping with the creation of the reverse mapping (colour spaces are *fun*!), and @coffeevalenbat for testing :)
Copy CMake-built test executables to test directory
Fixes CI failures on Windows.
CI should also have failed on Unices, but it turns out
we instead merely hit the fallback that the `make` path
relies on (building those executables with `make`),
which fails on Windows since `make` is not set up to
find libpng.
Allows better platform-agnostic path manipulation.
Also, using `std::optional` rather than empty strings allows
correctly handling empty arguments (treating them as such,
instead of acting as they were never passed).
Each symbol passed to purge is collected in a list before mass removal.
Fixes the issue described in gbdev/rgbds issue #1152.
---------
Co-authored-by: Rangi42 <[email protected]>
fix#1157 for the following source code
```
section "hSAVE_locals",HRAM
func3.hSpam: ds 1 ; no longer produces an error
;.hEggs: ds 1 ; uncomment this to see the new error
section "demo",ROM0
func3:
ldh a, [.hSpam]
ret
```
Remove two errors:
- `Not currently in the scope of 'func3'`
- `Local label 'func3.hSpam' in main scope`
Add one error:
- `Relative local label '.hSpam' in main scope`
Co-authored-by: Rangi <[email protected]>
The existing explanations have proven confusing to many,
and presenting the instructions in a more visual manner should help a lot.
Additionally, the Unicode codepoints are typed as-is in the source,
because that allows much more WYSIWYG editing, and god damn it this is the year
two thousand and twenty three, if your man and/or terminal software does't
support Unicode, then what are you *doing*.
The previous test did not expect failure when trying to stuff
$3000 bytes into $2000-byte WRAM; this is now fixed, and we
also test the reverse now (that we cannot stuff more than
$1000 bytes into WRAM0 without `-d`/`-w`.
The addition of SDCC objects required a change in the logic of
`mergeSections()` to dispatch based on `->data` instead of
`sect_HasData`, which implicitly assumes that `->data` is
always initialised (maybe NULL).
However, RGBDS sections did not do that!
- Changes most `/* comments */` to `// comments`
- Changes `/**` block comments consistently to `/*`
- Adds consistent license comments to all files
Also renames `T_POP_SET` to `T_Z80_SET`
Previously, indices would be wrapped after 256 even without a tilemap;
since RGBGFX can generate arbitrarily large tile data if `-N` is not used,
it should be possible to read those in.
Of course, this won't work if a tilemap is provided, but such "big blobs"
can't generate a meaningful tilemap anyway.
Replacing the big pre-formatted text block with a list brings:
- Better accessibility, obviously
- Responsiveness
- Better formatting (bold, etc.)
- Sub-sections that can now be linked to
- Hyperlink cross-refs to other pages
The slight disadvantage is that `ENDC` etc. are now individual
list items, whereas they'd be better as part of the same item.
No big deal though, it was much worse before.
Some descriptions have been overhauled for clarity, and some
outright corrected (such as Assertions' "Offset" field).
Co-authored-by: Antonio Vivace <[email protected]>
This function is made for text, e.g. accepts spaces, leading zeros, etc. before `%u`.
This way checks that the correct amount of bytes are read instead.
This requires a LOT of tricky code, mostly due to the format itself being,
er, not the most straightforward.
Everything is converted to existing RGBLINK concepts (sections, patches,
etc.), so the core code is essentially unchanged.
(A couple of genuine RGBLINK bugs were uncovered along the way, so some of
the core code *is* changed, notably regarding `SECTION FRAGMENT`s.)
All of this code was clean-roomed, so SDCC's GPLv2 license does not apply.
A black+white palette would turn into white+transparent, removing black pixels
from the palette (the third slot would be empty, and the 4th slot be set directly)
The sorting was performed without updating the mappings, which broke the world.
We can instead sort the IDs as they are inserted into the packing queue,
which should also be faster than moving the actual proto-pal objects around.
Not hooked to all RGBGFX flags yet, but good enough for most use cases
(and as a base for future development, should I need to `reset --hard`.)
TODOs marked appropriately.
Force version.c to be compiled as C++ (bodge, will need a proper fix for `version.c`)
Remove user-defined `ProtoPalette` assignment operator (same as default, anyway)
Currently missing from the old version:
- `-f` ("fixing" the input image to be indexed)
- `-m` (the code for detecting mirrored tiles is missing, but all of the
"plumbing" is otherwise there)
- `-C`
- `-d`
- `-x` (though I need to check the exact functionality the old one has)
- Also the man page is still a draft and needs to be fleshed out
More planned features are not implemented yet either:
- Explicit palette spec
- Better error messages, also error "images"
- Better 8x16 support, as well as other "dedup unit" sizes
- Support for arbitrary number of palettes & colors per palette
- Other output formats (for example, a "full" palette map for "streaming"
use cases like gb-open-world)
- Quantization?
Some things may also be bugged:
- Transparency support
- Tile offsets (not exposed yet)
- Tile counts per bank (not exposed yet)
...and performance remains to be checked.
We need to set up some tests, honestly.
It's unlikely that a specific value matters, though in theory possible;
however, an exhaustive test every time is really slow, and testing a couple
random values should, over time, cover everything.
It is fine to test random values because those tested will be logged,
so the problem can be manually reproduced later.
It does make CI technically not deterministic... but that should be fine.
Should not be followed for C files, to avoid mucking `blame`s,
but will be applied during the gradual C++ rollout, since that
will likely touch most lines in the codebase anyway.
Enhances some warnings as well as the sanitizers (Clang especially complained about it)
The `-f*` flags are to get better stack traces out of the sanitizers, as recommended
by Clang's docs: https://clang.llvm.org/docs/AddressSanitizer.html#usage
GCC's docs claim that these optimizations should not hinder the debugging
experience, and Clang's don't mention optimization flags at all.
Remove already-enabled warnings as pointed out by #969
Comments added to the CMakeLists because the Makefile format does not really allow them
"Plain" list also sorted alphabetically, the rest (somewhat) thematically
Attempting to build with this gave an "undefined reference to `yydebug'"
error (maybe a version issue with bison?), and I don't think it's been
used for recent parser debugging either.
Remove unreachable argument presence check (handled by `getopt`)
Merge allocation paths into a single `realloc` call
Avoid searching for string lengths multiple times
Tiny (compatible) change: no space between last dependent and colon if
`-MT` or `-MQ` is specified
* Add Bash completion script for RGBASM
Should have large feature parity with the Zsh completion
Co-authored-by: DaKnig <[email protected]>
* Register RGBASM Bash completion in `checkdiff`
Co-authored-by: ISSOtm <[email protected]>
* Add scramble flags to RGBLINK
-S and -W will scramble ROMX and WRAMX respectively.
* Modify scramble CLI
CLI now takes a list of comma-separated values.
Added arg_error to clean up messages.
Co-authored-by: Eldred Habert <[email protected]>
* Document scrambling functionality
Co-authored-by: Eldred Habert <[email protected]>
* Implement -Wtruncation=level
-Wtruncation=0 is the same as the current -Wno-truncation.
-Wtruncation=2 is the same as the current -Wtruncation.
-Wtruncation=1 is the new default; it's less strict, allowing
N-bit values to be between -2**N and 2**N (exclusive).
* Implement generic "parametrized warning" system
* Test more `Wtruncation` variants
Co-authored-by: ISSOtm <[email protected]>
* Reword label definition docs
A bunch of short sentences isn't very readable, this should be better
* Use correct wording for "computing difference"
Co-authored-by: Rangi <[email protected]>
* Explain how to define a label before mentioning local ones
* Move double-colon paragraph to make the explanation flow better
* Clarify what a label's value is
Co-authored-by: Rangi <[email protected]>
I think we've earned it, at least for being maintainers for a couple years.
I believe the work done by those before us is more important (relicensing & cleanup),
but we've both made significant changes on top of what they made possible.
`TIMER & BATTERY` is 0, but even `|` would only be checking if
*either* is set; instead, imply BATTERY as soon as TIMER is given,
printing a warning if it was not given.
`startCapture` did not initialize `lexerState->atLineStart`;
its final value is a consequence of the separate but similar
behaviors within `lexer_CaptureRept` and `lexer_CaptureMacroBody`.
- Don't warn if the non-zero byte being written is the same as the original byte
- Add a `-O` / `--overwrite` flag to silence all such warnings
Fixes#897
`DEF`, `REDEF`, etc disable EQUS expansion, and reading
macro or OPT arguments sets the lexer to raw mode.
Syntax errors resume normal parsing at the line's end,
but should resume normal tokenization even before that.
This message is only printed for identifiers parsed as
`T_LABEL` (since they're at the start of a line) but
already defined as macros. Otherwise it may not be
relevant, e.g. for `MyLabel;:` or `My Label::`.
The `git config --global --unset core.autocrlf` command
was failing in the "Create release artifacts" workflow,
so this is an alternate method of fixing issue #841.
Fixes#881 by moving the task from the lexer to the parser.
This both alleviates the need for backtracking in the lexer,
removing what is (was) arguably a hack, and causes tokenization
boundaries to be properly respected, fixing the issue mentioned above.
Co-authored-by: Rangi <[email protected]>
* Reinstate PUSHS clearing the SECTION scope
Otherwise you can use `PUSHS` to simulate the old `ds -21`, and possibly cause bugs
* Have PUSHS push LOAD block state as well
It does not make sense not to, and coud cause bugs.
Redefining an EQUS constant will now update its filename,
like redefining a SET/= constant.
Attempting to redefine as EQUS a non-EQUS constant will
print an appropriate error message.
This also standardizes on `sym` versus `symbol` as a
variable name (which pairs with `symName`).
- `out_PushSection` should not set `currentSection` to NULL because
PUSHS, PUSHC, and PUSHO consistently keep the current section,
charmap, and options, even though the stack has been pushed.
- `Callback__FILE__` does not need to assert that `fileName` is not
empty because `__FILE__`'s value is quoted, and can safely be empty.
- `YY_FATAL_ERROR` and `YYLMAX` are not needed since the lexer is
not generated with flex.
Expanding empty strings is valid but pointless;
macro args already skipped doing so, now other
`beginExpansion` calls do too.
This also fixes failed interpolations (which were
already non-fatal) to continue reading characters,
not evaluate to their initial '{' character.
This combines 0.5.0 and post-0.5.0 features:
print and println, strfmt, for loops,
def assignments, redef equs, {interpolation},
new macro syntax, and \<bracketed macro args>
Bison 3.1 introduces "typed midrule values", which would write
`<captureTerminated>{ ... }` and `$$` instead of `{ ... }` and
`$<captureTerminated>[1-9]`, but rgbds supports 3.0 or even lower.
Widths and fractional widths greater than 255 would overflow a
uint8_t and wrap around to smaller values.
Total formatted lengths greater than the avilable buffer size
would overflow it and potentially corrupt memory.
Fixes#830Closes#831
Widths and fractional widths greater than 255 would overflow a
uint8_t and wrap around to smaller values.
Total formatted lengths greater than the avilable buffer size
would overflow it and potentially corrupt memory.
Fixes#830Closes#831
Previously a '.' could be past the truncation limit but still
cause the identifier to be marked as local, violating an
assertion in `sym_AddLocalLabel`.
Fixes#832
This shortens the lexer by 100 lines and simplifies
access to expansion contents, since it usually needs the
deepest one, not the top-level one.
Fixes#813
lexerState->expansionOfs is always either set to 0, or updated by
adding a positive quantity:
if (distance > lexerState->expansions->distance) {
lexerState->expansionOfs += distance - lexerState->expansions->distance;
...
}
so it will always be positive or zero.
Only two sites were for distances greater than 1:
a `shiftChars(2)`, trivial to just do two `shiftChar()`s;
and `shiftChars(size)` in `reportGarbageChar`, which
can be a `for` loop, and should be fixed anyway to
"avoid having to peek further than 0".
Instead of defining `LOOKUP_PRE_NEST` and `LOOKUP_POST_NEST`,
pass a variable name and a block to `lookupExpansion`; it
will assign successive looked-up expansions to the variable
and use them in the block.
The technique of using `__VA_ARGS__` to allow commas within a
block passed to a macro is not original, and should be stable.
This macro was only used twice, in `beginExpansion` and
`lexer_DumpStringExpansions`, with `getExpansionAtDistance`
already containing an inlined and slightly modified version
of `lookupExpansion` (retaining the `LOOKUP_PRE_NEST` and
`LOOKUP_POST_NEST` macros, but with both of them doing nothing).
Not using an X macro here makes the actual control flow in both
places more obvious, and I think the repeated code is acceptable
for the same reasons as the similar-but-distinct implementations
of `readString`, `appendStringLiteral`, `yylex_NORMAL`, and
`yylex_RAW`.
This decoding required high lookahead, and was not even
consistently useful (the `garbage_char` test case was not
valid UTF-8 and so did not benefit from `reportGarbageChar`).
This limits UTF-8 handling to the `STRLEN` and `STRSUB`
built-in functions, and to charmap conversion.
Fixes the test case from #800
The `out.gb` output was corrected, since the two "test"
fragments have a different order in ROM than in SRAM.
It is effectively:
; ROM0[$0000], fragments ordered by size
jr Label
dw Label
db 0
; SRAM[$a000], fragments ordered by .o order
ds 1 ; db 0
ds 2 ; jr Label
Label: ; $a003
ds 2 ; dw Label
Macro args were already handled by `peek`, and character escapes
do not exist outside of string literals.
This only affects the error message printed when a non-whitespace
character comes after the backslash. Instead of "Illegal character
escape '%s'", it will print "Begun line continuation, but
encountered character '%s'".
It still gets written to the object file, generating Valgrind warnings
about using uninitialized memory. To silence those errors, and make
output more reproducible, init the line no to a dummy (0) value.
Explicit {interpolation} can still achieve this, but
to match DEF, REDEF, and PURGE, these new directives that
define symbols do not expand string EQUS.
Currently INCBIN gives an error if your start position == file size.
This means you cannot include an empty file.
It also means the text of the error message is incorrect
(as the start is not greater, but equal to the length of the file)
We'll use "-rcX" instead of "-pre" to allow multiple ones, jic
Additionally, they will be able to be detected using __RGBDS_RC__
Finally, adapt everything version-related to this new system
GitHub's ubuntu-18.04 environment installs gcc 7.3.0.
The latest ubuntu-20.04 environment installs gcc 9.3.0.
Version 9.3.0 is incompatible with building libpng 1.6.37,
as of 2021-03-04.
Fixes#783
* Store all the local symbols in object files, not only the ones that need linker patches.
This generates better map/sym files, and thus allows for better debugging as well.
* Add comments explaining the ->src and the (void)arg;
This was removed in b3c0db218d
(along with two unrelated changes).
Removing this hack introduced issue #742, whereby INCLUDing
a file without a trailing newline can cause a syntax error.
A more proper fix would involve Bison's tracking locations,
but for now the EOF-newline hack fixes the issue while only
affecting some reported errors (expecting "newline"
instead of "end of file").
Fixes#742
* Mark `error` as a `format` function, to properly scan its format
* Fix the call to error() from parser.y:
- Use '%s' to avoid passing an arbitrary format
- Simplify yyerror overall
* Fix size parameter of %.*s format being an int... bonkers standard.
* Report the number of arguments required and provided on a STRFMT mismatch
* Add an assert to check for a very unlikely bug
This applies to macro arguments, DB, DW, DL, DS,
PRINT, PRINTLN, EXPORT, PURGE, and OPT.
It also removes support for empty entries in DB/DW/DL.
(Deprecating it would require keeping parser support,
which is ambiguous with trailing commas.)
Fixes#753
* Fix compatibility of rgblink -O and -t
The -t "tiny mode" option makes ROM0 cover 0x8000 bytes,
not 0x4000. The -O "overlay" option fills areas uncovered
by sections with data from an overlay file. These needed
to cooperate so that the calculated uncovered overlay size
does not exceed the actual size of the ROM.
Fixes#754
* Print link test names like asm tests do
* Make the three test.sh scripts more similar
In exponent(), 'base *= base;' should not run
when base is 65536, since it overflows an int32_t.
This also optimizes exponent() based on
gcc and clang -O3 test cases in godbolt.org.
Fixes#683 and #691
The lexer's raw mode for reading macro args already attempted
to handle semicolons inside string literals, versus outside ones
which start comments. This change reuses the same function for
reading string literals in normal and raw modes, also handling:
- Commas in strings versus between macro args
- Character escapes
- {Interpolations} and \1-\9 args inside vs. outside strings
- Multi-line string literals
Macro args now allow escaping '\', '"', and '\\'.
A consistent model for expanding macro args and interpolations,
within macro args, string literals, and normal context:
- "{S}" should always equal the contents of S
- "\1" should always act like quoting the value of \1
In preparation for an upcoming change
Makes for nicer error messages, complaining about EOF instead of newlines
The hack had to be kept for the lexer raw mode to avoid a bug;
see the relevant code comment for more info.
Simplify the mess that was option setting (2 redundant variables !?)
Move options to a separate file
Have "modules" own their options, and OPT only access them (less redundancy)
Simplify code, respect naming conventions better
- Make it work inside pipelines
- Add RGBFIX tests to the suite
- Be more flexible in accepted MBC names
- Add warnings for dangerous or nonsensical input params
- Improve man page
This is not used in all bison versions, which causes a
"defined but not used" warning for the 'rgbasm_yytnamerr'
replacement, treated as an error by 'make develop'.
Removal of colon-less labels lifted the grammar ambiguity that
prevented `RL` from being usable as a variable declarator.
Thus, reinstate its functionality.
Fixes#669Closes#368Closes#624
Deprecate PRINTT, PRINTV, PRINTI, and PRINTF
Default STRFMT("%f") to 5 fractional digits like "{f:}"
Any use of string formatting will share this default
Document the existing `ROUND`, `CEIL`, and `FLOOR` functions
Also update the trig function docs for searchability
Implement `POW` and `LOG`
Addresses part of #675
Implement ** for integer exponents
** has higher precedence than -, like Python, so -3**4 == -(3**4) == 81
- POSIX sh actually does NOT support `+=`,
but Bash does even in compatibility mode
- `mawk` does not fully support POSIX EREs (regexes),
so work around its lack of brace support
Fixes building on Debian, apparently.
Use better constructs where possible
- <<< instead of echo|
- Parameter expansions instead of `cut`
- Etc.
Improves performance and reliability somewhat
Also, accept "d" between diff line info parts
They are auto-generated, so they wouldn't normally contain Liquid
(The post-processor may insert some, but it has control over the `raw` tags)
Some code introduced as examples contains `{{` due to nested brace expansions,
which was interpreted as (invalid) Liquid. This fixes such problems.
Additionally, Jekyll generates warnings about excerpts being part of a Liquid
block (the `{% raw %}`). This is fine, since doc pages don't use excerpts.
After discussion (starting there:
https://github.com/gbdev/rgbds/pull/594#issuecomment-706437458
), it was decided that plain nested macros should not be
allowed.
Since #590 is fixed, EQUS can be used as a workaround;
multiline strings (#589) will make that easier on the
user when implemented.
Fixes#588, supersedes and closes#594.
Additionally, closes#388.
The old "find symbol with auto scope" function is now three:
- One finds the exact name passed to it, skipping any checks
This is useful e.g. if such checks were already performed.
- One checks that the name is not scoped, and calls the first.
This is useful for names that cannot be scoped, such as checking for EQUS.
Doing this instead of the third should improve performance somehwat, since
this specific case is hit by the lexer each time an identifier is read.
- The last one checks if the name should be expanded (`.loc` → `Glob.loc`),
and that the local part is not scoped. This is essentially the old function.
Use C11 standard _Noreturn instead of attributes (except, of course, MSVC)
Remove unused helpers
Avoid trapping in release builds, in favor of just unreachability
Improve shim when __builtin_* are not available
When a user tried to overwrite a builtin symbol, it would change its
type despite the error, making the second try succeed. This is
problematic, as the location of a builtin symbol cannot be updated.
Build type no longer defaults to Release (!)
have separate options for extra warning flags and sanitizers
toss DEVELOP macro
Fix sanitizers with CMake while I'm at it :|
Since the lexer buffer wraps, the refilling gets handled in two steps:
First, iff the buffer would wrap, the buffer is refilled until its end.
Then, if more characters are requested, that amount is refilled too.
An important detail is that `read()` may not return as many characters as
requested; for this reason, the first step checks if its `read()` was
"full", and skips the second step otherwise.
This is also where a bug lied.
After a *lot* of trying, I eventually managed to reproduce the bug on an
OpenBSD VM, and after adding a couple of `assert`s in `peekInternal`, this
is what happened, starting at line 724:
0. `lexerState->nbChars` is 0, `lexerState->index` is 19;
1. We end up with `target` = 42, and `writeIndex` = 19;
2. 42 + 19 is greater than `LEXER_BUF_SIZE` (= 42), so the `if` is entered;
3. Within the first `readChars`, **`read` only returns 16 bytes**,
advancing `writeIndex` to 35 and `target` to 26;
4. Within the second `readChars`, a `read(26)` is issued, overflowing the
buffer.
The bug should be clear now: **the check at line 750 failed to work!** Why?
Because `readChars` modifies `writeIndex`.
The fix is simply to cache the number of characters expected, and use that.
They were expanded during the capture, and there was no easy way to
avoid expanding them (believe me, after three hours and somehow an OOM, I
gave up trying).
The biggest problem was simply that the length of children expansions was
not accounted for when skipping over the parent... this took a lot of
arduous debugging, but it finally works!
And fix line counting with expansion-made newlines.
This has the same bug as the old lexer (equs-newline's output does not
print the second warning as being part of the expansion).
Additionally, we regress equs-recursion, as we are no longer able to
catch this specific EQUS recursion. Simply enough, the new expansion
begins **after** the old one ends! I have found no way to handle that.
Add keywords and identifiers
Add comments
Add number literals
Add strings
Add a lot of new tokens
Add (and clean up) IF etc.
Improve reporting of unexpected chars / garbage bytes
Fix bug with and improved error messages when failing to open file
Add verbose-level messages about how files are opened
Enforce that files finish with a newline
Fix chars returned not being cast to unsigned char (may conflict w/ EOF)
Return null path when no file is open, rather than crash
Unify and improve error printing slightly
Known to be missing: macro expansion, REPT blocks, EQUS expansions
Description blurb is already inline from new stylesheet
`Xr` links are already handled by `mandoc` now
Handle spaces between both dashes in long options
Remove `<head>` modifications, as fragments are generated instead
Make links to other man pages work
Add a table of contents to rgbasm(5)
Make the OS at the bottom of all pages 'Linux'
Apply post-processor that used to be used
This requires some special-casing for `jr @` because the `jr` opcode has
already been emitted, but not the operand, so PC points to the middle.
Moved the RGBLINK test to RGBASM's folder, and created a new RGBLINK test.
The code expected to never get "known" expressions passed in, as RGBASM
otherwise patches the bytes in by itself; however, JR cannot be patched in
by RGBASM unless the section's base address is known as well!
This allows whitespace between the brackets and the register.
This also fixes#531
Note that `$ff00 + c` is still treated as a single token, because trying to
use an expression on the left side causes a shift/reduce conflict.
This isn't great, but most people seem to be either used to it as-is, or
using the new `ldh a, [c]` syntax.
If this causes problems with a lexer rewrite, it'll be deprecated; but for
now, keep it around, as the support is clunky but bearable.
Create a new file, platform.h, for platform-specific hacks
for MSVC, this includes defining strncasecmp to _stricmp and
strdup to _strdup, among other things like defining missing
stat macros
Change some things not supported in MSVC, like _Static_assert,
to their counterparts (in this case, static_assert)
Replace usage of VLAs with malloc and free
Update getopt_long and use the getopt implementation from musl
on Windows.
Use comments to show which functions from platform.h are being used
While compiling for empscripten there is a duplicate symbol conflict with these two variables. And as they are only used locally, and do not need a symbol exported they can just be static.
This should hopefully work torwards compatibility with more systems.
I've tried to make this as general as possible but some small assumptions
about the compiler are made. I've also tried to recreate the build process
as closely as possible, but I had to change some things slightly to work
with CMake (version strings, mainly).
For now, it doesn't allow in-source builds, as that could overwrite the
Makefile.
This adds:
- Support for more build systems
- Automatic dependency generation
- Performance gains (especially when using i.e. Ninja)
Defaults to Release build.
This one's really stupid; printing an int with %hhu gives a warning
even though a regular unsigned char is promoted to an int before
printing anyway. Silence it so the build with Clang works.
This is a regression introduced in 5c24de3
GCC with the -std=c11 defines __STRICT_ANSI__. DJGPP checks if
__STRICT_ANSI__ is defined and if so doesn't define some things
mandated by POSIX such as struct stat, PATH_MAX, and others.
The -std=gnu11 option does not define this macro, so use it instead.
_DEFAULT_SOURCE isn't needed as no GNU nor BSD-specific functions
are used. Remove it.
Fix the last two occurrences of incorrect format specifiers for standard
fixed-width integer types.
`macro_GetArg` had not been changed after the previous commit; however,
the old code relied on the `macroArgs->args` array being at least
`MAXMACROARGS` entries large (which was the case until the last commit).
The boundary against which it checked would have better been written as
`sizeof(macroArgs->args)/sizeof(macroArgs->args[0])`, I guess, but what's
done is done.
This should help make RGBDS portable to systems with 16-bit integers,
like DOS.
For kicks, use the macros for 16-bit and 8-bit integers.
Fix other miscellaneous things, like #include ordering and other
printf-format related things.
Reduce repitition in math.c while I'm there.
This didn't break unless the first uninitialized byte was non-zero,
which happened to be the case on someone's Windows machine.
Would it be worth it setting up Valgrind in CI?
Fixes#307
RGBFIX can handle padding, so there's no reason why
we can't add an option to disable padding in rgblink.
Signed-off-by: JL2210 <[email protected]>
This includes not checking the result of malloc and
using the wrong type in sizeof. Only the latter was
caught by scan-build.
Also use more idiomatic sizeof().
Signed-off-by: JL2210 <[email protected]>
It's possible that the unsigned integer may overflow to zero,
and then we might use zero-allocated memory.
This is incredibly unlikely, and I would even go so far as to say
that this is a false positive. Fix it anyway, to silence this warning:
src/link/patch.c:92:24: warning: Use of zero-allocated memory
stack.buf[stack.size] = value;
~~~~~~~~~~~~~~~~~~~~~ ^
Deal with overflow, and check for zero to get rid of the warning.
Signed-off-by: JL2210 <[email protected]>
Yet another case caught by scan-build:
src/link/patch.c:188:21: warning: Division by zero
value = popRPN() % value;
~~~~~~~~~^~~~~~~
Just copy over the code from the division case, with a few modifications.
Signed-off-by: JL2210 <[email protected]>
It's possible that if the FILE passed to yy_create_buffer is at the
end-of file, there may be a null pointer dereference.
This should hopefully fix that.
Found with clang-tools' scan-build:
src/asm/lexer.c:281:25: warning: Array access (via field 'pBuffer')
results in a null pointer dereference
pBuffer->pBuffer[size] = 0;
~~~~~~~ ^
1 warning generated.
Signed-off-by: JL2210 <[email protected]>
As 444 and 555 seem to be used for no apparent reason, use the more
conventional 644 and 755.
/man is typically unused or a symlink to /share/man now, so just use
/share/man.
Signed-off-by: JL2210 <[email protected]>
Note: I wanted to enable `-Og` on `develop`, but this generated warnings
(thus, errors) that aren't in `-O0`. Needs further investigation, but
annoyingly some of those are within `extern/` code, thus requiring
different flags, which AFAIK is only possible (sanely) with GNU Make.
This reverts commit 06fe27c516.
According to Microsoft's documentation
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/tmpfile?view=vs-2019
`tmpfile` attempts to create the temporary file at the **root** folder
This seems to assume that the user has admin rights; might be a compat
thing, idk, but it breaks on people's computers.
(CI didn't catch it, annoyingly.)
Reverting to make RGBASM usable on most Windows computers.
(Sanely-configured ones, at least.)
Another solution to #446 needs to be figured out, yay...
See individual comments within `rgbds.css` for more info
Not too fond of having to modify `mandoc.css`, but I did my best to
modify as little as possible
Adds links to argument descriptions in synopsis
Adds links to man pages in the set (not to external ones)
Removes artifact from the way long opts are encoded
Makes description blurb inline, consistently with terminal output
This touched a lot more code than initially expected, for two reasons.
First, this broke a big RGBASM assumption: that sections are always being
written to at their end. This plus other problems required touching
basically the entirety of `section.c`.
Second, I tried different solutions to solve the above problem, and along
the way I cleaned up many things around. (I believe that keeping this to
"cleanup" commits yields subpar results, and since it's boring they get
postponed anyways.)
RGBLINK support still needs to be added, but this will come next.
> Any kind of sequence can be defined using either left recursion or right
> recursion, but you should always use left recursion, because it can
> parse a sequence of any number of elements with bounded stack space.
https://www.gnu.org/software/bison/manual/html_node/Recursion.html
This should significantly improve performance: on pokecrystal builds, perf
reported as much CPU time spent on `yyparse` as on `sym_UseNewMacroArgs`
Measurements show ~6 seconds of improvement on that codebase.
This also fixes#321, as a bonus, due to saner management!
This was utterly stupid. The check right above ensured that `sym` was NULL,
ergo that the argument to `yyerror` *would* segfault.
The only two call sites cannot pass a non-NULL pointer anyways, which I'm
betting is why this went unnoticed.
I did what an optimizing compiler would do anyways: remove the dead code.
After discussing with some Windows user, this actually seems to be
a common thing to do (it seemed weird to me, but I know Windows' handling
of DLLs is weird anyways), so bundle that runtime and reinstate the
full test run for Win32.
As suggested by https://github.com/rednex/rgbds/issues/350#issuecomment-498030458
The order `count` then `byte` was decided after some discussion:
- First argument consistent with single-arg syntax
- Intuitive at least to some people other than myself
- Consistent with other assemblers, at least ca65
This causes it to auto-update whenever the current section's attributes are
updated, simplifying the code and eliminating redundancy.
This should also overall reduce overhead (one extra function call on each
PC evaluation, but less bookkeeping for each byte output)
This avoids redundancy between them (and also having to port fixes and features)
The error messages have been preserved through a string reporting mechanism
Make the bool field an actual bool
Rename `iReloc` to a more exact `isKnown` (as was already pointed out by some
comments)
Make the value of `BANK(symbol)` consistent when the argument is invalid
The syntax was `charmap "anything", "string"`; the second string was
already handled by `const`, but to the same effect... the ambiguous
declaration has been removed, leaving us at "only" two reduce/reduce
conflicts.
This was made separate with the intention of supporting multiple versions,
but after some discussion this was decided against, so better improve
readability instead.
"fixed" and "aligned" location checking advanced the target location to places
regardless of the associated free space, potentially breaking the assumption
that the location was always further in memory than the free space's base.
Rather than adding more code to try keeping that assumption true, harden
`isLocationSuitable` and handle that case as well.
This script allows diffing two Game Boy ROMs using `xxd` and `diff` and
attempts to augment the diff using information pulled from SYM files.
It's not part of RGBDS proper, but can be useful to debug tests (when ROMs fail
to match).
Windows does not honor `%F` nor `%T` in `strftime`. These are worked around
by writing the full format they serve as a short for.
However, Windows also treats `%z` and `%Z` identically, where SUS instead
requires `%z` to output a ±XXXX offset.
Since the current information is broken (no information), this isn't *breaking*
anything, but at least provides something a human will probably understand.
`__ISO_8601_UTC__` is unaffected because it hardcodes the timezone character,
only `__ISO_8601_LOCAL__` suffers from this.
The files being diffed (especially for variants) are temp files, so their
names are pretty nondescript. This improve error output, using ANSI escape
sequences to make those lines stand out.
Fixes rednex/#446.
I am not sure this is the best (in cases where the target directory
is not writable but the target file is), but maybe this can be
toggled via a flag, for example.
This option allows for automatic dependency detection and generation:
as soon as a missing file is found, it is output to the dep file, and
assembly immediately aborts. (No .o file is produced, even if `-o` was
speicified.) This doesn't cause an error, either; the point is that once
the file is added to the dep file, the Makefile is re-parsed, and this
time the file will be generated, so the dep list builds up automatically.
This mimicks GCC's option and behavior.
Allows overriding the output file in dependencies, which also allows
outputting those without also outputting the object file.
This, again, mimicks GCC's option.
Stop using that bitfield for everything, including what can be determined otherwise
It also makes it easier to have a sane state, since some bits were (supposedly)
mutually exclusive
This is actually not necessary, because RGBLINK would warn about missing labels.
Besides, through semi-esoteric ways, it is possible to define more labels in this scope,
and there's no reason to prevent that.
`char` has implementation-defined signedness, and if it's chosen to be unsigned,
then -1 gets converted to 255, which is then promoted back to `int` as... 255,
always failing the loop condition in src/link/main.c:118
`int8_t` has the correct signedness, but considering `musl_getopt_long_only`
returns `int`, better use that so as not to lose any bits
POSIX leaves undefined the order of output if stderr is injected into stdout,
and in practice it differs on Windows (Linux buffers both streams separately,
Windows interleaves them as they arrive without buffering).
This should help testing on other platforms
This macro hid away the arguments to the underlying call, and served
no purpose beyond saving the programmer some typing. This is 2019,
people have IDEs (even Vim!) with autocompletion.
That folder contains external code, and modifying it to conform to our
code style would make applying upstream patches, amongst others,
problematic. Therefore, skip checking it.
Ideally, the folder should also be excluded from `make checkpatch`,
but I haven't figured out a way to do that yet.
Since the RGBLINK rewrite, there have been no .l files in the whole codebase
(RGBASM has the C file directly, for better and for worse)
Since flex isn't used anymore, it's a good idea to remove it from the Makefile
so people don't think it's a dependency.
Those are only useful for locking file IO across threads, but RGBLINK is
single-threaded anyways, so they don't matter. Plus, they aren't provided by
mingw, so that'll remove part of the problem
The error was due to casting `const` away for permuting argv
elements, which is necessary for a libc for compatibility with older
systems, but not for us.
Checkpatch will complain about the style not being followed, but this is not
our code, so it can be ignored.
Both `getopt_long` and `getopt_long_only` are GNU-specific, so we'll be
copying musl's implementation for portability.
This was retrieved as of commit 90251cf73dfdd44e7a3f085d236e89a7dff1b00b.
musl is licensed as MIT, which is compatible (being identical...) to RGBDS'.
The file is being copied as-is, without a copyright notice or attribution,
but this is only to have a verbatim copy in the history. Those will be added
in the next commit.
The goal was to improve readability, but along the way a few things were
gained.
- Sorted sym and map files
- Infrastructure for supporting multiple .o versions
- Valgrind-proof, as far as my testing goes anyways
- Improved verbosity messages
- Added error checking
- Performance improvements, see end of commit message
The readability improvement was spurred while trying to make sense of the
old code while trying to implement features such as sorted sym and map
files.
I also did my best to remove hardcoded logic, such that modifications
should be doable; for example, "RAM loading" sections, which are linked
against a different location than the one they're stored at.
Some work remains to be done, see the "TODO:" and "FIXME:" comments.
Further, while regression tests pass, this new linker should be tested on
different codebases (ideally while instrumented with `make develop` and
under valgrind).
The few errors spotted in the man pages (alignment) need to be corrected.
Finally, documentation comments need to be written, I have written a lot of
them but not all.
This also provides a significant performance boost (benchmarked with a
51994-symbol project):
Current master RGBLINK:
2.02user 0.03system 0:02.06elapsed 99%CPU (0avgtext+0avgdata 84336maxresident)k
0inputs+11584outputs (0major+20729minor)pagefaults 0swaps
Rewritten RGBLINK:
0.19user 0.06system 0:00.63elapsed 40%CPU (0avgtext+0avgdata 32460maxresident)k
23784inputs+11576outputs (0major+7672minor)pagefaults 0swaps
The stdlib functions specify the difference between `err` and `errx`
is that the former prints a message obtained with `strerror`.
However, RGBDS' implementation breaks that contract, and `warn`'s puts the
added semicolon in the wrong place.
This is especially useful when an EQUS expands to another one, to help
track them.
This is done separately from the file stack as the EQUS stack is separate
(which is itself because EQUS are managed *way* differently).
Macro and rept buffers were not always being terminated with newlines
and/or were vulnerable to the final newline being escaped, allowing
buffer overflows to occur. Now, they are terminated with newlines using
the same mechanism as the file buffer.
Null characters in the middle of strings interact badly with the RGBDS
codebase, which assumes null-terminated strings. There is no reason to
support null characters in input source code, so the simplest way to deal
with null characters is to reject them early.
They do not take any room, so they can only be used to define symbols at
a given location. I ran into trouble with such a SECTION failing to be
placed where specified, which doesn't make sense.
This way, it also makes sense to have such a SECTION in the middle of
another one, but that should be fine, since there's no actual overlap.
f29d768 forgot to switch these two `expr->nVal` to `src1->nVal`
This won't break anything, since this wasn't published yet.
I checked, and didn't see any other missed changes.
Reported by pret, I confirmed that `1 && 1` evaluated to 0.
This is good for debugging Makefiles on systems I don't have (eg. macOS)
End users will not be affected by this, and CI provides a persistent log
so clobbering isn't much of an issue.
Some errors are only tripped in `out_WriteObject`, which was
basically a stub when `-o` wasn't specified. Now, instead,
errors are checked in a separate function before out_WriteFile
The old error stack was fairly obtuse and hard to use for debugging.
This improves it notably by ensuring all line numbers are relative
to the file, and not, say, the macro definition.
This is a breaking change if you were parsing the old stack, but
the change should be painless, and the new stack only brings more info.
The syntax is unchanged for files, macros see their name prefixed
with the file they're defined in and a pair of colors, REPT blocks
simply append a '::REPT~n' to the context they're in, where 'n' is
the number of iterations the REPT has done.
This is especially helpful in macro-heavy code such as rgbds-structs.
If a line ended with a string's closing quote, or a newline escape, then
skipping over that line via IF/ELIF/ELSE would fail to count that line,
offsetting the rest of the file.
I have no idea why but for some reason 9829be1 changed *specifically*
`if_skip_to_else` to have incorrect behavior on string endings. The incorrect
behavior on newline escapes seems to have been here since the beginning.
Also added a test to check for both of those behaviors in both functions.
Honestly, it baffles me that nobody ever noticed. I didn't until I started
working on #395.
Unlike macros, REPTs and INCLUDEs, this recursion depth is independent.
This is intentional, because string expansions work very differently.
While it's easy to know when a string expansion begins, checking where it
ends is much more complicated, since the expansion's contents are simply
injected back into the lex buffer. Therefore, the depth has to be checked
after lexing took place.
Because of this, the placement of the expansion end check is somewhat
haphazard, but I think it's good. While I have no certainty, all tests
ended with all expansions properly ended, and I couldn't find any pitfalls.
Finally, `pCurrentStringExpansion` has been made global so error printing
can use it to tell the user if an error occurred inside of an expansion.
REPT blocks nested in macros (and possibly other cases) leaked
memory on every call. Unlike most other memory leaks, which would
be freed at the end of program execution if they were done properly,
those piled up the more compilation went on.
I believe memory usage could have started being fairly high on
large projects following the "one master file INCLUDEs all the rest"
so this may have actually been worth it.
sym_SetMacroArgID used a `sprintf` that could write no \0.
In practice this was benign because %u cannot print 256 chars,
but better future-proof this.
(And REPT.)
Not exactly a *recursion* limit, more like a *stack depth* limit,
but calling it "recursion" conveys its purpose better.
The default of 64 is super overkill: even in a a project with
what I believe to be above-average levels of nesting, the
level only peaked at 6.
Keeping in mind the purpose of this is to catch infinite
recursion, which is still caught quickly (in usual cases, anyways),
this default seems sensible.
And it passes tests. What more do you need?
This adds two new directives: newcharmap and setcharmap.
newcharmap creates a new charmap and switches to it.
setcharmap switches to an existing charmap.
While working on #392, I noticed that the macro-@ test (as well
as the line-continuation test, but for that one see #393)
printed an additional '@(-1)' entry which doesn't make sense.
When trying to skip over nested if statements, if there was no whitespace
after an "if", then that "if" would not be recognized. That's a problem since
"if(" and "if{" are also valid ways to start an if statement. This change
will make it so that they are recognized correctly.
c75a953 broke my (previously-working) project that defined, via
macros, 'sizeof_.player'.
A test was added to confirm that those are indeed accepted
outside of macros.
Should partially cover #178 and close#270.
This allows printing numbers in different bases and without the dollar prefix
This is especially useful in macros because the dollar isnt a valid character
for symbol names, requiring heavy `STRSUB` usage.
When trying to fix a section becoming too large, the size it reached is necessary to know whether to optimize away a few bytes or split it entirely.
This error is also commonly encountered when INCBINing too large a slice of a file, in which case the amount of bytes by which the section is too large is again an useful information
Previously, a PUSHS before a SECTION directive would cause rgbasm to crash when encountering a subsequent POPS.
This is because the subsequently-called out_setCurrentSection() expected the new section to be non-null, which wasn’t the case in this situation. This has been addressed by allowing the ‘null’ section to be set in this function, and only dereferencing it (to set nPC) if a non-null section is to be set.
In practice, this means that PUSHS/POPS can now be used to push/restore a context without a section.
Charmap's previous structure was using brute-force comparison for
converting the strings in source files. It always compared given
strings to all of the strings in charmap, which was very costly
in huge projects.
For its improvement, I changed its structure into trie, which is
being used in many string-processing areas. It's now much faster
than before.
There is a bug in processing the comments in source files. It's
related to #326. And this bug comes out when you comment something
with the character ';', and include the quotation mark without its
pair in it.
The lastest version of rgbds compiler has a step to parse the given
source to convert its line endings to a unified one, and it
processes quotation marks even before it processes the comments.
I edited a little bit of the source, and it works fine now.
If the type char is signed, then in the function
yylex_GetFloatMaskAndFloatLen(), *s can have a negative value and be converted
to a negative int32_t which is then used as an array index. It should be
converted to uint8_t instead to ensure that the value is in the bounds of the
tFloatingFirstChar, tFloatingSecondChar, and tFloatingChars arrays.
The createpatch() function was using a fixed-size buffer. I've changed it
to be dynamically allocated. I saw that the RPN format used in patches is
slightly different from the one used internally in the assembler, so I
added a new member to the Expression struct to track the patch size.
I've also limited the RPN expression length to 1MB. I realized that the
patch RPN expression could potentially be longer than the internal RPN
expression, so the internal expression would need a limit smaller than
UINT32_MAX. I thought 1MB would be a reasonable limit.
When a macro arg appears in a symbol name, the contents are appended.
However, the contents of the macro arg were not being validated.
Any character, regardless of whether it was allowed in a symbol name,
would be appended. With this change, the contents of the macro arg
are now validated character by character. The symbol name is considered
to end at the last valid character. The remainder of the macro arg is
treated as though it followed the symbol name in the asm source code.
It seemed that the consensus in our discussions of signed integer
overflow, which invokes undefined behavior in C, was that integer
arithmetic should be two's complement and there should be no warning for
overflows. I have implemented that by converting values to unsigned types
when appropriate. These changes will mostly preserve existing behavior,
except for a few cases that were being handled incorrectly before.
The case of dividing INT_MIN by -1 previously resulted in a CPU
exception and program termination. Now, that case is detected and results
in a warning and a value of INT_MIN.
Similarly, INT_MIN % -1 would have resulted in a CPU exception. Since this
is a mathematically valid operation with a result of 0, it now simply
gives that result without a warning.
I noticed that in rpn.c, there were attempts in certain operation handlers
to validate the nVal members of the source expressions even when the
expressions may have been relocatable expressions with meaningless numbers
for the nVal member. This could have caused spurious errors/warnings, so I
made those handlers confirm that isReloc is false before validating nVal.
Also, integer constants that are too large now result in a warning. The
post-conversion values have not been changed, in order to preserve
backward compatibility.
Standalone bracketed symbols like the following weren't being zero-terminated.
X EQUS {Y}
This doesn't apply to bracketed symbols that aren't standalone, but are
instead found in a string. For example, the following works even without this
fix.
X EQUS "{Y}"
This branch was meant to contain changes that were considered too risky
to be in master. However, there is not enough activity in the repository
to justify its presence. Both branches are always pointing at the same
commit.
Signed-off-by: Antonio Niño Díaz <[email protected]>
When the while loop in `ParseSymbol` stops because of the symbol length,
`copied` will have the value of `MAXSYMLEN`, which is obviously not
greater than `MAXSYMLEN`. Changing the condition to `>=` fixes the
issue.
As a bonus, the correct union field will now be used. It shouldn't
matter, but it's technically UB to use a wrong one.
The linker script now allows you to assign a section with the same
attributes as in the source.
To do this, I've removed a check from AssignSectionAddressAndBankByName
that would never be triggered, due to that condition being checked
before. Shouldn't this and IsSectionSameTypeBankAndAttrs be condensed
into a single function?
Currently, all symbols are assigned a filename and line when they're
first encountered and added to the internal hash table. This is often
not expected and leads to erroneous error messages.
The original list only existed because the documentation was split
across multiple files. When all keywords are described in a single
document, Ctrl+F suffices to find them.
Two variables, pSection->Data and tSymbols, were previously set to
dummymem, a global variable that was otherwise not used.
As this can potentially cause alignment warnings on Clang, this commit
replaces that mechanism with a plain old NULL pointer, which is more
generally used as a dummy pointer value.
Signed-off-by: Antonio Niño Díaz <[email protected]>
We no longer assume that the test repos don’t exist when we run
run-tests.sh. This allows developers to choose to keep them, to allow
them to run the tests more quickly.
- Add the test repos to the .gitignore.
- Check if the directory for each repo already exists, before trying to
clone it.
- Do a git pull for each repo, to ensure that existing copies of repos
are up-to-date.
Signed-off-by: Antonio Niño Díaz <[email protected]>
This ensures that build breaks to any of the test projects don’t immediately cause rgbds tests to fail.
On clone, I’ve set it up to pull the commits since the day before the desired commit. Sadly, this will clone more recent commits that we’re not testing, but at least it ensures that the desired commit can be checked out. This is hopefully a good enough replacement for —depth=1.
Signed-off-by: Ben10do <[email protected]>
Two variables, pSection->Data and tSymbols, were previously set to ‘dummymem’, a global variable that was otherwise not used.
As this can potentially cause alignment warnings on Clang, this commit replaces that mechanism with a plain old NULL pointer, which is more generally used as a dummy pointer value.
Signed-off-by: Ben10do <[email protected]>
We no longer assume that the test repos don’t exist when we run run-tests.sh. This allows developers to choose to keep them, to allow them to run the tests more quickly.
- Add the test repos to the .gitignore.
- Check if the directory for each repo already exists, before trying to clone it.
- Do a `git pull` for each repo, to ensure that existing copies of repos are up-to-date.
Signed-off-by: Ben10do <[email protected]>
A regression was spotted in rgbfix 0.3.7, where we would accidentally include the first byte of the existing checksum when calculating a global checksum. We now correctly ignore both of the existing checksum bytes.
This was spotted when running rgbfix on the Pokémon Gold/Silver betas, as they have a non-zero global checksum.
Fixes#280.
Signed-off-by: Ben10do <[email protected]>
The following mnemonics are now valid:
- ld
- ldh
- ldio
The following are valid as operands:
- [$ff00+c]
- [$ff00 + c]
- [c]
This is done for consistency with 'ld [$FF00+n],a' and variations of it.
Signed-off-by: Antonio Niño Díaz <[email protected]>
The tests are not exhaustive, there are some conditions that aren't
checked. The tests are based in the C standard rules about undefined
behaviour.
This is a compatibility break but, hopefully, all projects are using
sane values. If not, there is no guarantee that the projects will build
in any platform where RGBDS can be compiled, so it would be better to
fix them.
Even though, technically, the left shift of a negative value is always
undefined, some projects rely on its current behaviour. This is the
reason why this doesn't cause a fatal error.
Signed-off-by: Antonio Niño Díaz <[email protected]>
GCC has an Undefined Behavior Sanitizer (ubsan), which enables run-time
checks of undefined behaviour. It has been enabled for the `develop`
build target.
A small bug detected with it has been fixed.
Signed-off-by: Antonio Niño Díaz <[email protected]>
The Linux kernel expects the SPDX license tag to be in the first line of
all source code files. The reason is that they have many different
license headers, and this simplifies the work for any tool that has to
determine the license of each file.
In this project, the license headers follow the same pattern, so it
isn't useful.
Signed-off-by: Antonio Niño Díaz <[email protected]>
Previously, the output file was opened before trying to open the
overlay. Because of this, rgblink generated an empty output file if the
overlay couldn't be opened.
Now the overlay is opened (and checked) before the output file, so the
output file is only generated if the overlay is found.
Signed-off-by: Antonio Niño Díaz <[email protected]>
Rewrite rgbfix to perform most actions in memory.
Limit file operations to a small number that can be reasonably checked,
and do the majority of the actual work on a buffered header in memory
where operations won't fail.
Signed-off-by: Antonio Niño Díaz <[email protected]>
Limit file operations to a small number that can be reasonably checked,
and do the majority of the actual work on a buffered header in memory
where operations won't fail.
Enable a lot of GCC warnings and cleanup Makefile and compiler macros
Most of the warnings in the GCC documentation have been enabled. As the
build system was using `-Werror`, this made the new builds very
susceptible to any minor change in the compiler. To fix that, this
option has been removed from the default Makefile target and it is only
used when the code is compiled through `make develop`.
The file `stdnoreturn.h` has been modified to remove support for
compilers that aren't actually tested. It now has defines for
`__attribute__((unused))` and it has been renamed to `helpers.h`.
Signed-off-by: Antonio Niño Díaz <[email protected]>
Added define 'unused_' for '__attribute__((unused))'. The oldest version
of GCC with online docs (GCC 2.95.3, released in March 16, 2001 [1])
already has support for this attribute, so it doesn't make sense to
check the version.
Renamed 'noreturn' to 'noreturn_' for consistency.
[1] https://gcc.gnu.org/onlinedocs/
Signed-off-by: Antonio Niño Díaz <[email protected]>
`__attribute__((noreturn))` has been supported since GCC 2.5, that was
released October 22, 1993. It doesn't make sense to check if the version
is at least that one, we are compiling for C99, that is more modern. [1]
Also, remove the MSVC check. This code is never compiled with it so
there may be problems that need to be solved to make it compile. All
releases cross-compiled from linux. If there is an actual need to
support MSVC, the compiler definitions can be added again.
Also, if the compiler is not supported, the compiler helpers default to
nothing, so the code can still compile.
[1] https://gcc.gnu.org/onlinedocs/
Signed-off-by: Antonio Niño Díaz <[email protected]>
Remove '-Werror' from the default make target to make it easy for
regular users of RGBDS to compile the source code. Only a few basic
warnings are left in that target.
All the warnings have been moved to a new target called 'develop'. This
target is now the one used in Travis CI to check for problems during
compilation.
Signed-off-by: Antonio Niño Díaz <[email protected]>
-Wformat-truncation is set to 1 instead of 2 because in some cases the
return value of snprintf is used to warn about truncations but GCC still
creates a warning for it, preventing the compilation from continuing.
Signed-off-by: Antonio Niño Díaz <[email protected]>
-Wsign-compare has been disabled because flex generates a comparison
that triggers a warning and cannot be fixed in the code.
Signed-off-by: Antonio Niño Díaz <[email protected]>
Fix a few warnings related needed to build the source with this option.
Add new exception to .checkpatch.conf.
Signed-off-by: Antonio Niño Díaz <[email protected]>
When '@' is used as argument of any instruction (LD, JR, JP, etc), the
address it refers to is the address of the first byte of the
instruction. When '@' is used with DB/DW/DL, it refers to the same
address it is being placed at. This means that instructions need an
offset of 1 byte and others need an offset of 0 bytes.
The assembler doesn't evaluate anything related to '@' because it would
only work in sections with a fixed base address. It is left to the
linker. This means that the offset needs to be added to the RPN
expression of the patch that is saved in the linker. It isn't enough by
adding an offset to the final expresion. The following value wouldn't be
calculated correctly (even if it doesn't make sense):
JP @ * @
The correct patch is `(@ - 1) * (@ - 1)`, not `(@ * @) - 1`.
This patch introduces an offset on 1 byte by default in every line, and
sets it to 0 only if a DB/DW/DL is detected.
Signed-off-by: Antonio Niño Díaz <[email protected]>
This change removes 2 reduce/reduce conflicts in the parser while
preserving the behaviour of the rules.
Note that now each list with empty elements will only print a warning
per line.
Signed-off-by: Antonio Niño Díaz <[email protected]>
Previously, JR was only allowed to labels (in the same section, or
different sections). When trying to JR to an address specified as a
numeric value, rgbasm would fail to calculate the JR offset (as it
doesn't know the final address of the JR so it can't calculate the
difference).
This patch makes rgblink calculate the offset whenever there is a JR.
Signed-off-by: Antonio Niño Díaz <[email protected]>
2018-03-29 23:38:54 +01:00
2242 changed files with 54953 additions and 22286 deletions
Please ensure that the packages below work properly.
Once that's done, replace this text with the changelog, un-draft the release, and update the `release` branch.
By the way, if you forgot to update `include/version.hpp`, RGBASM's version test is going to fail in the tag's regression testing! (Use `git push --delete origin <tag>` to delete it)
The RGBDS package consists of four programs: RGBASM, RGBLINK, RGBFIX, and RGBGFX.
- RGBASM is the assembler. It takes assembly code as input, and produces an RGB object file as output (and optionally a state file, logging the final state of variables and constants).
- RGBLINK is the linker. It takes object files as input, and produces a ROM file as output (and optionally a symbol and/or map file, logging where the assembly declarations got placed in the ROM).
- RGBFIX is the checksum/header fixer. It takes a ROM file as input, and outputs the same ROM file (or modifies it in-place) with the cartridge header's checksum and other metadata fixed for consistency.
- RGBGFX is the graphics converter. It takes a PNG image file as input, and outputs the tile data, palettes, tilemap, attribute map, and/or palette map in formats that the Game Boy can use.
In the simplest case, a single pipeline can turn an assembly file into a ROM:
This document describes how these four programs are structured. It goes over each source code file, noting which data is *global* (and thus scoped in all files), *owned* by that file (i.e. that is where the data's memory is managed, via [RAII](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization)) or *referenced* by that file (i.e. there are non-owning pointers to some data, and care must be taken to not dereference those pointers after the data's owner has moved or deleted the data).
We assume that the programs are single-threaded; data structures and operations may not be thread-safe.
## Folder Organization
The RGBDS source code file structure is as follows:
```
rgbds/
├── .github/
│ ├── scripts/
│ │ └── ...
│ └── workflows/
│ └── ...
├── cmake/
│ └── ...
├── contrib/
│ ├── bash_compl/
│ ├── zsh_compl/
│ │ └── ...
│ └── ...
├── docs/
│ └── ...
├── include/
│ └── ...
├── man/
│ └── ...
├── src/
│ ├── asm/
│ │ └── ...
│ ├── extern/
│ │ └── ...
│ ├── fix/
│ │ └── ...
│ ├── gfx/
│ │ └── ...
│ ├── link/
│ │ └── ...
│ ├── CMakeLists.txt
│ ├── bison.sh
│ └── ...
├── test/
│ ├── run-tests.sh
│ ├── external/
│ │ ├── fetch-repos.sh
│ │ └── ...
│ └── ...
├── .clang-format
├── .clang-tidy
├── CMakeLists.txt
├── CMakePresets.json
├── Dockerfile
└── Makefile
```
- **`.github/`:**
Files related to the integration of the RGBDS codebase with GitHub features.
* **`scripts/`:**
Scripts used by GitHub Actions workflow files.
* **`workflows/`:**
GitHub Actions CI workflow description files. Used for automated testing, deployment, etc.
- **`cmake/`**:
Files relevant to our CMake build system that are not required to be somewhere else (e.g. `CMakePresets.json`*has* to be at the root).
- **`contrib/`:**
Scripts and other resources which may be useful to RGBDS users and developers.
* **`bash_compl/`:**
Tab completion scripts for use with `bash`. Run them with `source` somewhere in your `.bashrc`, and they should auto-load when you open a shell.
* **`zsh_compl/`:**
Tab completion scripts for use with `zsh`. Put them somewhere in your `fpath`, and they should auto-load when you open a shell.
- **`docs/`:**
Documentation related to the development and maintenance of RGBDS itself.
(The directory's name, as well as some of its file's names, are
[recognized by GitHub](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/setting-guidelines-for-repository-contributors)
and linked to in parts of its UI.)
Documentation for RGBDS itself exists as manual pages in `man/`.
- **`include/`:**
Header files for the respective source files in `src`.
- **`man/`:**
Manual pages to be read with `man`, written in the [`mandoc`](https://mandoc.bsd.lv) dialect.
- **`src/`:**
Source code of RGBDS.
* **`asm/`:**
Source code of RGBASM.
* **`extern/`:**
Source code copied from external sources.
* **`fix/`:**
Source code of RGBFIX.
* **`gfx/`:**
Source code of RGBGFX.
* **`link/`:**
Source code of RGBLINK.
* **`CMakeLists.txt`:**
Defines how to build individual RGBDS programs with CMake, including the source files that each program depends on.
* **`bison.sh`:**
Script used to run the Bison parser generator with the latest flags that the user's version supports.
- **`test/`:**
Testing framework used to verify that changes to the code don't break or modify the behavior of RGBDS.
The `test.sh` scripts inside each of the subdirectories are the individual test drivers.
* **`run-tests.sh`:**
Script used to run tests, including internal test cases and external repositories. `run-tests.sh --help` describes its options.
* **`external/`:**
Directory for third-party repos making use of RGBDS, which get cloned here and built during testing.
- **`fetch-repos.sh`:**
Script used to clone the external repositories. `fetch-repos.sh --help` describes its options.
- **`*.cfg`:**
Each `.cfg` file defines the parameters to clone, build, and verify an external repository.
- **`.clang-format`:**
Code style for automated C++ formatting with [`clang-format`](https://clang.llvm.org/docs/ClangFormat.html) (for which we define the shortcut `make format`).
- **`.clang-tidy`:**
Configuration for C++ static analysis with [`clang-tidy`](https://clang.llvm.org/extra/clang-tidy/) (for which we define the shortcut `make tidy`).
- **`CMakeLists.txt`:**
Defines how to build RGBDS with CMake.
- **`CMakePresets.json`:**
Defines some [presets](https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html) useful for working with our CMake.
- **`Dockerfile`:**
Defines how to build RGBDS with Docker (which we do in CI to provide a [container image](https://github.com/gbdev/rgbds/pkgs/container/rgbds)).
- **`Makefile`:**
Defines how to build RGBDS with `make`, including the source files that each program depends on.
## RGBDS
These files in the `src/` directory are shared across multiple programs: often all four (RGBASM, RGBLINK, RGBFIX, and RGBGFX), sometimes only RGBASM and RGBLINK.
- **`backtrace.cpp`:**
Generic printing of location backtraces for RGBASM and RGBLINK. Allows configuring backtrace styles with a command-line flag (conventionally `-B/--backtrace`). Renders warnings in yellow, errors in red, and locations in cyan.
- **`cli.cpp`:**
A function for parsing command-line options, including RGBDS-specific "at-files" (a filename containing more options, prepended with an "`@`").
This is the only file to use the extern/getopt.cpp variables and functions.
- **`diagnostics.cpp`:**
Generic warning/error diagnostic support for all programs. Allows command-line flags (conventionally `-W`) to have `no-`, `error=`, or `no-error=` prefixes, and `=` level suffixes; allows "meta" flags to affect groups of individual flags; and counts how many total errors there have been. Every program has its own `warning.cpp` file that uses this.
- **`linkdefs.cpp`:**
Constants, data, and functions related to RGBDS object files, which are used for RGBASM output and RGBLINK input.
This file defines two *global* variables, `sectionTypeInfo` (metadata about each section type) and `sectionModNames` (names of section modifiers, for error reporting). RGBLINK may change some values in `sectionTypeInfo` depending on its command-line options (this only affects RGBLINK; `sectionTypeInfo` is immutable in RGBASM).
- **`opmath.cpp`:**
Functions for mathematical operations in RGBASM and RGBLINK that aren't trivially equivalent to built-in C++ ones, such as division and modulo with well-defined results for negative values.
- **`style.cpp`:**
Generic printing of cross-platform colored or bold text. Obeys the [`FORCE_COLOR`](https://force-color.org/) and [`NO_COLOR`](https://no-color.org/) environment variables, and allows configuring with a command-line flag (conventionally `--color`).
- **`usage.cpp`:**
Generic printing of usage information. Renders headings in green, flags in cyan, and URLs in blue. Every program has its own `main.cpp` file that uses this.
- **`util.cpp`:**
Utility functions applicable to most programs, mostly dealing with text strings, such as locale-independent character checks.
- **`verbosity.cpp`:**
Generic printing of messages conditionally at different verbosity levels. Allows configuring with a command-line flag (conventionally `-v/--verbose`).
- **`version.cpp`:**
RGBDS version number and string for all the programs.
## External
These files have been copied ("vendored") from external authors and adapted for use with RGBDS. Both of our vendored dependencies use the same MIT license as RGBDS.
- **`getopt.cpp`:**
Functions for parsing command-line options, including conventional single-dash and double-dash options.
This file defines some *global*`musl_opt*` variables, including `musl_optarg` (the argument given after an option flag) and `musl_optind` (the index of the next option in `argv`). Copied from [musl libc](https://musl.libc.org/).
- **`utf8decoder.cpp`:**
Function for decoding UTF-8 bytes into Unicode code points. Copied from [Björn Höhrmann](https://bjoern.hoehrmann.de/utf-8/decoder/dfa/).
## RGBASM
- **`actions.cpp`:**
Actions taken by the assembly language parser, to avoid large amounts of code going in the parser.y file.
- **`charmap.cpp`:**
Functions and data related to charmaps.
This file *owns* the `Charmap`s in its `charmaps` collection. It also maintains a static `currentCharmap` pointer, and a `charmapStack` stack of pointers to `Charmap`s within `charmaps` (which is affected by `PUSHC` and `POPC` directives).
- **`fixpoint.cpp`:**
Functions for fixed-point math, with configurable [Q*m*.*n*](https://en.wikipedia.org/wiki/Q_(number_format)) precision.
- **`format.cpp`:**
`FormatSpec` methods for parsing and applying format specs, as used by `{interpolations}` and `STRFMT`.
- **`fstack.cpp`:**
Functions and data related to "fstack" nodes (the contents of top-level or `INCLUDE`d files, macro expansions, or `REPT`/`FOR` loop iterations) and their "contexts" (metadata that is only relevant while a node's content is being lexed and parsed).
This file *owns* the `Context`s in its `contextStack` collection. Each of those `Context`s *owns* its `LexerState`, and *refers* to its `FileStackNode`, `uniqueIDStr`, and `macroArgs`. Each `FileStackNode` also *references* its `parent`.
- **`intern.cpp`:**
[Interned strings](https://en.wikipedia.org/wiki/String_interning), which are used for the names of keywords, symbols (labels, constants, variables, string constants, macros, etc), and charmaps.
This file *owns* the strings in its `internedStrings` collection, which its `internedIndexes` collection *references* by view and index.
- **`lexer.cpp`:**
Functions and data related to [lexing](https://en.wikipedia.org/wiki/Lexical_analysis) assembly source code into tokens, which can then be parsed.
This file maintains static `lexerState` and `lexerStateEOL` pointers to `LexerState`s from the `Context`s in `fstack.cpp`.
Each `LexerState`*owns* its `content` and its `expansionStack`'s content. Each `Expansion` (the contents of an `{interpolation}` or macro argument) in turn *owns* its `contents`.
The lexer and parser are interdependent: when the parser reaches certain tokens, it changes the lexer's mode, which affects how characters get lexed into tokens. For example, when the parser reaches a macro name, it changes the lexer to "raw" mode, which lexes the rest of the line as a sequence of string arguments to the macro.
- **`macro.cpp`:**
`MacroArgs` methods related to macro arguments. Each `MacroArgs`*references* its arguments' contents.
- **`main.cpp`:**
The `main` function for running RGBASM, including the initial handling of command-line options.
This file defines a *global*`options` variable with the parsed CLI options.
- **`opt.cpp`:**
Functions for parsing options specified by `OPT` or by certain command-line options.
This file *owns* the `OptStackEntry`s in its `stack` collection (which is affected by `PUSHO` and `POPO` directives).
- **`output.cpp`:**
Functions and data related to outputting object files (with `-o/--output`) and state files (with `-s/--state`).
This file *owns* its `assertions` (created by `ASSERT` and `STATIC_ASSERT` directives). Every assertion gets output in the object file.
This file also *references* some `fileStackNodes`, and maintains static pointers to `Symbol`s in `objectSymbols`. Only the "registered" symbols and fstack nodes get output in the object file. The `fileStackNodes` and `objectSymbols` collections keep track of which nodes and symbols have been registered for output.
- **`parser.y`:**
Grammar for the RGBASM assembly language, which Bison preprocesses into a [LALR(1) parser](https://en.wikipedia.org/wiki/LALR_parser).
The Bison-generated parser calls `yylex` (defined in `lexer.cpp`) to get the next token, and calls `yywrap` (defined in `fstack.cpp`) when the current context is out of tokens and returns `EOF`.
- **`rpn.cpp`:**
`Expression` methods and data related to "[RPN](https://en.wikipedia.org/wiki/Reverse_Polish_notation)" expressions. When a numeric expression is parsed, if its value cannot be calculated at assembly time, it is built up into a buffer of RPN-encoded operations to do so at link time by RGBLINK. The valid RPN operations are defined in [man/rgbds.5](/man/rgbds.5).
- **`section.cpp`:**
Functions and data related to `SECTION`s.
This file *owns* the `Section`s in its `sections` collection. It also maintains various static pointers to those sections, including the `currentSection`, `currentLoadSection`, and `sectionStack` (which is affected by `PUSHS` and `POPS` directives). (Note that sections cannot be deleted.)
- **`symbol.cpp`:**
Functions and data related to symbols (labels, constants, variables, string constants, macros, etc).
This file *owns* the `Symbol`s in its `symbols` collection, and the various built-in ones outside that collection (`PCSymbol` for "`@`", `NARGSymbol` for "`_NARG`", etc). It also maintains a static `purgedSymbols` collection to remember which symbol names have been `PURGE`d from `symbols`, for error reporting purposes.
- **`warning.cpp`:**
Functions and data for warning and error output.
This file defines a *global*`warnings` variable using the `diagnostics.cpp` code for RGBASM-specific warning flags.
## RGBFIX
- **`fix.cpp`:**
Functions for fixing the ROM header.
- **`main.cpp`:**
The `main` function for running RGBFIX, including the initial handling of command-line options.
This file defines a *global*`options` variable with the parsed CLI options.
- **`mbc.cpp`:**
Functions and data related to [MBCs](https://gbdev.io/pandocs/MBCs.html), including the names of known MBC values.
- **`warning.cpp`:**
Functions and data for warning and error output.
This file defines a *global*`warnings` variable using the `diagnostics.cpp` code for RGBFIX-specific warning flags.
## RGBGFX
- **`color_set.cpp`:**
`ColorSet` methods for creating and comparing sets of colors. A color set includes the unique colors used by a single tile, and these sets are then packed into palettes.
- **`main.cpp`:**
The `main` function for running RGBGFX, including the initial handling of command-line options.
This file defines a *global*`options` variable with the parsed CLI options.
- **`pal_packing.cpp`:**
Functions for packing color sets into palettes. This is done with an ["overload-and-remove" heuristic](https://arxiv.org/abs/1605.00558) for a pagination algorithm.
- **`pal_sorting.cpp`:**
Functions for sorting colors within palettes, which works differently for grayscale, RGB, or indexed-color palettes.
- **`pal_spec.cpp`:**
Functions for parsing various formats of palette specifications (from `-c/--colors`).
- **`palette.cpp`:**
`Palette` methods for working with up to four GBC-native (RGB555) colors.
- **`png.cpp`:**
`Png` methods for reading PNG image files, standardizing them to 8-bit RGBA pixels while also reading their indexed palette if there is one.
- **`process.cpp`:**
Functions related to generating and outputting files (tile data, palettes, tilemap, attribute map, and/or palette map).
- **`reverse.cpp`:**
Functions related to reverse-generating RGBGFX outputs into a PNG file (for `-r/--reverse`).
- **`rgba.cpp`:**
`Rgba` methods related to RGBA colors and their 8-bit or 5-bit representations.
- **`warning.cpp`:**
Functions and data for warning and error output.
This file defines a *global*`warnings` variable using the `diagnostics.cpp` code for RGBGFX-specific warning flags.
## RGBLINK
- **`assign.cpp`:**
Functions and data for assigning `SECTION`s to specific banks and addresses.
This file *owns* the `memory` table of free space: each section type is associated with a list of each bank's free address ranges, which are allocated to sections using a [first-fit decreasing](https://en.wikipedia.org/wiki/Bin_packing_problem#First-fit_algorithm) bin-packing algorithm.
- **`fstack.cpp`:**
Functions related to "fstack" nodes (the contents of top-level or `INCLUDE`d files, macro expansions, or `REPT`/`FOR` loop iterations) read from the object files. At link time, these nodes are only needed for printing of location backtraces.
- **`layout.cpp`:**
Actions taken by the linker script parser, to avoid large amounts of code going in the script.y file.
This file maintains some static data about the current bank and address layout, which get checked and updated for consistency as the linker script is parsed.
- **`lexer.cpp`:**
Functions and data related to [lexing](https://en.wikipedia.org/wiki/Lexical_analysis) linker script files into tokens, which can then be parsed.
This file *owns* the `LexerStackEntry`s in its `lexerStack` collection. Each of those `LexerStackEntry`s *owns* its `file`. The stack is updated as linker scripts can `INCLUDE` other linker script pieces.
The linker script lexer is simpler than the RGBASM one, and does not have modes.
- **`main.cpp`:**
The `main` function for running RGBLINK, including the initial handling of command-line options.
This file defines a *global*`options` variable with the parsed CLI options.
- **`object.cpp`:**
Functions and data for reading object files generated by RGBASM.
This file *owns* the `Symbol`s in its `symbolLists` collection, and the `FileStackNode`s in its `nodes` collection.
- **`output.cpp`:**
Functions and data related to outputting ROM files (with `-o/--output`), symbol files (with `-n/--sym`), and map files (with `-m/--map`).
This file *references* some `Symbol`s and `Section`s, in collections that keep them sorted by address and name, which allows the symbol and map output to be in order.
- **`patch.cpp`:**
Functions and data related to "[RPN](https://en.wikipedia.org/wiki/Reverse_Polish_notation)" expression patches read from the object files, including the ones for `ASSERT` conditions. After sections have been assigned specific locations, the RPN patches can have their values calculated and applied to the ROM. The valid RPN operations are defined in [man/rgbds.5](/man/rgbds.5).
This file *owns* the `Assertion`s in its `assertions` collection, and the `RPNStackEntry`s in its `rpnStack` collection.
- **`script.y`:**
Grammar for the linker script language, which Bison preprocesses into a [LALR(1) parser](https://en.wikipedia.org/wiki/LALR_parser).
The Bison-generated parser calls `yylex` (defined in `lexer.cpp`) to get the next token, and calls `yywrap` (also defined in `lexer.cpp`) when the current context is out of tokens and returns `EOF`.
- **`sdas_obj.cpp`:**
Functions and data for reading object files generated by [GBDK with SDCC](https://gbdk.org/). RGBLINK support for these object files is incomplete.
- **`section.cpp`:**
Functions and data related to `SECTION`s read from the object files.
This file *owns* the `Section`s in its `sections` collection.
- **`symbol.cpp`:**
Functions and data related to symbols read from the object files.
This file *references* the `Symbol`s in its `symbols` and `localSymbols` collections, which allow accessing symbols by name.
- **`warning.cpp`:**
Functions and data for warning and error output.
This file defines a *global*`warnings` variable using the `diagnostics.cpp` code for RGBLINK-specific warning flags.
2. Make sure that you have installed `groff` and `mandoc`. You will
need `mandoc` 1.14.5 or later to support `-O toc`.
3. Inside of the `man` directory, run <code><i><path/to/rgbds-www></i>/maintainer/man_to_html.sh <i><tag></i> *</code> then <code><i><path/to/rgbds-www></i>/maintainer/new_release.sh <i><tag></i></code>.
This will render the RGBDS documentation as HTML and PDF and copy it to
`rgbds-www`.
If you do not have `groff` installed, you can change
`groff -Tpdf -mdoc -wall` to `mandoc -Tpdf -I os=Linux` in
/* MS Visual Studio 2003/.NET Framework 1.1 or newer */
#define noreturn _declspec(noreturn)
#else
/* Unsupported, but no need to throw a fit */
#define noreturn
#endif
#endif /* EXTERN_STDNORETURN_H */
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Eldred Habert
Rangi
Eldred Habert
Rangi
vulcandth
ReiquelApplegate
Antonio Vivace
John Millikin
robbi-blechdose
Quinn
JL2210
Rangi42
sukus
Ruben Zwietering
Chris Pickel
orbea
SnDream
Evie
Marcus Huderle
Ron Nelson
Damian Yerrick
Damian YerrickandRangi
Tian(Maxwell) Yang
Antonio Vivace
rlewicki
EievuiandISSOtm
DaKnig
aaaaaa123456789
martendoandEldred Habert
Jakub KądziołkaandISSOtm
GreenAndEievui
daid
dannye
anderoonies
Brigham Campbell
Antonio Niño Díaz
Marco Spataro
Matt Currie
Brayden Morris
Jason Yuan
Trung Lê
dbrotz
clach04
Anthony J. Bentley
Ben Hetherington
jidoc01
Quint Guvernator
Phil Smith
jmle
Simon Harms
mid-kid
karas
yenatch
Ben10do