Add keywords and identifiers
Add comments
Add number literals
Add strings
Add a lot of new tokens
Add (and clean up) IF etc.
Improve reporting of unexpected chars / garbage bytes
Fix bug with and improved error messages when failing to open file
Add verbose-level messages about how files are opened
Enforce that files finish with a newline
Fix chars returned not being cast to unsigned char (may conflict w/ EOF)
Return null path when no file is open, rather than crash
Unify and improve error printing slightly
Known to be missing: macro expansion, REPT blocks, EQUS expansions
This should significantly improve performance: on pokecrystal builds, perf
reported as much CPU time spent on `yyparse` as on `sym_UseNewMacroArgs`
Measurements show ~6 seconds of improvement on that codebase.
This also fixes#321, as a bonus, due to saner management!
This is especially useful when an EQUS expands to another one, to help
track them.
This is done separately from the file stack as the EQUS stack is separate
(which is itself because EQUS are managed *way* differently).
The old error stack was fairly obtuse and hard to use for debugging.
This improves it notably by ensuring all line numbers are relative
to the file, and not, say, the macro definition.
This is a breaking change if you were parsing the old stack, but
the change should be painless, and the new stack only brings more info.
The syntax is unchanged for files, macros see their name prefixed
with the file they're defined in and a pair of colors, REPT blocks
simply append a '::REPT~n' to the context they're in, where 'n' is
the number of iterations the REPT has done.
This is especially helpful in macro-heavy code such as rgbds-structs.
Unlike macros, REPTs and INCLUDEs, this recursion depth is independent.
This is intentional, because string expansions work very differently.
While it's easy to know when a string expansion begins, checking where it
ends is much more complicated, since the expansion's contents are simply
injected back into the lex buffer. Therefore, the depth has to be checked
after lexing took place.
Because of this, the placement of the expansion end check is somewhat
haphazard, but I think it's good. While I have no certainty, all tests
ended with all expansions properly ended, and I couldn't find any pitfalls.
Finally, `pCurrentStringExpansion` has been made global so error printing
can use it to tell the user if an error occurred inside of an expansion.
(And REPT.)
Not exactly a *recursion* limit, more like a *stack depth* limit,
but calling it "recursion" conveys its purpose better.
The default of 64 is super overkill: even in a a project with
what I believe to be above-average levels of nesting, the
level only peaked at 6.
Keeping in mind the purpose of this is to catch infinite
recursion, which is still caught quickly (in usual cases, anyways),
this default seems sensible.
And it passes tests. What more do you need?
With permission from the main authors [1], most of the code has been
relicensed under the MIT license.
SPDX license identifiers are used so that the license headers in source
code files aren't too large.
Add CONTRIBUTORS.rst file.
[1] https://github.com/rednex/rgbds/issues/128
Signed-off-by: Antonio Niño Díaz <antonio_nd@outlook.com>
Not all occurrences have been replaced, in some cases they have been
left as they were before (like in rgbgfx and when they are in the
interface of a C standard library function).
Signed-off-by: Antonio Niño Díaz <antonio_nd@outlook.com>
