They were expanded during the capture, and there was no easy way to
avoid expanding them (believe me, after three hours and somehow an OOM, I
gave up trying).
The biggest problem was simply that the length of children expansions was
not accounted for when skipping over the parent... this took a lot of
arduous debugging, but it finally works!
And fix line counting with expansion-made newlines.
This has the same bug as the old lexer (equs-newline's output does not
print the second warning as being part of the expansion).
Additionally, we regress equs-recursion, as we are no longer able to
catch this specific EQUS recursion. Simply enough, the new expansion
begins **after** the old one ends! I have found no way to handle that.
Add keywords and identifiers
Add comments
Add number literals
Add strings
Add a lot of new tokens
Add (and clean up) IF etc.
Improve reporting of unexpected chars / garbage bytes
Fix bug with and improved error messages when failing to open file
Add verbose-level messages about how files are opened
Enforce that files finish with a newline
Fix chars returned not being cast to unsigned char (may conflict w/ EOF)
Return null path when no file is open, rather than crash
Unify and improve error printing slightly
Known to be missing: macro expansion, REPT blocks, EQUS expansions
Create a new file, platform.h, for platform-specific hacks
for MSVC, this includes defining strncasecmp to _stricmp and
strdup to _strdup, among other things like defining missing
stat macros
Change some things not supported in MSVC, like _Static_assert,
to their counterparts (in this case, static_assert)
Replace usage of VLAs with malloc and free
Update getopt_long and use the getopt implementation from musl
on Windows.
Use comments to show which functions from platform.h are being used
This should help make RGBDS portable to systems with 16-bit integers,
like DOS.
For kicks, use the macros for 16-bit and 8-bit integers.
Fix other miscellaneous things, like #include ordering and other
printf-format related things.
Reduce repitition in math.c while I'm there.
It's possible that if the FILE passed to yy_create_buffer is at the
end-of file, there may be a null pointer dereference.
This should hopefully fix that.
Found with clang-tools' scan-build:
src/asm/lexer.c:281:25: warning: Array access (via field 'pBuffer')
results in a null pointer dereference
pBuffer->pBuffer[size] = 0;
~~~~~~~ ^
1 warning generated.
Signed-off-by: JL2210 <larrowe.semaj11@gmail.com>
This should significantly improve performance: on pokecrystal builds, perf
reported as much CPU time spent on `yyparse` as on `sym_UseNewMacroArgs`
Measurements show ~6 seconds of improvement on that codebase.
This also fixes#321, as a bonus, due to saner management!
This avoids redundancy between them (and also having to port fixes and features)
The error messages have been preserved through a string reporting mechanism
Macro and rept buffers were not always being terminated with newlines
and/or were vulnerable to the final newline being escaped, allowing
buffer overflows to occur. Now, they are terminated with newlines using
the same mechanism as the file buffer.
Null characters in the middle of strings interact badly with the RGBDS
codebase, which assumes null-terminated strings. There is no reason to
support null characters in input source code, so the simplest way to deal
with null characters is to reject them early.
Unlike macros, REPTs and INCLUDEs, this recursion depth is independent.
This is intentional, because string expansions work very differently.
While it's easy to know when a string expansion begins, checking where it
ends is much more complicated, since the expansion's contents are simply
injected back into the lex buffer. Therefore, the depth has to be checked
after lexing took place.
Because of this, the placement of the expansion end check is somewhat
haphazard, but I think it's good. While I have no certainty, all tests
ended with all expansions properly ended, and I couldn't find any pitfalls.
Finally, `pCurrentStringExpansion` has been made global so error printing
can use it to tell the user if an error occurred inside of an expansion.
