075f132d77
Use a smaller unordered_map just for looking up line-leading keywords
2026-05-27 22:01:20 -04:00
31e1d2ec87
Do not build up a std::string when skipping leading keywords
2026-05-27 22:01:20 -04:00
007672f080
Intern identifier strings only after checking for keywords
2026-05-25 23:55:57 -04:00
55db252a8f
Simplify the lexer by removing BufferedContent and inlining ViewedContent ( #1981 )
...
Instead of reading 64 characters at a time into a rolling buffer
as `shiftChar()` is called, we read 8192 characters at a time into
a complete buffer before any `peek()`/`shiftChar()` operations.
2026-05-25 21:05:59 -04:00
cfa0adf295
Intern strings used as identifiers (for labels, constants, macros, charmaps, etc) ( #1980 )
2026-05-25 16:22:29 -04:00
4b992bfea5
Use the name of sym_GetPC() instead of hard-coding "@" again
2026-05-25 14:21:14 -04:00
141a7fe22b
Add assume checks for expected Token types, like RPNValue has for RPNCommands
2026-05-25 14:14:34 -04:00
090768e2c9
Some Expansion references can be const
2026-05-23 13:06:09 -04:00
4a173130b9
Optimize skipToLeadingKeyword for the common ViewedContent case ( #1968 )
2026-05-22 19:06:12 -04:00
dce14fd4b8
Use C++20 concepts to require constraints on template parameters ( #1977 )
2026-05-22 17:54:34 -04:00
48fcd9a0ca
Use templates to reduce the redundant number-lexing functions ( #1963 )
2026-05-21 23:13:09 +02:00
c50a849757
Verbose tracing shows lexed and captured values ( #1969 )
2026-05-09 12:01:29 -04:00
f0161b41c8
Update readHexNumber for consistency with other routines
2026-04-29 12:31:00 +02:00
ca23210f18
Refactor and rename some numeric literal lexing for asm and linkerscript consistency
2026-04-29 12:15:14 +02:00
85e044f5be
Use std::move when relevant in Token constructor
2026-04-28 13:32:45 +02:00
60e76b2da4
Avoid UB if new OOMs by removing std:nothrow
2026-04-28 13:28:32 +02:00
3c7488c131
Some refactoring, renaming, and debug assertions for clarity and safety
...
- Add more `assume()` checks in the lexer
- Replace double negative "`disable* = false`" with "`enable* = true`"
- Naming convention of "`std::deque<> *Stack`" when relying on
`.push_front()` and `.pop_back()`
2026-04-27 18:47:06 +02:00
9fd0c0297f
Fix invalid character in bracketed macro argument
...
The invalid character should not be consumed by `shiftChar()`.
2026-04-27 17:11:20 +02:00
c1c7e64249
Add two assume calls to verify lexer arguments
2026-04-27 15:35:10 +02:00
a18b2f1049
Consistently lex local labels after keywords, even when skipping/capturing
2026-04-27 15:28:30 +02:00
eeb3a73210
Small optimization to skipToLeadingKeyword
2026-04-27 14:26:47 +02:00
cfec017fed
Consolidate, refactor, and bugfix the lexer's handling of captures and skips ( #1957 )
...
- Do not error about local labels following keywords in skips or captures (fixes #1955 )
- Do not incompletely attempt to handle line continuations in skips (fixes #1956 )
- Rename `skipToLeadingIdentifier` to `skipToLeadingKeyword`, refactor to merge
`skipToEOL` into it, and use it for both skips and captures
2026-04-20 13:04:20 +02:00
6b994b1737
Fix some warnings raised by MSVC
2026-04-16 00:57:33 +02:00
df0b557638
Don't allow invalid interpolations to occur ( #1925 )
...
Fixes #1921
2026-04-11 18:16:07 +02:00
9b4b4a581b
Check for overflow in fixed-point precision suffix ( #1918 )
...
Fixes #1917
2026-04-07 17:06:54 -04:00
11f6278d95
Refactor lexing of fixed-point numbers ( #1915 )
...
This incidentally fixes a bug with too-long fixed-point literals
that have precision suffixes.
2026-04-06 21:45:34 -04:00
ede9405daf
Fix some fixed-point literal lexing issues ( #1914 )
...
- No fractional digits are necessary after the decimal point,
e.g. `42.` is valid instead of `42.0`
- Error messages refer to "fixed-point" not "integer" constants
- Test more carefully for lexing unrelated to underscores
2026-04-04 13:41:45 -04:00
66e521e9e5
Factor out common error-handling functions for lexing numbers of different bases ( #1888 )
2026-03-19 15:32:25 -04:00
752e2b3620
Symbol names with more than two '.'s could be defined as constants
...
Dot-only names could also trip an assertion in `make develop`
when used as labels
2025-12-04 15:15:41 -05:00
ad3188f038
Fix garbage characters at EOF causing an infinite loop
2025-12-04 15:15:41 -05:00
df5162edca
Use loops instead of tail calls and musttail
...
gcc 15.2.1 20250813 complains "address of automatic variable can
escape to `musttail` call" from `-Wmaybe-musttail-local-addr`,
and guaranteeing tail-call optimization cross-platform is more
trouble than it's worth.
2025-10-27 12:05:27 -04:00
2873e0b8c8
Use musttail attribute to guarantee tail recursion ( #1849 )
2025-10-20 15:56:22 -04:00
23b9039716
Give clearer names to template parameters
2025-10-08 14:55:43 -04:00
089fc11e31
A local label starting with a keyword (e.g. jr.local) is an error
2025-10-08 12:23:08 -04:00
538395b92c
Prevent simple syntax highlighters from breaking on /* in a string
2025-09-29 11:11:40 -04:00
634fd853d1
Factor out a single parseNumber utility function ( #1839 )
2025-09-22 15:15:24 -04:00
d8aff148bb
Factor out RRANGE macro like RANGE
2025-09-19 16:53:44 -04:00
e31bcabbaa
Implement === and !== string comparison operators ( #1832 )
2025-09-19 14:06:36 -04:00
67741ab428
Trim left space from macro args, even past block comments ( #1831 )
2025-09-19 13:44:18 -04:00
1dfc1d3231
Factor out isBinDigit and parseHexDigit utility functions
2025-09-04 13:23:10 -04:00
cc96b4d517
Two small improvements
...
- Check whether `.read()` completed
- `.reserve()` expected space ahead of time
2025-09-01 11:46:41 -04:00
0ccdbf509a
Simplify format specs to not use a per-character state machine
2025-08-30 12:23:01 -04:00
531278961f
Require underscores to actually be digit separators ( #1812 )
...
Multiple, trailing, or next to decimal point are errors
2025-08-30 10:44:20 -04:00
fcfc931867
Shorten lexing of simple tokens
2025-08-20 22:37:03 -04:00
3d155d5695
Some refactoring and cleanup ( #1806 )
...
* Use clang-tidy `misc-include-cleaner` for IWYU `#include` cleanup
* Use `std::optional<size_t>` instead of `ssize_t`
* Rename some functions in linkdefs.hpp
* Fix header order
2025-08-20 16:09:04 -04:00
386fb5f398
Add more character utility functions
2025-08-19 19:17:40 -04:00
94e9ef5213
Factor shared code out of lexer_CaptureRept and lexer_CaptureMacro
2025-08-19 18:11:50 -04:00
b7e0783ae7
Implement ? suffix to "quiet" a context and exclude it from backtraces ( #1800 )
2025-08-18 21:34:58 -04:00
272019beb0
Fix line numbers from nested expansions ( #1802 )
2025-08-14 11:13:50 -04:00
db6793f444
Don't count single quote ' as garbage ( #1801 )
...
Also copy the "blank space" (space or tab) vs "whitespace" (space,
tab, or newline) convention from `<ctype.h>`
2025-08-14 10:10:59 -04:00
Rangi
Rangi
ISSOtm