Skip to content

Fix: simplecpp ## fails to expand function-like macro when '(' is not adjacent - #655

Open
paulafy6 wants to merge 2 commits into
cppcheck-opensource:masterfrom
paulafy6:fix-preprocessor-concat
Open

Fix: simplecpp ## fails to expand function-like macro when '(' is not adjacent#655
paulafy6 wants to merge 2 commits into
cppcheck-opensource:masterfrom
paulafy6:fix-preprocessor-concat

Conversation

@paulafy6

@paulafy6 paulafy6 commented May 27, 2026

Copy link
Copy Markdown

When the ## operator concatenates two tokens to form a function-like macro name (e.g. PREFIX_ ## kind → PREFIX_SCALAR), simplecpp looked for the argument list '(...)' only at B->next. In PAR-style indirection patterns the '(' is separated from B by a comma or a variadic parameter token:

  #define PAR(a, ...) a __VA_ARGS__
  #define PREFIX_SCALAR(T, N) T N
  #define DISPATCH(kind, ...) PAR(PREFIX_ ## kind, (__VA_ARGS__))
  DISPATCH(SCALAR, int, x)   // was: [unknownMacro] — now: int x

Because '(' was not found, expansion was aborted and the macro was reported as unknownMacro, causing cppcheck to skip the entire translation unit.

Fix: when B->next is not '(' and we are in the appendTokens context (expandResult==false), walk forward on the same line skipping ',' separators and resolving named parameter tokens via expandArg(). The first '(' found (literally or as the head of an expanded argument) is used as lpar and passed to appendTokens() as before. The forwardScan flag ensures expandToken() is called on the result even when expandResult is false.

The forward scan is restricted to expandResult==false to avoid unintended side-effects in the main expansion loop.

@danmar

danmar commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the contribution!

@firewave

Copy link
Copy Markdown
Collaborator

No need to repeat the whole issue description and what the fix does in the PR message as well.

I know AI loves to do that but it often gets out-of-sync if the code in the PR or the issue description changes (it did in this case) and might end up misleading (redundancies are never great).

Comment thread simplecpp.cpp Outdated
// Only active in the appendTokens context (expandResult==false) to
// avoid unintended side-effects inside the main expansion loop.
const Token *scan = nextTok;
while (scan && sameline(B, scan)) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this scan && is redundant it's done in the sameline function.

Suggested change
while (scan && sameline(B, scan)) {
while (sameline(B, scan)) {

Comment thread simplecpp.cpp Outdated
// and is separated from B by a comma in the replacement text.
// Only active in the appendTokens context (expandResult==false) to
// avoid unintended side-effects inside the main expansion loop.
const Token *scan = nextTok;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since scan is only used in the while loop, I believe it would be nicer to write a for loop.

@paulafy6
paulafy6 force-pushed the fix-preprocessor-concat branch from ffb123e to 130ee4c 比较 August 18, 2026 08:56
… adjacent

When the ## operator concatenates two tokens to form a function-like macro
name (e.g. PREFIX_ ## kind → PREFIX_SCALAR), simplecpp looked for the
argument list '(...)' only at B->next. In PAR-style indirection patterns
the '(' is separated from B by a comma or a variadic parameter token:

  #define PAR(a, ...) a __VA_ARGS__
  #define PREFIX_SCALAR(T, N) T N
  #define DISPATCH(kind, ...) PAR(PREFIX_ ## kind, (__VA_ARGS__))
  DISPATCH(SCALAR, int, x)   // was: [unknownMacro] — now: int x

Because '(' was not found, expansion was aborted and the macro was
reported as unknownMacro, causing cppcheck to skip the entire translation
unit.

Fix: when B->next is not '(' and we are in the appendTokens context
(expandResult==false), walk forward on the same line skipping ',' separators
and resolving named parameter tokens via expandArg(). The first '(' found
(literally or as the head of an expanded argument) is used as lpar and
passed to appendTokens() as before. The forwardScan flag ensures
expandToken() is called on the result even when expandResult is false.

The forward scan is restricted to expandResult==false to avoid unintended
side-effects in the main expansion loop.
…like_par_indirection)

Covers the fix in expandHashHash(): when ## concatenation produces a
function-like macro name but '(' is not immediately adjacent in the
replacement text (hidden behind a comma/parameter), the forward scan
must locate '(' and complete the expansion.

  #define PAR(a, ...) a __VA_ARGS__
  #define PREFIX_SCALAR(T, N) T N
  #define DISPATCH(kind, ...) PAR(PREFIX_ ## kind, (__VA_ARGS__))
  DISPATCH(SCALAR, int, x)   // expected: int x
@paulafy6
paulafy6 force-pushed the fix-preprocessor-concat branch from 130ee4c to a2f6f52 比较 August 18, 2026 09:06
@paulafy6

Copy link
Copy Markdown
Author

Sorry for the delay, I just took into account your review and pushed force the changes.

@danmar

danmar commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

I asked AI to review this. My experience is that AI reviews are almost always wrong, so take it with a mountain of salt. But please look at it and write your response..

What it claims to fix: when ## concatenates two tokens into a function-like macro name, and the ( for that macro's call is separated from it by
a comma (e.g. PAR(PREFIX_ ## kind, (VA_ARGS))), simplecpp aborts expansion instead of finding the (.

The motivating bug is real, and the target behavior is correct. I verified against gcc and clang:

#define PAR(a, ...) a VA_ARGS
#define PREFIX_SCALAR(T, N) T N
#define DISPATCH(kind, ...) PAR(PREFIX_ ## kind, (VA_ARGS))
DISPATCH(SCALAR, int, x)

  • gcc/clang: int x
  • master (built from HEAD): PREFIX_SCALAR ( int , x ) — under-expanded
  • PR branch: int x — matches

This is legitimate standard-mandated behavior: PREFIX_ ## kind concatenates during argument substitution of DISPATCH, producing
PAR(PREFIX_SCALAR, (int, x)); rescanning that expands PAR to PREFIX_SCALAR (int, x); rescanning that finally invokes PREFIX_SCALAR. It's
ordinary multi-level rescanning (per 6.10.3.4), not something specific to ##.

However, the fix itself is not a correct implementation of that rescanning — it's a narrow pattern match that silently corrupts output on a
nearby, equally valid variant. The new code, when it doesn't find ( immediately after the concatenated token, walks forward on the same line
skipping , and expanding named tokens, looking for any (. It grabs whatever parenthesized thing it finds and treats it as the call's argument
list, without regard for which macro invocation that ( structurally belongs to.

Counter-example — just change PAR's body to put a comma between its parameters (equally valid C):

#define PAR2(a, ...) a, VA_ARGS
#define PREFIX_SCALAR(T, N) T N
#define DISPATCH2(kind, ...) PAR2(PREFIX_ ## kind, (VA_ARGS))
DISPATCH2(SCALAR, int, x)

  • gcc/clang: PREFIX_SCALAR, (int, x) (correct — no adjacency, so no call)
  • master: PREFIX_SCALAR , ( int , x ) — matches gcc/clang
  • PR branch: int x , — wrong, and silently corrupted (no error at all)

Root cause: the forward scan in DISPATCH2's own replacement list sees PREFIX_ ## kind (→PREFIX_SCALAR), skips the comma, and grabs
(VA_ARGS) — but that (...) is actually PAR2's second argument, not call-parens for PREFIX_SCALAR. It swallows that argument into a
synthetic PREFIX_SCALAR(int, x) call, which fully expands to int x, leaving PAR2 invoked with only one argument and an empty variadic tail —
hence the mangled int x ,.

So the heuristic happens to work for the PR's own test cases only because PAR's body (a VA_ARGS) puts the substituted pieces directly
adjacent with no comma — it isn't simulating standard rescanning, it's guessing based on the concatenation site's local token layout, which has
no reliable relationship to where the eventual ( ends up after the enclosing macro's own substitution.

I confirmed the existing test suite (testrunner) still passes on the PR branch, and it compiles clean under -Wall -Wextra -pedantic -Werror —
but the PR's own regression test doesn't cover this case, so the corruption goes undetected.

Conclusion: the bug being fixed is real and the desired output is standards-correct, but this specific fix is unsound — it introduces a
silent-corruption regression for macros structurally very close to the motivating example. I'd recommend not merging as-is. A more robust
approach would extend the existing generic rescan machinery (rescanMacro/recursiveExpandToken from #682) so the
concatenated-name-followed-later-by-( case is handled through genuine iterative rescanning rather than an ad hoc local comma-skipping scan that
can't tell which enclosing call a ( belongs to.

注册 for free to join this conversation on GitHub. Already have an account? 登录 to comment

标签

None yet

项目

None yet

Development

Successfully merging this pull request may close these issues.

expandHashHash(): function-like macro not expanded when '(' is not adjacent (PAR-style indirection)

3 participants