fix(cypher): refuse a query the parser did not fully read - #1875
Open
CaptainMittens wants to merge 1 commit into
Open
fix(cypher): refuse a query the parser did not fully read#1875CaptainMittens wants to merge 1 commit into
CaptainMittens wants to merge 1 commit into
Conversation
cbm_parse built a query and returned success without checking that it had read every token. The grammar accepts at most one WITH and treats RETURN as optional, so the parser stopped at the first thing it did not understand and reported success anyway. The dropped tail took the filter and the RETURN with it. The engine then answered from the fragment it had parsed, using its default projection. It reported success and returned wrong rows, which is worse than a refusal, because nothing tells the caller to look. Two shapes hit this: MATCH (f:Function) WHERE f.name = 'x' RETURN f.name AS n BANANA SPLIT 99 -> one row, no error, the trailing words silently dropped MATCH (f:Function) OPTIONAL MATCH (a)-[:CALLS]->(f) WITH f, count(a) AS calls OPTIONAL MATCH (b)-[:USAGE]->(f) WITH f, calls, count(b) AS usages WHERE calls = 0 AND usages = 0 RETURN f.name AS n -> every function, unfiltered, under the default column names cbm_parse now checks that the cursor sits on the end-of-input token before it returns success, and names the leftover token when it does not. The message also says that only one WITH clause is supported, because that is the limit people actually meet. The check exposed a second defect. parse_post_where parses the branch after UNION by calling cbm_parse on a slice of the same tokens, using a separate parser. The outer parser's cursor never moved past the UNION keyword, so a valid UNION query looked unfinished. That cursor was already wrong; nothing caught it, because nothing checked where the cursor ended up. The UNION branch now moves the cursor to the end after the sub-parse succeeds, which is sound because that sub-parse no longer returns success with tokens left over. Three tests cover both directions. Removing only the guard turns the two rejection tests red with rc == 0 -- the parser reporting success on input it never finished reading -- while the acceptance test stays green, so the guard is load-bearing for exactly these two behaviours. Verified on macOS with Apple clang: make -f Makefile.cbm test-focused TEST_SUITES=cypher -> 186 passed guard removed, tests kept -> 184 passed, 2 failed make -f Makefile.cbm cbm -> exit 0, no warnings The full suite reports 7623 passed, 2 failed. Both failures are in tests/test_cli.c (lines 1748 and 6723) and reproduce on a clean tree with this change stashed out. They depend on the coding agents installed on the machine, not on this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
注册 for free
to join this conversation on GitHub.
Already have an account?
登录 to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
cbm_parsebuilt a query and returned success without checking that it hadread every token. The grammar accepts at most one
WITHand treatsRETURNas optional, so the parser stopped at the first thing it did not understand
and reported success anyway.
The dropped tail took the filter and the
RETURNwith it. The engine thenanswered from the fragment it had parsed, using its default projection. It
reported success and returned wrong rows — which is worse than a refusal,
because nothing tells the caller to look.
Reproductions (both on
main, before this change)Returns one row, no error. The trailing words are silently dropped.
Returns every function in the project, unfiltered, under the default column
names rather than the requested alias. I hit this one for real while hunting
dead code: it returned 223 rows including functions I had just proven had
callers. The only clue anything was wrong was the column names — the second
WITHand everything after it had been dropped, andWHERE calls = 0neverran.
That second case is the reason I am filing this. A query that refuses is a
minor annoyance. A query that answers confidently with the wrong rows sends
you off to act on data that was never filtered.
The fix
cbm_parsenow checks that the cursor sits on the end-of-input token beforereturning success, and names the leftover token when it does not. The message
also states that only one
WITHclause is supported, since that is the limitpeople actually meet in practice.
Why this touches UNION
This is the part worth reviewing hardest, and it is not scope creep — the
check could not land without it.
parse_post_whereparses the branch afterUNIONby callingcbm_parseon aslice of the same token array, using a separate parser struct. The outer
parser's cursor therefore never moved past the
UNIONkeyword. That cursorwas already wrong on
maintoday; nothing caught it, because nothing checkedwhere the cursor ended up. Adding the end-of-input check is what exposed it —
three
UNIONtests went red the first time I ran the suite.The
UNIONbranch now moves the outer cursor to the end after the sub-parsesucceeds. That is sound only because the recursive
cbm_parseno longerreturns success with tokens left over, so the two changes depend on each
other and neither is safe alone.
Tests
Three tests in
tests/test_cypher.c, covering both directions:cypher_parse_rejects_trailing_tokensBANANA SPLIT 99case is an errorcypher_parse_rejects_second_with_clausecypher_parse_accepts_single_with_clauseWITH+WHERE+RETURNstill parsesThe third one is the control. A guard like this is easy to over-tighten, and
without it a green suite would not distinguish "still accepts valid queries"
from "started rejecting everything".
Red-green evidence
Removing only the guard, keeping the tests:
rc == 0is the defect itself:cbm_parsereporting success on input itnever finished reading. Restoring the guard returns 186 passed. The control
test passes in both states, so the guard is load-bearing for exactly these two
behaviours and nothing else.
A note on the full suite
make -f Makefile.cbm testreports 7623 passed, 2 failed, 8 skipped on mymachine (macOS 15, Apple clang). Both failures are in
tests/test_cli.c(lines 1748 and 6723) and I confirmed they reproduce on a clean tree with this
change stashed out — 284 passed, 2 failed, same two line numbers. They fail
with
error: one or more agent cleanup operations failed, which depends onthe coding agents installed on the machine, not on this change.
Flagging it because a contributor running the full suite will see red and may
assume they caused it. Happy to open a separate issue if that is not already
known.
Scope
Per CONTRIBUTING.md this is filed without a prior issue under the bug-fix
exception (line 124). It is one defect, plus the
UNIONcursor fix that thedefect's own check uncovered and which cannot be separated from it. 82 lines
added, nothing removed, two files.
Checklist
git commit -s) — required, CI rejectsunsigned commits (DCO, see CONTRIBUTING.md)
make -f Makefile.cbm test) — not ticked, andhere is why: 7623 pass, 2 fail. Both failures are in
tests/test_cli.c(1748, 6723), reproduce on a clean tree with thischange stashed out, and depend on the coding agents installed on the
machine. The cypher suite this change touches is 186/186. I would
rather leave the box honest than tick it with a footnote.
make -f Makefile.cbm lint-ci) — cppcheck, clang-formatand the NOLINT whitelist check all pass
🤖 Generated with Claude Code