Limit bracket nesting in the lexer like CPython - #3
Conversation
Source like `'[' * 5000 + '1' + ']' * 5000` makes the parser recurse until
the native stack runs out, killing the process with SIGSEGV instead of
raising a Python error.
CPython stops this in the tokenizer, not the parser: it rejects an opening
bracket once `tok->level` hits `MAXLEVEL` (200), so the parser never
recurses that deep. Do the same here using the nesting counter the lexer
already keeps. As in CPython, `(`, `[` and `{` share one counter and report
the same message.
Checked against CPython 3.14.6: depth 200 parses, depth 201 raises
`SyntaxError: too many nested parentheses`, and brackets inside strings,
comments and f-strings are ignored. Deep recursion without brackets, such
as long operator chains, still overflows and needs a separate fix.
Refs RustPython/RustPython#7655
CPython reference: https://github.com/python/cpython/blob/main/Parser/lexer/lexer.c#L582-L600
Assisted-by: Claude Code:claude-opus-5
|
Important Review available on request
Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
This is looking like a bug in Ruff. Could you please submit a patch about this to Ruff upstream? If they reject the patch, I will accept this one. Sorry for late review! |
|
No worries. It seems that Ruff is addressing this issue in astral-sh#24810 and astral-sh#25464. |
|
let's see if #4 fix this |
Summary
'[' * 5000 + '1' + ']' * 5000makes the parser recurse until the native stack runs out, so the process dies with SIGSEGV instead of raising a Python error.CPython stops this in the tokenizer rather than the parser: it rejects an opening bracket once
tok->levelreachesMAXLEVEL(200), so the parser never recurses that deep. This does the same, reusing the nesting counter the lexer already keeps — one comparison per opening bracket, no extra pass. As in CPython,(,[and{share the counter and report the same message.Deep recursion without bracket nesting, such as long operator chains, still overflows. That needs a parser-level stack guard and is out of scope here.
Refs RustPython/RustPython#7655
CPython: https://github.com/python/cpython/blob/main/Parser/lexer/lexer.c#L582-L600
Test Plan
Verified through RustPython against CPython 3.14.6:
SyntaxError: too many nested parenthesestest_grammar(includingtest_max_level),test_syntax,test_compile,test_exceptions,test_fstring,test_ast,test_tokenize,test_symtable,test_unparse,test_string_literals,test_codeopThis crate's own tests don't build in this fork — a pre-existing
serdefailure inrustpython-ruff_python_ast, reproducible onmainwithout this change — so verification was done through RustPython instead.Investigated and implemented with Claude Code; reviewed and verified manually before submitting.