Skip to content

fix(combos): make moderation-refusal prose matching linear (CodeQL js/redos) - #841

Closed
luvs01 wants to merge 7 commits into
devfrom
devin/security-fix-redos-failover
Closed

luvs01 wants to merge 7 commits into
devfrom
devin/security-fix-redos-failover

Conversation

@luvs01

@luvs01 luvs01 commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Fixes CodeQL js/redos alerts fix(history): contain paginated metadata writes and preserve restore artifacts lidge-jun/opencodex#4342 and fix(update): retire observed exited pinned-start children lidge-jun/opencodex#4343 on MODERATION_REFUSAL_PROSE (src/combos/failover.ts), which ran upstream-supplied error prose through ^(?:verdict)(?:tail)*$. Its category list swallowed separator words (", and retry") that sibling tail options also accept, and its ,? and |, separators overlapped, so a crafted provider error body forced exponential backtracking before rejection — a small DoS surface on the failover path.
  • Replaced the single nested regex with an anchored two-stage scan in isModerationRefusalProse: one verdict match, then tail phrases consumed greedily in order. The Azure wording now leads the verdict alternation because "the response was filtered" is a strict prefix of it, and the category separators are disjoint by first byte, so every position has exactly one parse.
  • Language preserved exactly: verified identical accept/reject against the old pattern on a 29-case corpus covering every verdict and tail spelling. On a rejecting attack string (refused category: a + , and retry category: a × N + !), the old regex exploded exponentially (22ms at 15 reps → 698ms at 20 reps → minutes at the 2000-char cap); the new scan is flat ~0ms.

Verification

Current integration — 2026-11-19

Current head: 31e5078742f928592d7f88287cdb98a688cbc975. Actual target-repository dev: 0dc315b95a2493ed1fc6193a555f36d645ea186b, included by a normal merge (zero commits behind at publication). The published GitHub tree exactly matches the locally verified tree 88a9825130a2e8cfa46d5b583d26ec98ef953542.

This normal merge brings the branch forward from 73 commits behind actual dev. The bounded moderation matcher is preserved alongside dev's newer unoffered-tool-call classifier and branches. Both test-layout files retain current-dev mappings and this PR's regression entry.

  • bun test tests/routing/router-combo-failover-classification.test.ts tests/routing/combo-failure-transport-parity.test.ts tests/routing/combo-failover-moderation-prose.test.ts tests/test-layout.test.ts tests/test-layout-tooling.test.ts: 287 passed / 0 failed / 1,902 assertions.
  • Runs used the normal repository preload, its unmodified network-egress guard, isolated temporary filesystems and local fixtures. No live provider, live SSH endpoint or installed application was exercised.
  • bun run typecheck: 2,589 files, no errors. Structure validation, privacy scan, file-size ratchet and whitespace checks against actual dev passed.
  • bun test tests/ci-workflows/docs-spanish-localization.test.ts tests/ci-workflows/file-size-ratchet.test.ts: 57 passed / 0 failed / 3,388 assertions.
  • There is no documentation delta against current dev; the docs literal-parity and file-size guards above passed.

Independent read-only integration review confirmed that the only runtime delta against dev is the moderation matcher: anchored verdict/tail scanning and the 2,000-character guard are preserved, and the unoffered-tool-call classifier and branches remain intact. No concrete blocker was found.

The repository's focused-validation exception applies to this integration and concurrent worktrees. The full local suite and broad test:changed were not run; hosted CI provides broader PR coverage. Native macOS/Windows and live-provider acceptance are not claimed.

Hosted CI and review — 2026-11-19 03:38 UTC

Main CI, attempt 2 completed successfully for the exact current head above at 03:35:14 UTC. The retried shard 2/5 completed successfully from 03:20:57 to 03:35:08 UTC, including its test and post steps. The aggregate ci check is successful. Shards 1/5, 3/5, 4/5 and 5/5 retain their successful original executions. Catalog async contracts also completed successfully.

Attempt 1's shard 2/5 had been cancelled at the configured 15-minute job deadline/finalization boundary after all 61/61 batches reported 0 fail and the test/post steps succeeded. The final batch reported 164 passed / 2 skipped / 0 failed / 860 assertions; the captured log contained no failed test result or nonzero failure summary. That incomplete attempt was not treated as a pass. Only the cancelled job received one bounded retry on this head; previously successful jobs and the entire workflow were not manually rerun. No CI label, timeout, assertion or workflow policy was changed.

Current-head Devin Review completed successfully. CodeRabbit's current-head check is skipped because that app requires a manual review request; its earlier completed review remains historical coverage. All existing inline review threads remain resolved. No new external review comment or review-request message was posted.

The existing test-layout fixture delta selected the full Ubuntu shard lane under the unchanged repository policy. Successful hosted CI covers that selected lane; conditionally skipped native-platform jobs are not claimed as passed. The PR remains Ready for review. Maintainer security approval and applicable native-platform acceptance remain separate merge requirements.

Earlier evidence

The previous exact head e8dc03a6371da59771e61b0af3b30f442cc81217 had incomplete main CI, attempt 2: shards 2 and 3 again reached the 15-minute job limit in batches 60/63 and 63/64, without assertion failures in either captured log. Shards 1, 4 and 5 passed; that run's aggregate remained failed. One bounded failed-job retry had already been used. The earlier 29-case old-versus-new language corpus and timing comparison remain historical evidence.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed. (No user-visible config or behavior change; comments in-place document the scan invariant.)
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Link to Devin session: https://app.devin.ai/sessions/3f957f3362db40dea0c40cb34195ac55
Open in Devin Desktop: https://app.devin.ai/desktop/session/3f957f3362db40dea0c40cb34195ac55?variant=devin
Requested by: @luvs01


🟢 Devin Review found nothing to fix · View findings · Follow Devin's work

Devin Review

Summary by CodeRabbit

  • Bug Fixes
    • Moderation refusal messages with advice or category details are now recognized more consistently. Unrelated, incomplete, or ambiguous messages continue to be rejected, including unusually long category lists.
  • Tests
    • Added coverage for accepted and rejected message formats, including checks that complex category lists are handled promptly.

…/redos)

MODERATION_REFUSAL_PROSE wrapped a verdict alternation and a tail
alternation in one ^(?:v)(?:t)*$ regex. Its category list consumed
separator words ("and retry") that a sibling tail option also accepts,
and its ",? and |, " separators overlapped, so a crafted upstream
error body (e.g. "refused category: a" + ", and retry category: a"
repeated) forced exponential backtracking before rejection
(CodeQL alerts lidge-jun#4342, lidge-jun#4343).

Split it into two anchored regexes consumed by an explicit scan: one
verdict, then tail phrases eaten greedily in order. The Azure wording
leads the verdict alternation because "the response was filtered" is a
strict prefix of it, and category separators are disjoint by first
byte, so every position has exactly one parse. Behavior verified
identical on a 29-case corpus; the attack string runs in ~0ms instead
of exploding exponentially.

Verified: bun test tests/routing/router-combo-failover-classification.test.ts
tests/routing/combo-failure-transport-parity.test.ts (228 pass),
bun run typecheck.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
@devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Explain Change →

Walkthrough

The moderation refusal prose matcher is now exported. It parses anchored verdicts followed by recognized category or advice tails. New routing tests cover accepted and rejected text, including long category lists, and the test layout registers the test in the routing shard.

Changes: Moderation refusal prose matching

Layer / File(s) Summary
Parse moderation verdicts and tails
src/combos/failover.ts
The matcher uses separate anchored expressions for verdicts and tails. It collapses whitespace, rejects text over 2,000 characters, and returns false when no verdict matches or text remains after tail parsing.
Test and register matcher cases
tests/routing/combo-failover-moderation-prose.test.ts, scripts/test-layout/layout.json, tests/fixtures/test-layout-expected.json
The tests cover accepted and rejected phrases and check that two repeated category-list inputs complete in under 1,000 ms. The test layout and its expected fixture include the test in the routing shard.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: lidge-jun, ingwannu, omar391

Merge Risk: Low · previous review 31e50

The change makes moderation-refusal matching linear-time, but a few unusual category-list phrasings that used to be recognized may no longer be. The impact is limited. Fix the category parsing and add regression tests before merging if full compatibility matters.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: making moderation-refusal prose matching linear to fix a CodeQL js/redos finding. The raw summary confirms that isModerationRefusalProse in src/combos/failover.ts was …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: making moderation-refusal prose matching linear to fix a CodeQL js/redos finding. The raw summary confirms that isModerationRefusalProse in src/combos/failover.ts was rewritten to use an anchored verdict match followed by anchored tail scanning, and the objectives describe the same replacement of the nested regex. The title is 76 characters, which is longer than ideal but still concise and specific.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
Fix failing CI checks

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions github-actions Bot added the intake: hygiene-blocked Deterministic PR hygiene checks failed label Nov 18, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Deterministic hygiene checks failed.

  • missing_regression_test — Behavior changed under src/ or gui/src/ without a test change. Add focused coverage or obtain test-exception-approved.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

✅ READY

  • all PR quality gates passed.

Hygiene

✅ Deterministic PR hygiene checks passed.

@github-actions
github-actions Bot marked this pull request as draft November 18, 2026 01:34
@github-actions github-actions Bot added the bug Something isn't working label Nov 18, 2026
Direct coverage of isModerationRefusalProse (exported for tests): every
accepted verdict and tail spelling, non-refusal bodies that must keep
dev's decision, and both rejecting and accepting 80x-repeat category
lists completing under a 1s bound (the previous nested-regex shape grew
exponentially on the rejecting one).

Registers the file in scripts/test-layout/layout.json and
tests/fixtures/test-layout-expected.json under routing.

Verified: bun test tests/routing/combo-failover-moderation-prose.test.ts
tests/test-layout.test.ts tests/test-layout-tooling.test.ts (55 pass),
bun run typecheck, bun run privacy:scan.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
@github-actions github-actions Bot removed the intake: hygiene-blocked Deterministic PR hygiene checks failed label Nov 18, 2026
@github-actions
github-actions Bot marked this pull request as ready for review November 18, 2026 01:35
… scan

Devin Review flagged two accepted-wording regressions on the rewritten
matcher: the bare-verb verdict alternative won before the noun-phrased one
that reaches `by ... system` (HTTP 400/422 + moderation code strings like
"content refused by moderation system." were rejected), and the greedy
category-list separator `, and ` swallowed retry-advice openings so
"refused category: harassment, and retry later." failed to parse.

Order the verdict scan Azure-first, then noun-phrased, then bare verb,
then policy wording, and put a negative lookahead on the `, and `
separator for retry/try/please/to openings. The committed-test corpus now
covers both regression shapes; equivalence vs the pre-fix regex holds at
0/42 divergences and the attack strings stay ~0.1ms vs >3.4s before.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 finding. Devin automatically fixed it.

Fixed by Devin (1)
  • Previously supported moderation refusals stop combo failover — Confirmed both regressions: the bare-verb verdict alternative ran before the noun-phrased one reaching by ... system, and the , and category separator swallowed retry-advice openings. Fixed in cedbd90 by ranking verdict alternatives longest-first (Azure → noun-phrased → bare → policy) and adding a lookahead (?!retry\b|try\b|please\b|to\b) on the , and separator. Both reported strings added to the committed regression corpus; equivalence vs the pre-fix regex holds at 0/42 divergences and 216 focused tests pass.

View findings in Devin Review · Follow Devin's work

Devin Review

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
@luvs01

luvs01 commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner Author

No description provided.

@luvs01

luvs01 commented Nov 19, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]

This comment was marked as resolved.

@luvs01

luvs01 commented Nov 20, 2026

Copy link
Copy Markdown
Owner Author

No description provided.

@luvs01

luvs01 commented Nov 20, 2026

Copy link
Copy Markdown
Owner Author

The change is now tracked in lidge-jun/opencodex#8978, targeting upstream dev at d61cca122d00899b0a917162875c7c2c659f4952. I compared this fork head, 21a4b6db51e783f54e8e896d4d4cfdc0ee97d2e7, with upstream PR head 6fc8e0ff1aa6abdcadd6accb9338bdfdd968d856. Their matcher implementations agree apart from the fork's test-only export. The upstream diff includes both category-list regressions and checks that the message alone remains insufficient.

The two regression cases failed on the earlier fork head and passed after the correction. Local verification passed for 30 fork prose cases, 9 upstream classification cases, type checking, and the structure gate on both branches.

Closing this fork PR as superseded by the open upstream PR. Review and CI continue upstream.

@luvs01 luvs01 closed this Nov 20, 2026
lidge-jun added a commit that referenced this pull request Nov 21, 2026
…cktracking

The added category-boundary cases also matched the old regex, so nothing failed if the
scan were reverted. This case uses the CodeQL js/redos attack shape from #841:
on origin/dev it takes about 976 ms per body (Bun's backtracking limit), on this head 1.4 ms.
lidge-jun added a commit to lidge-jun/opencodex that referenced this pull request Nov 21, 2026
…#8978)

* fix(combos): preserve moderation category boundaries in a single scan

Carry the bounded matcher from luvs01#841 and retain both category-list readings until the end of the message.

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: keep category-list guidance within the structure budget

* test(combos): pin the moderation category scan against exponential backtracking

The added category-boundary cases also matched the old regex, so nothing failed if the
scan were reverted. This case uses the CodeQL js/redos attack shape from luvs01#841:
on origin/dev it takes about 976 ms per body (Bun's backtracking limit), on this head 1.4 ms.

---------

Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: JUN <jun@lidgeai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant