Repository navigation
feat: add scoped Spec Studio audits and confirmed remediation - #260
Coding-Dev-Tools wants to merge 26 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3a7e1dd391
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 99189b0990
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d87bb31d9c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ffa3cc1be1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8d2f945319
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 819d80f10a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 722e191ab0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| continue | ||
| toks_i = node_tokens_map[id_i] | ||
| toks_j = node_tokens_map[id_j] | ||
| sim = jaccard(toks_i, toks_j) |
There was a problem hiding this comment.
At the supported limit of 150 memories, each containing up to 100,000 characters, unrelated records with similarly sized token sets pass the length-ratio prefilter and execute this Jaccard calculation for nearly every pair. Orphan selection and redundancy detection later repeat comparable quadratic scans, so a valid read-only workspace audit can occupy a server worker for well over a minute. Cap the tokens considered or compute/cache similarities over a bounded candidate shortlist.
Useful? React with 👍 / 👎.
| if alias in title.lower(): | ||
| scores[axis] += 3 |
There was a problem hiding this comment.
Substring matching lets unrelated titles inherit an axis with the higher title weight. For example, Preview configuration contains the alias review, so it is classified as review coverage and suppresses the missing-review policy gap even though no review policy exists. Tokenize the title or apply the same word-boundary matching used for the combined text.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fd84c285a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| # Instruction-like payloads are data, flagged once per section. | ||
| for section in sections: | ||
| signals = detect_payload_signals(text[section.start:section.end], title=section.title) |
There was a problem hiding this comment.
When a specification documents an attack string inside fenced or inline code, this scans the entire section rather than excluding the ranges identified by _code_ranges. For example, a fenced Ignore previous instructions... test fixture receives an injection flag and a 10-point penalty even though code is otherwise defined as read-but-never-flagged, producing false alarms in security playbooks and prompt-defense specs.
Useful? React with 👍 / 👎.
| # Tokens & vague words | ||
| sec = _Section(index=0, title="node", axis="context", heading_start=0, | ||
| start=0, end=len(content)) | ||
| raw_toks, _ = _tokenize(content, [sec], [], classifier) |
There was a problem hiding this comment.
For a memory containing more than MAX_SPEC_WORDS tokens, _tokenize reports truncation but that result is discarded here. Qualifiers after the first 5,000 tokens are therefore omitted from flag_count, and the node can be counted as clean in quality_points while the response exposes neither the incomplete analysis nor the true word count; retain the truncation result in the node/report or avoid treating truncated nodes as fully analyzed.
Useful? React with 👍 / 👎.
|
|
||
| ## 9. Smart MCP Tools | ||
|
|
||
| The two read-only crawls are available to hosted/team viewers; clarification and resolution require member access. |
There was a problem hiding this comment.
In hosted/team Smart MCP, this promises member access for clarification and resolution, but minimum_role() hard-codes engraphis_execute_action to admin in engraphis/mcp_server.py:444-447, and both mutation tools are dispatched through that stateful executor. A member can therefore discover these actions but cannot execute them through Smart MCP; either document the admin-only Smart boundary or provide a member-authorized execution path.
AGENTS.md reference: AGENTS.md:L307-L307
Useful? React with 👍 / 👎.
This branch has not been deployed
2 participants
Adds offline Spec Studio audits for instruction documents and scoped, approved memories, with explicit confirmation for remediation and temporal supersession history. Audits report sampling/truncation and keep inferred links separate from stored connectivity.
Clarification storage is opt-in across the dashboard, HTTP, and MCP, and an explicitly supplied session stays session-scoped. Code-span lookup uses merged intervals and binary search; unrelated subjects cannot hide directive conflicts, and nested negative directives do not recommend retiring agreeing rules. HTTP/MCP schemas agree on required endpoints and list limits. Exact Git bytes are preserved for checksum-bound fixtures across platforms, and fresh immutable evidence and chart links bind the changed source.
Validation: focused spec, API, MCP, portable-skill, evidence, frozen-corpus and vendor tests; all 10 Spec Studio Chromium tests; nine offline eval gates; Ruff, Pyright, commercial manifest, and strict-CSP drift checks. Remote CI validates the current pushed head.