Repository navigation
Conversation
…ceholder sep-9998) agents-wg PR 22 has no number in modelcontextprotocol/modelcontextprotocol yet, and both traceability filters are numeric, so the rows and check IDs carry the placeholder 9998. The header explains why and how to rename. 31 graded rows and 18 excluded. Every row quotes the proposal at 9b16840e, matched with line wrapping ignored, and points at the section it sits under.
…ged scenarios Discovery grades the capability, the roster (shape, paging, cache hints, cards, unique names), the details for every listed agent, tool parity with tools/list, and the two error reasons. The hidden-agent and broken-agent rows read their names from the server's agents_conformance_fixture control and report untestable without it. List-changed opens a subscriptions/listen stream with agentsListChanged, grades the acknowledgement either way round, changes the roster through agents_conformance_add_agent, and waits for the notification and the re-listed roster. Registered as pending, since the everything-server does not implement the extension.
A stateless fixture built per test, conformant by default, with one option per rule. 40 tests: a conformant run with every check passing, then one broken rule per test asserting its check flips.
commit: |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
Adds a requirement-traceability yaml for the agent discovery extension, agents-wg#22, and two server scenarios that grade an implementation against it, plus 40 negative controls.
sep-9998-agents-discoveryio.modelcontextprotocol/agentscapability;agents/listshape, cards and pagination; the flatagents/getresult andmodelRoutingHint; every returned tool matching itstools/listentry;agent_not_foundandinvalid_agent_definition; a hidden agent indistinguishable from an unknown onesep-9998-agents-list-changednotifications/agents/list_changedoversubscriptions/listenwithagentsListChanged, and its acknowledgementThe yaml declares 49 rows quoting the proposal at
9b16840e, each with aurlto its section, checked by script against the pinned file. 31 are graded. The other 18 are declared and excluded, each with its reason: client and host obligations, rules that compare two authorization contexts (a run holds one credential), and step-up, a MAY that needs a credential the harness can raise.Things to look at first
modelcontextprotocol/modelcontextprotocolPR number, and the traceability tooling drops non-numeric ids. The yaml header explainssep-9998(9999 was the MCP Events placeholder before SEP-3415 existed) and the scripted rename once a number exists.list_changedrows need the server to provoke a state over the wire. The scenarios look foragents_conformance_fixtureandagents_conformance_add_agentintools/list, and report untestable without them, the same pattern as feat(sep-3415): Conformance suite for triggers and events #521.--force. The extension isn't on the spec timeline, so the runner selects these scenarios only with--force.ttlMsandcacheScopeare graded as required, because both results areCacheableResult, where the draft schema requires them.Evidence
Against mcpkit's implementation (
examples/agents/discovery --conformance-agents), the result is 28/28 and 4/4. Breaking that implementation four ways, by showing the hidden agent, never sendinglist_changed, returning a wrongagent_not_foundreason, or omittingttlMs, turns the matching rows red. Running against the Python reference fork is next, once it serves the extension.npm run build,npm run lintandnpm testpass.Prerequisite knowledge
9b16840eis the text every row quotes. Retrieving Agent Details, Error Handling and Security Implications carry most of the graded rows.AGENTS.md§ Scenario design and § Check conventions explain why this is two scenarios, and why a missing control fails rather than skips.