Repository navigation
feat(sep-1036): URL mode elicitation conformance scenarios - #558
Open
nbarbettini wants to merge 1 commit into
Open
nbarbettini wants to merge 1 commit into
nbarbettini wants to merge 1 commit into
Conversation
Add client and server scenarios for URL mode elicitation on the 2026-07-28 revision, where elicitation requests travel inside an InputRequiredResult (SEP-2322). - elicitation-sep1036-client-url-mode: rounds of URL-mode requests, mixed URL and form inputs, requests without mode, no prefetch of a declined URL, and retries the server holds open (JSON and SSE). - elicitation-sep1036-url-mode: URL-mode request shape and url validity, decline/cancel handling, no unsupported modes, -32021 for clients without URL mode, and no -32042. - sep-1036.yaml requirement rows; sep-2322.yaml gains client constructs-inputs rows and the decline/cancel row. - everything-client handler (plus SSE response parsing), everything-server test_url_elicitation tool and form-tool gating, a broken server, and negative tests. Closes modelcontextprotocol#542. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
commit: |
nbarbettini
marked this pull request as ready for review
October 15, 2026 15:41
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #542.
TL;DR
Adds conformance coverage for URL mode elicitation (SEP-1036), targeting the 2026-07-28 revision. Until now the suite had no URL elicitation checks.
elicitation-sep1036-client-url-mode), with eight checks:requestStateeach time;modeas form mode;elicitation-sep1036-url-mode), with seven checks:urlis valid;-32021(HTTP 400) to a client that didn't declare it, and never sends a mode the client didn't declare;-32042error never appears.Background
URL elicitation (SEP-1036) has been in the protocol since 2025-11-25, and got a major update with MRTR in 2026-07-28. However, it hasn't been represented in the conformance suite until now.
This PR adds scenarios to help both client and server authors stress-test their implementations against the expected spec behavior for URL elicitation.
Both scenarios target 2026-07-28 only. That revision changed how elicitation works: a server now asks for input by returning an
InputRequiredResult(MRTR, SEP-2322), and the client answers by retrying the original request. This PR doesn't test the 2025-11-25 shape.Client scenario:
elicitation-sep1036-client-url-modeHere the harness plays the MCP server, and the SDK's conformance client is the thing under test. The server lists six tools. Each one answers a
tools/callwith anInputRequiredResult, and the scenario watches how the client answers it and retries.The client follows a small contract, written out in the scenario description:
elicitation: { "form": {}, "url": {} }on every request.requestStateechoed back.DECLINE:, and accept the rest.{ "confirmed": true }.The
DECLINE:rule is how the scenario can tell that the client asked its handler (standing in for the user) about each request, rather than reusing an earlier answer.A client that doesn't support URL mode can declare form mode only. The mock server then answers its URL tools with
-32021, the way a correct server should, and the URL checks report SKIPPED rather than failing.sep-1036-client-supports-declared-mode. The spec says a client that declares theelicitationcapability must support at least one mode, form or URL. For backward compatibility, an emptyelicitation: {}counts as form mode only, so it passes. The only way to fail this check is to declare elicitation with neither mode named, like{ "foo": {} }. It's a narrow check, and that's on purpose: it tests exactly what the sentence says and nothing more.sep-2322-client-constructs-inputs. This is the main client check. The server sends three URL-mode requests in a row, each with a new key and a newrequestState. The first two should be accepted. The third reuses the first round's URL but asks for a decline. Under MRTR, a client that receives input requests has to construct answers to them before it retries, and each round is a new request, so the client should go back to its handler every time. The check catches three kinds of bug:Prompting the user again each time is correct behavior here. The server is the one deciding what it needs.
sep-2322-client-request-state-echoed. Same flow as above. On every retry, the client has to send back the newestrequestStateexactly as it received it. A client that echoes an older round's state fails, and the error message says which round it came from.sep-2322-client-constructs-inputs-mixed. OneInputRequiredResultcarries a URL-mode request and a form-mode request together. The client has to answer both: an accept for the URL, and an accept with content for the form. This has its own check ID because a client can get the round-by-round flow right and still drop one of two requests sent at once. It's skipped for clients that only declare URL mode, since the server can't send them a form request.sep-1036-client-missing-mode-is-form. The spec says a request without amodefield is form mode. The test request deliberately carries a strayurlfield alongside itsrequestedSchema, to catch a client that decides the mode by looking for a URL. A correct client accepts it with form content. A client that answers it like a URL request, with an accept and no content, fails.sep-1036-client-no-url-prefetch. Clients must not automatically pre-fetch a URL-mode URL or any of its metadata. The scenario serves each declined URL from a separate local HTTP server, so any request that reaches it (a favicon, aHEAD, a link preview) shows up. The check is only judged on the path the user declined, because that's the only path where a fetch is unambiguous:sep-1036-client-held-json-retry-completeandsep-1036-client-held-sse-retry-complete. After the client accepts, the server holds its retry open for five seconds before answering. The spec allows that; the URL Mode Flow notes the server "may need to block until the request is fulfilled". One tool holds the response as plain JSON, with no bytes sent until it's ready. The other holds it on an SSE stream with keep-alive comments, plus progress notifications if the client sent aprogressToken. A client that closes the request before the answer arrives fails. Five seconds is well inside any reasonable client timeout; the TypeScript SDK defaults to 60.These two have no requirement rows in
sep-1036.yaml. The spec gives servers permission to hold the retry, but no MUST or SHOULD says clients have to wait for it. They're flow checks, like the existingsep-2322-multi-round-r*checks.Server scenario:
elicitation-sep1036-url-modeHere the harness plays the client, and the SDK's server is under test. The server needs one new test tool,
test_url_elicitation, which can only do its job through URL mode:elicitation.url, it returns a-32021MissingRequiredClientCapabilityError, with HTTP 400 anddata.requiredCapabilities: { "elicitation": { "url": {} } }.InputRequiredResultholding one URL-mode request.isError: true.The scenario calls this tool several times, declaring different elicitation modes each time. The calls that declare both modes come first, so a server that remembers capabilities from an earlier request gets caught by the form-only calls that follow.
sep-1036-url-request-params. The URL-mode request has to saymode: "url"and carry a stringmessageand a stringurl. The check also flags a request that has aurlbut forgot themode, because clients are required to treat a request withoutmodeas form mode.sep-1036-url-valid. Theurlhas to be a valid URL. "Valid" here means exactly what the spec's JSON schema means byformat: uri: an absolute RFC 3986 URI, checked by the sameajv-formatsvalidator the wire-schema check uses, so the two can never disagree. Relative URLs and strings with spaces fail. The spec doesn't restrict the scheme beyond a SHOULD about HTTPS outside development, so the check doesn't either.sep-2322-server-handles-decline-cancel. The spec says servers must handle the user declining or cancelling. The scenario runs two separate flows: one where it retries withdecline, and one where it retries withcancel. Any well-formed answer counts as handling it: a tool result (includingisError: true), anotherInputRequiredResultoffering an alternative, or a deliberate JSON-RPC error. What fails is an internal error (-32603), a timeout, or a dropped connection, which are the signs of a server that only coded the accept path. This follows the existing MRTR checks:sep-2322-reject-tampered-statealready treats a deliberate JSON-RPC error as correct handling. The requirement row lives insep-2322.yaml, because the sentence was added to the elicitation page by the MRTR PR, not by SEP-1036.sep-1036-server-no-unsupported-mode. Servers must not send elicitation requests in modes the client didn't declare. The scenario calls the URL tool twice while declaring form mode only. Once it useselicitation: {}, which means form mode under the backward-compatibility rule; this is the one most likely to catch a server that just checks "did the client declare elicitation?". Once it uses{ "form": {} }. In both cases the response must not contain a URL-mode request.It also checks the reverse direction: a client that declares URL mode only calls the existing SEP-2322 form tool,
test_input_required_result_elicitation, and must not get a form request back. If a server doesn't have that tool, the reverse probe is recorded as not exercised and the check still grades the two form-only calls. This complementssep-2322-respect-client-capabilities, which only covers a client that declares no elicitation at all.sep-2575-server-rejects-undeclared-capabilityandsep-2575-missing-capability-http-400. These are existing SEP-2575 check IDs, exercised here for the URL mode case. The 2026-07-28 base protocol says that when a request needs a capability the client didn't declare, the server must return-32021, list the missing capability indata.requiredCapabilities, and use HTTP 400. Sincetest_url_elicitationcan't work without URL mode, the form-only calls should get exactly that. The response must also nameelicitation.urlspecifically. A bare{ "elicitation": {} }would be inaccurate, because the client did declare elicitation; it's only the URL mode that's missing.sep-1036-server-no-url-elicitation-required-error. The 2026-07-28 base protocol lists error codes from earlier revisions that implementations must not emit any more, and-32042(URLElicitationRequiredError) is one of them. Every response in this scenario is checked for it. A server ported from 2025-11-25 is likely to still return that error where it should return anInputRequiredResult. That sentence came from spec PR #2907 (error code allocation), which isn't a SEP, so there's no natural SEP file for it. The row is insep-1036.yamlbecause-32042was SEP-1036's error code, with a comment saying where the sentence really came from.A server that doesn't have
test_url_elicitationat all gets every check reported as untestable (FAILURE withdetails.untestable, per #248). That includes the checks that would otherwise pass trivially because a missing tool never sends anything.Requirement tracking
src/seps/sep-1036.yamlis new. It has 7 check rows and 29 excluded rows. The exclusions are client UI requirements (consent, showing the full URL, highlighting the domain, Punycode warnings) and server requirements that happen out of band or inside the server (binding state to the user, phishing protections, third-party credentials). None of those can be seen on the wire.src/seps/sep-2322.yamlgains three rows:sep-2322-client-constructs-inputs, its mixed-modes variant, andsep-2322-server-handles-decline-cancel.traceability.jsonisn't regenerated here; the traceability workflow normally does that.Fixtures and tests
test_url_elicitation. Its existing MRTR form tool now answers-32021to a client that declared URL mode only, instead of sending form mode anyway. All 14 MRTR scenarios still pass.examples/servers/typescript/sep-1036-url-elicitation-broken-server.tsbreaks each server requirement on purpose.all-scenarios.test.tsagainst the everything-server.npm test(661 tests),npm run lint, andnpm run typecheckpass. CLI runs of both scenarios against the in-repo fixtures pass, includingwire-schema-valid.What SDKs need to do
Neither scenario affects tier scoring. Both were added after the frozen 2026-07-28 requirement set, and both currently live in the draft suite.
When an SDK bumps to a conformance release that includes them:
test/conformance/src/everythingClient.ts, since itsClientalready drives MRTR rounds through the registeredelicitation/createhandler. I ran it against typescript-sdkmain(324299b2) and it passes all eight checks pluswire-schema-valid.test_url_elicitation, plus capability gating on its MRTR form tool.Deliberately not graded
InputRequiredResult, or do something else; the spec allows several ways and defines no "pending" signal. This PR only checks that clients cope with a server that holds the request open. It doesn't grade servers on which way they choose.🤖 Generated with Claude Code
📝 Edited by Nate, the human