Skip to content

test: add MCP Events conformance checks and a shared test server - #1192

Open
alexluong wants to merge 5 commits into
feat/mcp-subscribe-endpointsfrom
feat/mcp-test-harness
Open

alexluong wants to merge 5 commits into
feat/mcp-subscribe-endpointsfrom
feat/mcp-test-harness

Conversation

@alexluong

@alexluong alexluong commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Add a library that checks an MCP Events (draft SEP-3415) server against the draft's webhook flow, and move the e2e test client, receiver and server into it so the e2e suites and the checks share one implementation. Test code only, under internal/mcpevents/mcptest; Outpost itself doesn't change.

Stacked on #1183.

Changes

  1. Move. The e2e MCP client, receiver and reference server move to internal/mcpevents/mcptest/{mcpclient,mcpreceiver,mcpserver}. The e2e suites change imports only. None of the packages imports internal/mcpevents, so the checks never reuse the code they check.
  2. Stateless mode and discovery. The client can send the protocol version, client capabilities and client info per request in _meta instead of using a session, and the reference server answers server/discover with the events extension. The initialize result doesn't advertise a top-level capabilities.events: the draft defines events only as an extension.
  3. Standalone receiver. The receiver can listen on any address and public URL, answers 401 without echoing a challenge to requests no configured secret verifies, and checks X-MCP-Subscription-Id and signed gap and terminated envelopes. It has two rule profiles: outpost (the e2e default: Outpost's own choices such as the webhook-id form) and spec (only what the draft requires).
  4. Conformance checks. mcpcheck.Run runs the webhook flow against a server and reports one result per rule with the draft section behind it: PASS, FAIL (a MUST is missed), WARN (a SHOULD is missed, or something the draft allows is unusual), SKIP (a precondition failed). It covers discovery, events/list descriptors and paging, auth and header checks, subscribe errors, callback verification, delivery envelope, signature and headers, retries, refresh and secret rotation, argument filtering, isolation between principals, and unsubscribe.

Behavior changes

Test code only. The e2e receiver requires X-MCP-Subscription-Id on every delivery and an object in data; a miss means a real Outpost bug.

Tests

TestMCP_Check (e2e) runs mcpcheck against the reference server in front of the e2e Outpost, publishing through Outpost's API, and fails on any FAIL. The mcpcheck unit tests include two injected faults (the server ignores Mcp-Name, signs with a wrong secret), so the checks are shown to fail.

To run: go test -short -race ./internal/mcpevents/mcptest/... for the unit tests; make up/test, then TESTINFRA=1 make test/e2e RUN=TestMCP_Check for the e2e check.

Not in this PR

  • A command to run the checks from a terminal: chore: add mcp-dev command and make up/mcp for trying MCP Events #1193.
  • Only the webhook mode is exercised: no events/poll, events/stream, listChanged.
  • Verification by allowlist, out of band or a well-known document shows as "no challenge" (WARN).
  • Replay, maxAgeMs, truncated, gap and terminated are shape-checked when they appear and never triggered. v1a signatures are ignored. Address-guard and redirect rules aren't probed; the Outpost e2e suites cover them.
  • The existing e2e suites still use initialize and sessions; moving them to stateless discovery is a follow-up.

🤖 Generated with Claude Code

alexluong and others added 5 commits October 31, 2026 03:41
…events/mcptest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eference server

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alexluong
alexluong force-pushed the feat/mcp-subscribe-endpoints branch from c374170 to 5cf245d Compare October 30, 2026 20:51
@alexluong
alexluong force-pushed the feat/mcp-test-harness branch from 92e557e to ae4fc36 Compare October 30, 2026 20:51
@alexluong alexluong changed the title feat: add MCP Events conformance checks and a shared test server test: add MCP Events conformance checks and a shared test server Oct 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant