diff --git a/docs/extensibility/supervisor-middleware/configure.mdx b/docs/extensibility/supervisor-middleware/configure.mdx index de2f22d76b..2bf7d22ad1 100644 --- a/docs/extensibility/supervisor-middleware/configure.mdx +++ b/docs/extensibility/supervisor-middleware/configure.mdx @@ -96,12 +96,12 @@ network_middlewares: -Run two middleware for the same hosts. `regex-demo` runs before `content-guard` because it has the lower `order`, and neither runs for `trusted.example.com`: +Run two registered services for the same hosts. `secret-scanner` runs before `content-guard` because it has the lower `order`, and neither runs for `trusted.example.com`: ```yaml showLineNumbers={false} network_middlewares: - regex-demo: - middleware: openshell/regex + secret-scanner: + middleware: secret-scanner order: 10 endpoints: include: ["*.example.com"] @@ -146,7 +146,7 @@ The map key, such as `content-guard`, is the entry's stable identity in logs. Ea - Destination hosts to run the middleware for. Uses the same host patterns as network policy endpoints. + Destination hosts to run the middleware for. Uses the same host patterns as network policy endpoints. A pattern matches a host on every port and cannot include a port. An IPv6 host must be a complete address without brackets, such as `2001:db8::1`. @@ -158,7 +158,7 @@ The map key, such as `content-guard`, is the entry's stable identity in logs. Ea - What happens when the middleware fails. Refer to [Choose Failure Behavior](#choose-failure-behavior). + What happens when the middleware fails. Refer to [Choose Failure Behavior](#choose-failure-behavior). Services that use HTTP hooks accept only `fail_closed`. @@ -171,6 +171,10 @@ The map key, such as `content-guard`, is the entry's stable identity in logs. Ea [Policy Schema](/how-it-works/policies/schema#network-middleware) lists every field and limit. +### Mix HTTP Hook Versions + +Each HTTP request or response runs through middleware of one [HTTP hook version](/extensibility/supervisor-middleware/operations#http). The gateway rejects a policy in which two entries whose endpoint selectors may overlap use different HTTP hook versions for the same operation, requests or responses. To fix it, separate the selectors or move both services to the same hook version. If a chain still mixes hook versions at runtime, OpenShell blocks the traffic with the reason `middleware_hook_versions_mixed`. + ## Choose Failure Behavior Middleware fails when it cannot complete its check, for example when its service is unavailable, times out, returns an invalid result, or receives a payload over its limit. By default, OpenShell blocks the affected request, response, or WebSocket connection. The `on_error` field on a policy entry controls this behavior. @@ -181,6 +185,8 @@ Middleware fails when it cannot complete its check, for example when its service +Middleware that uses HTTP hooks always fails closed. The gateway rejects `fail_open` on an entry whose service uses HTTP hooks, including a service that also inspects WebSocket messages. A policy stored before the service moved to HTTP hooks still loads, and supervisors block its HTTP traffic on failure and log a warning. + A denial from middleware always blocks traffic, regardless of `on_error`. Middleware decisions also stay enforced when the endpoint uses `enforcement: audit`. To observe traffic without blocking it, have your middleware allow the traffic and report findings. `on_error` does not extend what middleware can inspect. The [current limitations](/extensibility/supervisor-middleware#current-limitations) apply under both settings. With the default behavior, an entry cannot select a `tls: skip` endpoint, because OpenShell cannot inspect that traffic. diff --git a/docs/extensibility/supervisor-middleware/index.mdx b/docs/extensibility/supervisor-middleware/index.mdx index c17a79d252..89e84538a8 100644 --- a/docs/extensibility/supervisor-middleware/index.mdx +++ b/docs/extensibility/supervisor-middleware/index.mdx @@ -47,13 +47,13 @@ OpenShell checks network policy first, so middleware only sees traffic that poli Each middleware decides whether to allow, deny, or change the traffic. A denial blocks the request even when network policy allows it. If middleware fails or its service is unavailable, OpenShell blocks the affected traffic by default. -Middleware can handle three operations: HTTP requests, HTTP responses, and the text messages an agent sends over a WebSocket connection. Each middleware declares which operations it supports. [Supported Operations](/extensibility/supervisor-middleware/operations) describes each one. +Middleware can handle three operations: HTTP requests, HTTP responses, and the text messages an agent sends over a WebSocket connection. Each middleware declares which operations it supports. [Supported Operations](/extensibility/supervisor-middleware/operations) describes each one. HTTP middleware uses the HTTP hooks or the legacy HTTP hooks. The [HTTP hooks](/extensibility/supervisor-middleware/operations#http) stream request and response bodies of any size and always fail closed. The legacy HTTP hooks are deprecated for removal in 0.2.0. ## Current Limitations - Middleware does not inspect WebSocket binary messages or messages that the external service sends to the agent. - Response middleware cannot inspect the body of compressed, partial, or `Cache-Control: no-transform` responses. -- Middleware cannot inspect traffic to endpoints that use `tls: skip`. +- Middleware cannot inspect traffic to endpoints that use `tls: skip`, h2c, unsupported tunnels, raw TCP, or SQL passthrough. OpenShell [refuses or allows](/extensibility/supervisor-middleware/operations#uninspectable-traffic) such connections according to `on_error`. If your deployment requires inspection of this traffic, middleware cannot enforce that requirement today. @@ -85,7 +85,7 @@ Every middleware service implements these RPCs: - `Describe` returns the service manifest. The manifest lists the operations the service supports, with a payload limit and optional timeout for each. It also carries [protocol negotiation](/extensibility/overview#protocol-negotiation) metadata and the [expected audience](/extensibility/overview#confirm-the-audience-at-startup). - `ValidateConfig` checks the `config` object from a policy entry. The gateway calls it before it accepts a policy. -The service then implements an evaluation RPC for each operation it supports, as described in [Supported Operations](/extensibility/supervisor-middleware/operations). +The service then implements an evaluation RPC for each operation it supports, as described in [Supported Operations](/extensibility/supervisor-middleware/operations). New HTTP middleware should implement [HTTP Hooks](/extensibility/supervisor-middleware/operations#http). Most gRPC servers reject incoming messages larger than 4 MiB by default. Raise your server's limit to at least 4.5 MiB so that a maximum-size payload and the rest of the message fit. diff --git a/docs/extensibility/supervisor-middleware/operations.mdx b/docs/extensibility/supervisor-middleware/operations.mdx index 50a01b33f9..4d10855c9d 100644 --- a/docs/extensibility/supervisor-middleware/operations.mdx +++ b/docs/extensibility/supervisor-middleware/operations.mdx @@ -14,19 +14,23 @@ A middleware service handles one or more operations, such as HTTP requests or We | Operation | Phase | When OpenShell calls your service | RPC | | --- | --- | --- | --- | -| `HTTP_REQUEST` | `PRE_CREDENTIALS` | After network policy allows a request, before OpenShell injects provider credentials. | `EvaluateHttpRequest` | -| `HTTP_RESPONSE` | `PRE_RETURN` | After the external service responds, before the sandbox receives the response. | `HttpResponsePreReturn.Evaluate` | +| `HTTP_REQUEST` | `PRE_CREDENTIALS` | After network policy allows a request, before OpenShell injects provider credentials. | `HttpHooks.EvaluateRequest`, or legacy `EvaluateHttpRequest` | +| `HTTP_RESPONSE` | `PRE_RETURN` | After the external service responds, before the sandbox receives the response. | `HttpHooks.EvaluateResponse`, or legacy `HttpResponsePreReturn.Evaluate` | | `WEBSOCKET_MESSAGE` | `PRE_CREDENTIALS` | When the sandbox opens a WebSocket connection, and for each text message it sends. | `EvaluateWebSocketSession` | Your service declares each operation it supports as a binding, an operation and its phase, in its `Describe` manifest. OpenShell calls your service only for the operations it declares. +A service that requires the `openshell.supervisor-middleware.http-hooks` capability receives its HTTP bindings through the [HTTP hooks](#http). Any other service receives them through the legacy HTTP hooks, which are deprecated for removal in 0.2.0. + When several middleware match the same traffic, OpenShell calls them in ascending `order`, and each middleware sees the changes that earlier middleware made. ## Common Inputs and Results Every operation includes a request context. The context always includes `sandbox_id`. Use it for authorization, persistence, and correlation. The context also includes `sandbox` and `workspace` names when available. Names can be reused, so use them only for display and logging. -Operations that include headers deliver them in wire order, and repeated headers stay separate. OpenShell removes credential, routing, and hop-by-hop headers before sending them to your service. It also removes framing headers from request input. Response preflight retains upstream `Content-Length`, `Content-Encoding`, and `Content-Range` as read-only metadata, unless `Connection` names them as hop-by-hop headers. Your service cannot change or remove these fields; OpenShell handles downstream framing. +When OpenShell resolves the process that opened the connection, `context.originating_process` includes its binary and ancestors (nearest parent first), with `pid` set to `0`. Every request on a reused connection carries the process resolved when the connection opened. The field is absent when OpenShell cannot resolve the caller. To keep the context within 4 KiB, OpenShell drops the farthest ancestors of a deep process tree, and omits the process when its binary path alone is too long. Treat a missing process as unknown, not as a denial. + +Operations that include headers deliver them in wire order, and repeated headers stay separate. OpenShell removes credential, routing, and hop-by-hop headers before sending them to your service. It also removes framing headers from request input. Response preflight includes `Content-Encoding` and `Content-Range` as read-only metadata. The first stage retains upstream `Content-Length`. For later stages, OpenShell updates it to the known input length and omits it when unknown. Headers named by `Connection` remain omitted. Your service cannot change or remove these fields; OpenShell handles downstream framing. Every result can include an optional reason code, findings, and metadata: @@ -38,131 +42,124 @@ OpenShell never forwards free-form text from your service to the sandbox or to l ## HTTP -HTTP middleware can check a request before it leaves the sandbox, and the response before the sandbox receives it. A service can support either operation or both. +HTTP middleware inspects requests before OpenShell injects provider credentials and responses before the sandbox receives them. Implement `HttpHooks.EvaluateRequest` for `HTTP_REQUEST`, `HttpHooks.EvaluateResponse` for `HTTP_RESPONSE`, or both. + +Serve `HttpHooks` on the same endpoint as `SupervisorMiddleware`, which handles discovery and configuration. Each RPC invocation serves one binding phase, identified by `phase` in its preflight. Requests currently use `PRE_CREDENTIALS`; responses use `PRE_RETURN`. + +In `Describe`, add `openshell.supervisor-middleware.http-hooks` to both `extension.supported_capabilities` and `extension.required_capabilities`. This selects the `HttpHooks` RPCs for all HTTP bindings in your service. Older gateways and supervisors refuse the service because they do not support this capability. ### Requests -OpenShell calls your service after network policy allows a request and before it injects provider credentials. Your service can allow, deny, or change the request. +Your service receives the request target, headers, and context before it receives body bytes. Earlier stages may already have processed the body. At preflight, return one of these results: -```mermaid -sequenceDiagram - participant Sandbox - participant Supervisor - participant Middleware - participant Service as External service +- `continue_without_body` to allow the request without inspecting its body, with optional header changes. +- `inspect` to choose a [body mode](#body-inspection). +- `deny` to deny the request, with an optional reason code. Use it, with a reason code that says so, when your service cannot inspect the message. - Sandbox->>Supervisor: HTTP request - Supervisor->>Supervisor: Check network policy - Supervisor->>Middleware: EvaluateHttpRequest - Middleware-->>Supervisor: Allow, deny, or change - Supervisor->>Supervisor: Inject provider credentials - Supervisor->>Service: Forward request -``` +OpenShell applies middleware changes before it injects provider credentials. Middleware never sees those credentials. If middleware replaces a body, OpenShell checks it against body-aware policy, such as GraphQL, JSON-RPC, or MCP rules, before forwarding it. - - +Request output streams to the external service while the sandbox uploads. OpenShell instead buffers output when a stage selects WHOLE_BODY, the route needs SigV4 signing or body credential rewriting, or the request uses the forward proxy, HTTP/1.0, or `Upgrade`. JSON-RPC, MCP, and GraphQL endpoints also buffer output so OpenShell can check the replaced body against policy. Buffered output can be up to 10 MiB for SigV4 signing, 256 KiB for body credential rewriting, and 4 MiB otherwise. The preflight reports this limit to DUPLEX stages as `limits.max_output_body_bytes`. Larger output fails with `body_over_limit`. + +### Responses -The request body, target, headers, and request context. +Your service receives the response status, headers, and request context. It can return `continue_without_body` or choose a body mode with `inspect`. - - +The response preflight also carries `request_headers`: the request's headers after every request stage's changes and before OpenShell adds credentials or replaces `Accept-Encoding`, with the same headers omitted as in the request preflight. The request context's `request_id` matches the request's preflights. -- Allow the request unchanged. -- Deny the request with an optional reason code. -- Replace the body, up to the same payload limit as the input. -- Add, change, or remove headers. +To stop the response from reaching the sandbox, return `block_delivery` at preflight or during body inspection. If delivery has started, OpenShell aborts the response. The external service has already handled the request, so blocking delivery does not undo it. Retrying may repeat side effects. -When your service replaces the body, OpenShell checks the new body against body-aware policy, such as GraphQL, JSON-RPC, or MCP rules, before the next middleware or the external service sees it. A replacement cannot introduce an operation that policy denies. +Interim `1xx` responses pass through unchanged. A `101` protocol upgrade skips response middleware. - - +When a response stage might inspect the body, OpenShell sends `Accept-Encoding: identity` upstream in place of the sandbox's value, so the response arrives uncompressed. Request middleware still sees the sandbox's value. A binding with `max_payload_bytes: 0` never inspects bodies, so it leaves the header unchanged. -When your service denies a request, the sandbox receives a structured error and OpenShell does not contact the external service: +### Body Inspection -```json -{ - "error": "middleware_denied", - "detail": "Request rejected by configured middleware", - "policy": "api-policy", - "middleware": "content-guard", - "reason_code": "content_match" -} -``` +OpenShell opens one bidirectional stream per HTTP message and middleware stage. Request and response streams are separate. Each stage receives its preflight after the preceding stage establishes its output headers. For a buffered stage, this happens at `whole_body_result`; for a streaming stage, at `output_start`. A stage that continues without inspection passes its updated headers immediately. Later stages therefore choose inspection using the representation they will receive. -When middleware fails, the sandbox receives `error: middleware_failed` instead. +The preflight lists `permitted_body_modes`, unavailable modes with their reasons, and the applicable limits. Select a permitted mode or continue without inspecting the body. A binding with `max_payload_bytes: 0` receives no body modes. - - +| Mode | Exchange | +| --- | --- | +| `WHOLE_BODY` | Select `whole_body` with a positive `max_body_bytes` within the advertised limit. After `begin`, receive one `whole_body` with the complete body and trailers. Return `whole_body_result` with `unchanged` or `replacement`. Empty replacement bytes delete the body. This stage withholds its output until its `whole_body_result`. Later stages can select `DUPLEX`. | +| `DUPLEX` | After `begin`, receive `input_chunk` events and one `input_end` with trailers. Send `output_start`, any number of `output_chunk` results, and `finish` after `input_end`. Send every output byte, including unchanged bytes. Input and output chunk counts can differ. | + +A DUPLEX stage must send `output_start` even for empty output. Set its `output_body_bytes` when you know the exact output length. OpenShell uses `Content-Length` when the output declares a length and chunked framing otherwise. It flushes each server-sent event chunk. + +Your service can deny a request with `deny` or stop a response with `block_delivery` throughout inspection. After a terminal result, close your result stream or wait for the best-effort `session_end` notification. An unset, unknown, or out-of-order result fails closed. + +Until all stages finish, OpenShell withholds the final byte or terminating chunk that would complete the message. This prevents a late denial from leaving the recipient with a response or request that appears complete. + +#### Body Eligibility + +OpenShell removes body modes that cannot safely inspect or transform the message: + +| Condition | Requests | Responses | +| --- | --- | --- | +| Content encoding other than `identity` | No body modes | No body modes | +| Bodyless response, partial response, or `Cache-Control: no-transform` | Not applicable | No body modes | +| Declared length exceeds the stage's payload limit | No WHOLE_BODY | No WHOLE_BODY | +| `text/event-stream` or `multipart/x-mixed-replace` | No additional restriction | No WHOLE_BODY | +| Request or response is not HTTP/1.1 | No additional restriction | No DUPLEX | -A result can include an ordered list of header changes. A `write` change sets how it treats an existing header with the same name: +A WHOLE_BODY body that exceeds the selected limit fails closed. OpenShell does not switch it to DUPLEX. If no mode suits your service, continue without inspection or deny the message with a reason code that says why. + +The sandbox chooses a request's content encoding, so a service that must see request bodies should deny an encoded request rather than let it continue. An encoded response means the upstream ignored `Accept-Encoding: identity`. + +### Header and Trailer Changes + +A result can include an ordered list of header changes. A `write` change sets how it treats existing values: - `append` adds another value. - `overwrite` replaces all existing values. - `skip` keeps the existing values. -A `remove` change removes all values for the name. OpenShell protects credential, routing, framing, and connection headers, and for responses also status, authentication challenge, content coding, range, and security policy headers. Header values cannot contain control characters, and request header values cannot contain OpenShell credential placeholders. +A `remove` change removes all values for the name. OpenShell protects credential, routing, framing, connection, and content coding headers. For responses, it also protects status, authentication challenge, range, and security policy headers. Header values cannot contain control characters. Request values cannot contain OpenShell credential placeholders. -OpenShell applies each middleware's changes as a unit. If one change is invalid, it discards all of them and treats the result as a middleware failure. +A request header whose value contains an OpenShell credential placeholder, such as `X-Api-Key: openshell:resolve:env:API_KEY`, is a credential header: OpenShell omits every value of that name from the request preflight, `begin`, and `request_headers`, and your service cannot change or remove it. Legacy HTTP hooks still receive these headers. - - +OpenShell does not update integrity headers, such as `ETag`, `Content-MD5`, `Digest`, `Content-Digest`, and `Signature`, when a stage changes a body. If your stage changes the body, remove these headers or write new values in `whole_body_result` or `output_start`. A DUPLEX stage can read all input before it sends `output_start` if it needs the complete body to compute a value. -### Responses +OpenShell applies each stage's preflight changes and body-time changes before the next stage receives its preflight. `begin` carries that stage's preflight head plus its own preflight changes. Later stages cannot change an earlier stage's input. Trailer changes can modify or remove existing trailers but cannot add new names. -OpenShell calls your service after the external service responds and before the sandbox receives the response. At preflight, your service can skip inspection, block delivery, or choose how to inspect the response. During inspection, it can change or block the response. +OpenShell applies each stage's changes as a unit. If one change is invalid, it discards all of them and fails the stage. -```mermaid -sequenceDiagram - participant Sandbox - participant Supervisor - participant Middleware - participant Service as External service +### Failures - Service-->>Supervisor: HTTP response - Supervisor->>Middleware: Preflight with status and headers - Middleware-->>Supervisor: Skip, inspect, or block - Supervisor->>Middleware: Body, when inspecting - Middleware-->>Supervisor: Pass, change, or stop - Supervisor-->>Sandbox: Deliver response -``` +HTTP hooks always fail closed. The gateway rejects `on_error: fail_open` for these services. Each request or response chain must use one HTTP hook version; see [Mix HTTP Hook Versions](/extensibility/supervisor-middleware/configure#mix-http-hook-versions). - - +| Result | Before forwarding starts | After forwarding starts | +| --- | --- | --- | +| Request `deny` | `403 Forbidden`, `error: middleware_denied` | Close the upstream connection before the request completes. | +| Request failure | `403 Forbidden`, `error: middleware_failed` | Close the upstream connection before the request completes. | +| Response `block_delivery` | `403 Forbidden`, `error: middleware_denied` | Abort response delivery without an error body. | +| Response failure | `502 Bad Gateway`, `error: response_delivery_failed` | Abort response delivery without an error body. | -The preflight includes the status and headers. A preflight result can skip inspection, block delivery with `block_delivery`, or inspect the response with one body mode: +Denials include the policy entry's `middleware` name and the validated `reason_code`, if supplied. Failures include a platform-owned `failure_code` and identify the failed stage when known: -| Mode | Behavior | +| `failure_code` | Cause | | --- | --- | -| `HEADERS_ONLY` | Inspects the status and headers without the body. | -| `WHOLE_BODY_BYTES` | Receives the complete body in one piece before OpenShell sends anything to the sandbox. | -| `STREAM_BYTES` | Receives the body in ordered pieces of at most 64 KiB, so delivery can start before the body ends. | - -Body inspection is unavailable for responses without a body, partial responses, compressed responses, and responses with `Cache-Control: no-transform`. For those, a middleware that requests body inspection fails. Interim `1xx` responses pass through unchanged, and a `101` protocol upgrade skips response middleware. +| `middleware_cannot_inspect` | OpenShell could not provide the message head to the stage. | +| `middleware_timeout` | A binding, chain, or WHOLE_BODY deadline expired. | +| `stream_idle_timeout` | A DUPLEX stage stalled. | +| `middleware_unavailable` | The service was unreachable, ended with a gRPC status, or evaluation capacity was unavailable. | +| `middleware_lifecycle_violation` | An unexpected result, unimplemented RPC, or output length mismatch. | +| `body_over_limit` | A body exceeded its selected limit. | +| `header_mutation_*` | A header or trailer change broke the rules above. | +| `middleware_hook_versions_mixed` | The chain mixed HTTP hook versions. | - - +No free-form service error text reaches the sandbox. If evaluation capacity remains full for the chain budget, OpenShell returns `503 Service Unavailable`. A stalled request upload receives `408 Request Timeout` with `error: request_timeout`. -- Pass the response or body piece unchanged. -- Replace the body or a body piece. -- Add, change, or remove headers, with the same rules as requests. -- Block delivery with an optional reason code. + - - +Services without the `http-hooks` capability use `EvaluateHttpRequest` and `HttpResponsePreReturn.Evaluate`. These RPCs are deprecated for removal in 0.2.0. Legacy requests use a single buffered evaluation. Legacy responses select `HEADERS_ONLY`, `WHOLE_BODY_BYTES`, or `STREAM_BYTES` and use `block_delivery` to stop delivery. -If your service blocks the response before OpenShell starts sending it, the sandbox receives `403 Forbidden` with the same `middleware_denied` error as a denied request. If middleware fails before then, the sandbox receives `502 Bad Gateway` with `error: response_delivery_failed`. If delivery has already started, OpenShell aborts the response in both cases. +To migrate, require the `http-hooks` capability, implement the `HttpHooks` RPCs, and use `fail_closed`. Replace request evaluation and `WHOLE_BODY_BYTES` with WHOLE_BODY, `STREAM_BYTES` with DUPLEX, and `HEADERS_ONLY` with `continue_without_body`. Keep `block_delivery` for responses. The gateway and each sandbox supervisor read a service's `Describe` manifest when they connect to it, so after changing the service, restart the gateway, re-apply affected policies, and recreate sandboxes that use the service. Separate selectors from services still using legacy hooks, including `openshell/regex`. -The external service has already handled the request, so blocking its response does not undo it, and retrying the request may repeat side effects. +Upgrade the gateway before deploying the service. Recreate sandboxes whose supervisors do not support HTTP hooks. To support both contracts during migration, use separate service registrations. - -A service that inspects the body also receives trailers after the body. A preflight can declare new trailer names, and the service can then add them. - -After middleware changes the body, OpenShell recalculates `Content-Length` and framing, so your service never handles transfer encoding. - - - +The [content guard example](https://github.com/NVIDIA/OpenShell/tree/main/examples/supervisor-middleware-content-guard) implements both HTTP operations. ## WebSocket @@ -234,18 +231,29 @@ A denied message closes the connection with code `1008`. When middleware fails, +## Uninspectable Traffic + +OpenShell cannot pass connections to `tls: skip` endpoints, h2c, unsupported tunnels, raw TCP, or SQL passthrough to middleware. Middleware selects them by host like any other connection. If a selected entry is `fail_closed`, OpenShell refuses the connection. If every selected entry sets `on_error: fail_open`, OpenShell lets it through and records a finding. + +The gateway also rejects a policy in which a `fail_closed` entry can select a `tls: skip` endpoint. To allow such connections to a host, exclude the host from the entry's selector. + ## Limits | Limit | Applies to | Value | | --- | --- | --- | -| Payload per middleware | All operations | The registration's `max_payload_bytes`, up to 4 MiB. Applies to a request body, a whole response body or one streamed piece, or one text message, and to each replacement. | -| Time per evaluation or stream exchange | All operations | The smaller of the registration `timeout` and the binding timeout in the service manifest. Accepted HTTP response and WebSocket streams have no connection-wide deadline. | -| Streamed body piece | HTTP responses | 64 KiB. | -| Time for all middleware on one streamed piece | HTTP responses | 30 seconds. | -| Fragments per message | WebSocket messages | 4,096. | -| Time to assemble a message | WebSocket messages | 30 seconds without progress, or 2 minutes in total. | -| Time to forward a message | WebSocket messages | 2 minutes. | - -When a payload exceeds one middleware's limit, that middleware fails. Other middleware keep their own limits. - -Middleware evaluation capacity in each sandbox is shared by all operations. When it is full, OpenShell returns `503 Service Unavailable` for an HTTP request before reading its body. When WebSocket message assembly capacity is full, OpenShell closes the connection with code `1013`. +| Preflight | HTTP hooks | Binding timeout per stage. Initial preflights share a 30-second setup budget; each deferred preflight gets a new 30-second setup budget when its input headers arrive. Capacity waits use that budget. | +| Concurrent HTTP requests and responses | HTTP hooks | 64 in preflight or WHOLE_BODY processing, and 256 with DUPLEX or deferred stages, per sandbox. WHOLE_BODY stages of one request or response share a slot. A DUPLEX stage releases the buffering slot until another stage needs it. | +| Whole body | HTTP hooks | Stage payload limit, up to 4 MiB. Two minutes to receive, process, and deliver; individual exchanges also obey the binding timeout. | +| Duplex body | HTTP hooks | No total duration limit. No total size limit unless the preflight sets `limits.max_output_body_bytes` for buffered request output. Chunks are at most 64 KiB or the stage payload limit, whichever is smaller. | +| Middleware stall | HTTP DUPLEX | 30 seconds to accept the next input, or between results after `input_end`. Waiting for the sandbox or external service to produce input does not count. | +| Request upload stall | HTTP DUPLEX or deferred stages | 30 seconds without progress from the sandbox. | +| Payload per middleware | Legacy HTTP hooks and WebSocket | Registration `max_payload_bytes`, up to 4 MiB, for each body, response piece, message, or replacement. | +| Evaluation or stream exchange | Legacy HTTP hooks and WebSocket | Smaller of registration and binding timeouts. Accepted streams have no connection-wide deadline. | +| Streamed response piece | Legacy HTTP hooks | 64 KiB, with a 30-second chain budget per piece. | +| Fragments per message | WebSocket | 4,096. | +| Message assembly | WebSocket | 30 seconds without progress, or two minutes total. | +| Message forwarding | WebSocket | Two minutes. | + +A payload that exceeds one stage's limit fails that stage. Other stages retain their own limits. For HTTP payloads near 4 MiB, configure your gRPC server to send and receive at least 4.5 MiB messages because the envelope adds overhead. + +Middleware evaluation capacity is shared across operations. When it is full, OpenShell returns `503 Service Unavailable` before reading a request body. When WebSocket message assembly capacity is full, OpenShell closes the connection with code `1013`. diff --git a/docs/how-it-works/gateways/configuration.mdx b/docs/how-it-works/gateways/configuration.mdx index 3d5680301d..c58b4ad43b 100644 --- a/docs/how-it-works/gateways/configuration.mdx +++ b/docs/how-it-works/gateways/configuration.mdx @@ -778,7 +778,7 @@ max_payload_bytes = 262144 timeout = "500ms" ``` -Each service advertises operation and phase pairs through `Describe`. A sandbox policy attaches the service by its registration `name`; the gateway operator controls which bindings that attachment can use. Registration names must be unique and cannot start with the reserved `openshell/` namespace. The name in the service manifest is for diagnostics only. +Each service advertises operation and phase pairs through `Describe`. A sandbox policy attaches the service by its registration `name`; the gateway operator controls which bindings that attachment can use. Registration names must be unique and cannot start with the reserved `openshell/` namespace. The name in the service manifest is for diagnostics only. A service that requires the `openshell.supervisor-middleware.http-hooks` capability receives its HTTP bindings through the [HTTP hooks](/extensibility/supervisor-middleware/operations#http). | `binding_policy` | What the gateway and supervisor require | | --- | --- | @@ -801,7 +801,7 @@ The gateway connects to every registered service and validates `Describe` before Startup also requires compatible [extension protocol metadata](/extensibility/overview#protocol-negotiation). Upgrade legacy gateways and operator-run services together during a coordinated outage; neither mixed legacy/current pairing is supported. A service rejects a gateway that omits peer metadata, and a gateway rejects a service that omits metadata, uses another protocol major, or requires unsupported gateway capabilities. Protected `GetGatewayInfo` and `openshell gateway info` report the resulting non-secret snapshot. -`max_payload_bytes` is the shared operator limit for inspectable logical payloads across the bindings authorized for this service. It caps HTTP request and response units, replacement bodies, and complete WebSocket text messages and replacements. Whole-response inspection uses it as the stage's total body limit. Streaming response inspection applies it to each unit. The value must be greater than zero, no larger than each binding's advertised `max_payload_bytes` capability, and no larger than the 4 MiB platform maximum. OpenShell rejects oversized values instead of silently clamping them. Binary WebSocket messages are not exposed to V1 middleware, so this field does not limit binary pass-through. Middleware gRPC servers should allow messages of at least 4 MiB plus 293 KiB so a maximum-size payload and its protobuf envelope fit on the transport. +`max_payload_bytes` limits how much payload the service inspects at once: an HTTP body or body unit, a replacement body, or a WebSocket text message. It applies to every binding the registration authorizes. The value must be at most 4 MiB and no larger than any of those bindings advertises; OpenShell rejects larger values instead of clamping them. Binary WebSocket messages pass through without inspection, so the limit does not apply to them. Configure middleware gRPC servers to accept messages of at least 4 MiB plus 293 KiB so a maximum-size payload and its protobuf envelope fit. `timeout` is the operator-configured service-wide RPC timeout. It accepts the same compact duration syntax as gateway interceptors: an integer followed by `ms` or `s`, such as `500ms` or `2s`. Values must be between `10ms` and `30s`, inclusive. Omit the field to use the 500 ms platform default. A binding may advertise a shorter `timeout` in the `Describe` manifest, but it cannot extend the operator-configured deadline; OpenShell uses the smaller value. OpenShell validates both levels before accepting the service. The operator-configured service timeout applies to `Describe` and `ValidateConfig`. The effective binding timeout applies to HTTP request evaluation, HTTP response preflight and unit exchanges, WebSocket preflight, and each WebSocket message. Accepted streaming protocols have no connection-wide RPC deadline. diff --git a/docs/how-it-works/policies/schema.mdx b/docs/how-it-works/policies/schema.mdx index 693efb209c..15123537c5 100644 --- a/docs/how-it-works/policies/schema.mdx +++ b/docs/how-it-works/policies/schema.mdx @@ -543,13 +543,16 @@ network rules allow, in ascending `order`. | `endpoints.exclude` | list of strings | No | Host patterns to skip. Takes precedence over `include`. | | `order` | integer | No | Run order. Lower values run first. Values must be unique. Defaults to `0`. | | `config` | object | No | Configuration for the middleware. | -| `on_error` | string | No | `fail_closed` blocks traffic when the middleware fails. `fail_open` skips it. Defaults to `fail_closed`. | +| `on_error` | string | No | `fail_closed` blocks traffic when the middleware fails. `fail_open` skips it. Defaults to `fail_closed`. Services that use HTTP hooks accept only `fail_closed`. | | `name` | string | No | Display name. Defaults to the key. | -Host patterns match the same way as endpoint hosts, up to 32 patterns per -configuration. A `fail_closed` configuration cannot apply to an endpoint with -`tls: skip`. Refer to [Supervisor -Middleware](/extensibility/supervisor-middleware). +Host patterns match the same way as endpoint hosts and cannot include a port, +up to 32 patterns per configuration. An IPv6 host must be a complete address +without brackets, such as `2001:db8::1`. A `fail_closed` configuration +cannot apply to an endpoint with `tls: skip`. Two configurations whose +host patterns may overlap must use the same HTTP hook version for the operations +both handle. Refer to [Supervisor Middleware +Configuration](/extensibility/supervisor-middleware/configure#mix-http-hook-versions). ```yaml showLineNumbers={false} network_middlewares: diff --git a/docs/observability/logging.mdx b/docs/observability/logging.mdx index d7f068dca9..9334ce14bd 100644 --- a/docs/observability/logging.mdx +++ b/docs/observability/logging.mdx @@ -250,9 +250,9 @@ An upstream that the proxy cannot reach returns `502 Bad Gateway`: } ``` -The `error` field is a short machine-readable code (`policy_denied`, `middleware_denied`, `middleware_failed`, `ssrf_denied`, `upstream_unreachable`, `unsupported_l7_protocol`, `token_grant_failed`). The `detail` field is a human-readable explanation suitable for display in an agent transcript. The optional `reason` field, when present, provides the specific denial cause from the policy engine (for example, which binary was not allowed or which rule was missing). +The `error` field is a short machine-readable code (`policy_denied`, `middleware_denied`, `middleware_failed`, `ssrf_denied`, `upstream_unreachable`, `unsupported_l7_protocol`, `token_grant_failed`, or `request_timeout` with `408` when the sandbox stops sending a request body that streaming middleware holds). The `detail` field is a human-readable explanation suitable for display in an agent transcript. The optional `reason` field, when present, provides the specific denial cause from the policy engine (for example, which binary was not allowed or which rule was missing). -For L7 REST policy denials, the body also includes structured policy fields such as `method`, `path`, `rule_missing`, and `next_steps`. When the policy advisor is enabled, the body also includes `agent_guidance`, a short plain-language instruction telling the agent to read `/etc/openshell/skills/policy_advisor.md`, propose the narrowest rule through `http://policy.local/v1/proposals`, wait for `policy_reloaded: true`, and retry. A middleware denial instead identifies the policy-local config in `middleware` and can include a validated `reason_code`. A fail-closed runtime failure uses `middleware_failed` with platform-owned text. Both middleware responses omit `rule_missing`, `next_steps`, and `agent_guidance` because no policy rule is missing. +For L7 REST policy denials, the body also includes structured policy fields such as `method`, `path`, `rule_missing`, and `next_steps`. When the policy advisor is enabled, the body also includes `agent_guidance`, a short plain-language instruction telling the agent to read `/etc/openshell/skills/policy_advisor.md`, propose the narrowest rule through `http://policy.local/v1/proposals`, wait for `policy_reloaded: true`, and retry. A middleware denial instead identifies the policy-local config in `middleware` and can include a validated `reason_code`. A fail-closed runtime failure uses `middleware_failed` with platform-owned text. Under [HTTP hooks](/extensibility/supervisor-middleware/operations#failures), it also carries a platform-owned `failure_code` and, when one stage failed, its `middleware`. Both middleware responses omit `rule_missing`, `next_steps`, and `agent_guidance` because no policy rule is missing. A failed [dynamic token grant](/how-it-works/providers/profiles#token-grant-failures) returns `502 Bad Gateway` with an `x-openshell-request-id` header. The forward proxy body is `{"error":"token_grant_failed","detail":"dynamic token grant failed"}`; L7 relays send an empty body. Neither states the cause, so give the request ID to someone who can read the sandbox. diff --git a/proto/supervisor_middleware.proto b/proto/supervisor_middleware.proto index 6c0cc947da..14ffe09f21 100644 --- a/proto/supervisor_middleware.proto +++ b/proto/supervisor_middleware.proto @@ -10,8 +10,8 @@ import "google/protobuf/duration.proto"; import "extension.proto"; // SupervisorMiddleware discovers and configures one operator-run middleware. -// It evaluates HTTP requests and WebSocket messages before credentials. -// Phase-specific services share the same registration. +// It evaluates legacy HTTP requests and WebSocket messages. +// Hook-specific services share the same registration and endpoint. service SupervisorMiddleware { // Describe returns the service manifest and declared bindings. rpc Describe(MiddlewareDescribeRequest) returns (MiddlewareManifest); @@ -19,9 +19,14 @@ service SupervisorMiddleware { // ValidateConfig checks service-specific configuration for one binding. rpc ValidateConfig(ValidateConfigRequest) returns (ValidateConfigResponse); + // Deprecated: legacy HTTP hooks, removed in 0.2.0. Use + // HttpHooks.EvaluateRequest. // EvaluateHttpRequest returns an allow, deny, or mutation decision for one - // buffered HTTP request. - rpc EvaluateHttpRequest(HttpRequestEvaluation) returns (HttpRequestResult); + // buffered HTTP request. OpenShell calls it for the HTTP_REQUEST bindings of + // a service that does not require the HTTP hooks capability. + rpc EvaluateHttpRequest(HttpRequestEvaluation) returns (HttpRequestResult) { + option deprecated = true; + } // EvaluateWebSocketSession opens one ordered, phase-specific stream for a // single middleware stage and WebSocket upgrade attempt. The current @@ -34,20 +39,50 @@ service SupervisorMiddleware { returns (stream WebSocketSessionEventResult); } +// HttpHooks evaluates HTTP messages for one operator-run middleware. +// It shares SupervisorMiddleware's registration, discovery, configuration, and +// endpoint. Each invocation serves one binding phase, carried in its preflight. +service HttpHooks { + // HTTP hooks for requests. OpenShell calls it for the HTTP_REQUEST + // bindings of a service whose manifest lists + // openshell.supervisor-middleware.http-hooks in + // extension.required_capabilities. Peers that predate HTTP hooks + // reject such a service at Describe. One stream per request and middleware + // stage. The first event is a request preflight. + rpc EvaluateRequest(stream HttpRequestHookEvent) + returns (stream HttpRequestHookResult); + + // HTTP hooks for responses, for the HTTP_RESPONSE bindings of a + // service that requires the HTTP hooks capability. One stream per + // response and middleware stage. The first event is a preflight of the + // response head. + rpc EvaluateResponse(stream HttpResponseHookEvent) + returns (stream HttpResponseHookResult); +} + message MiddlewareDescribeRequest { // Gateway or supervisor protocol metadata. Middleware must reject unmet // requirements before accepting traffic. openshell.extension.v1.PeerMetadata gateway = 1; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. Use +// HttpHooks.EvaluateResponse. // HttpResponsePreReturn evaluates one response for one middleware stage before -// OpenShell returns it to the sandbox. +// OpenShell returns it to the sandbox. OpenShell calls it for the HTTP_RESPONSE +// bindings of a service that does not require the HTTP hooks +// capability. service HttpResponsePreReturn { + option deprecated = true; + + // Deprecated: legacy HTTP hooks, removed in 0.2.0. // Evaluate starts with preflight and may continue with selected body units // and trailers. A body unit marked end_of_stream ends body inspection, not // the event stream. Trailers and one best-effort session_end may follow. rpc Evaluate(stream HttpResponseEvent) - returns (stream HttpResponseEventResult); + returns (stream HttpResponseEventResult) { + option deprecated = true; + } } // MiddlewareManifest describes one middleware service and its bindings. @@ -81,7 +116,10 @@ message MiddlewareBinding { // Supported phase. SupervisorMiddlewarePhase phase = 2; // Maximum request body, WebSocket message, or response body unit/replacement. - // Required for payload-bearing operations. + // Required for payload-bearing operations. An HTTP binding of a service that + // requires the HTTP hooks capability may leave this zero: OpenShell + // then offers it no body modes, so it can only continue or deny at + // preflight. uint64 max_payload_bytes = 3; // Optional binding-specific RPC timeout. Empty uses the operator-configured // service timeout, or the 500ms platform default when that is also omitted. @@ -106,8 +144,380 @@ message ValidateConfigResponse { string reason = 2; } +// HTTP hooks. One ordered event in a HttpHooks.EvaluateRequest +// stream. Preflight is always first and sees the preceding stage's completed +// output headers. An inspect result is followed by begin +// and exactly one body lifecycle: whole_body for WHOLE_BODY, or input_chunk +// events ending with input_end for DUPLEX. session_end is best effort and has +// no result. +message HttpRequestHookEvent { + oneof event { + HttpRequestHookPreflight preflight = 1; + HttpBegin begin = 2; + HttpWholeBody whole_body = 3; + HttpInputChunk input_chunk = 4; + HttpInputEnd input_end = 5; + MiddlewareSessionEnd session_end = 6; + } +} + +// HTTP hooks. One ordered event in a HttpHooks.EvaluateResponse +// stream, in the order HttpRequestHookEvent describes. +message HttpResponseHookEvent { + oneof event { + HttpResponseHookPreflight preflight = 1; + HttpBegin begin = 2; + HttpWholeBody whole_body = 3; + HttpInputChunk input_chunk = 4; + HttpInputEnd input_end = 5; + MiddlewareSessionEnd session_end = 6; + } +} + +// HTTP hooks. One ordered result in a HttpHooks.EvaluateRequest +// stream. Preflight requires preflight_result or deny. WHOLE_BODY ends with +// whole_body_result or deny. DUPLEX sends output_start before any +// output_chunk, even for an empty output, and ends with finish or deny. An +// unset or unknown result fails the exchange closed. +// +// A service may close its result stream right after the result that ends its +// part in the exchange (continue_without_body, whole_body_result, finish, or +// deny), or keep it open until session_end. OpenShell accepts both. +message HttpRequestHookResult { + oneof result { + HttpPreflightResult preflight_result = 1; + HttpWholeBodyResult whole_body_result = 2; + HttpOutputStart output_start = 3; + HttpOutputChunk output_chunk = 4; + HttpFinish finish = 5; + HttpDenial deny = 6; + } +} + +// HTTP hooks. One ordered result in a HttpHooks.EvaluateResponse +// stream, under the rules HttpRequestHookResult describes, with +// block_delivery instead of deny. Blocking delivery does not undo the +// request already handled by the external service. +message HttpResponseHookResult { + oneof result { + HttpPreflightResult preflight_result = 1; + HttpWholeBodyResult whole_body_result = 2; + HttpOutputStart output_start = 3; + HttpOutputChunk output_chunk = 4; + HttpFinish finish = 5; + HttpDenial block_delivery = 6; + } +} + +// HTTP hooks body lifecycle a stage selects at preflight. +enum HttpBodyMode { + HTTP_BODY_MODE_UNSPECIFIED = 0; + // One complete body and one terminal result. OpenShell holds the body in + // memory, up to the selected max_body_bytes. A larger body, such as a + // chunked body without a declared length, fails the exchange; OpenShell + // does not fall back to DUPLEX. + HTTP_BODY_MODE_WHOLE_BODY = 1; + // Independent ordered input and output streams. The stage returns every + // output byte and owns any working storage. No total time limit, and no + // total size limit unless limits.max_output_body_bytes is set. + HTTP_BODY_MODE_DUPLEX = 2; +} + +// Why OpenShell does not offer a body mode for a message. +enum HttpBodyUnavailableReason { + HTTP_BODY_UNAVAILABLE_REASON_UNSPECIFIED = 0; + // The response has no body: the request method is HEAD, or the status is + // 1xx, 204, or 304. + HTTP_BODY_UNAVAILABLE_REASON_BODYLESS = 1; + // The response carries part of a representation: status 206, a + // Content-Range header, or multipart/byteranges content. + HTTP_BODY_UNAVAILABLE_REASON_PARTIAL = 2; + // Cache-Control no-transform forbids changing the response body. + HTTP_BODY_UNAVAILABLE_REASON_NO_TRANSFORM = 3; + // The request or response body has a content coding other than identity. + // Middleware would only see the encoded bytes. The sandbox chooses a + // request's coding, so a stage that must see request bodies denies an + // encoded one. When a response stage may select a body mode, OpenShell + // sends Accept-Encoding: identity upstream, so an encoded response means + // the upstream ignored it. + HTTP_BODY_UNAVAILABLE_REASON_ENCODED = 4; + // The declared body length exceeds the stage's payload limit. + HTTP_BODY_UNAVAILABLE_REASON_OVER_LIMIT = 5; + // The response is open-ended, such as text/event-stream or + // multipart/x-mixed-replace, and has no complete body to buffer. + HTTP_BODY_UNAVAILABLE_REASON_OPEN_ENDED = 6; + // The client could not detect a truncated response body, because the + // client request or the response status line uses HTTP/1.0 and the + // response cannot be chunked. + HTTP_BODY_UNAVAILABLE_REASON_TRUNCATION_UNDETECTABLE = 7; +} + +// A body mode a preflight does not offer, and why. +message HttpBodyModeUnavailable { + HttpBodyMode mode = 1; + HttpBodyUnavailableReason reason = 2; +} + +// HTTP hook preflight for one request stage: what is evaluated, the +// body modes offered, and limits. +message HttpRequestHookPreflight { + // A request after network-policy admission and before credentials. + HttpRequestPreflightHead request = 1; + // Phase of the HTTP_REQUEST binding this stream serves: PRE_CREDENTIALS. + SupervisorMiddlewarePhase phase = 2; + // Sandbox, request, and originating process identity. Limited to 4 KiB + // encoded. + RequestContext context = 3; + // Built-in middleware name or operator-owned registration name. + string middleware_name = 4; + // Validated service configuration. Limited to 64 KiB encoded. + google.protobuf.Struct config = 5; + // Body modes this request is eligible for. Empty permits only + // continue_without_body or deny. When no permitted mode suits the stage, + // it continues or denies, with a reason_code saying why. + repeated HttpBodyMode permitted_body_modes = 6; + // One entry for each body mode that permitted_body_modes omits, with the + // rule that removed it. Empty when the binding's max_payload_bytes is 0, + // which offers no body mode. + repeated HttpBodyModeUnavailable unavailable_body_modes = 7; + HttpBodyLimits limits = 8; + // Declared length of this stage's input, when known from the original + // framing or the preceding stage's output. Unset when that length is unknown. + optional uint64 declared_body_bytes = 9; +} + +// HTTP hook preflight for one response stage: what is evaluated, the +// body modes offered, and limits. +message HttpResponseHookPreflight { + // A final upstream response before it reaches the sandbox. + HttpResponsePreflightHead response = 1; + // Phase of the HTTP_RESPONSE binding this stream serves: PRE_RETURN. + SupervisorMiddlewarePhase phase = 2; + // Sandbox, request, and originating process identity. request_id matches + // the request preflights of the request this response answers. Limited to + // 4 KiB encoded. + RequestContext context = 3; + // Built-in middleware name or operator-owned registration name. + string middleware_name = 4; + // Validated service configuration. Limited to 64 KiB encoded. + google.protobuf.Struct config = 5; + // Body modes this response is eligible for. Empty permits only + // continue_without_body or block_delivery. When no permitted mode suits the + // stage, it continues or blocks delivery, with a reason_code saying why. + repeated HttpBodyMode permitted_body_modes = 6; + // One entry for each body mode that permitted_body_modes omits, with the + // rule that removed it. A stage can tell a bodyless response, which it may + // let continue, from a body it cannot inspect. Empty when the binding's + // max_payload_bytes is 0, which offers no body mode. + repeated HttpBodyModeUnavailable unavailable_body_modes = 7; + HttpBodyLimits limits = 8; + // Declared length of this stage's input, when known from the original + // framing or the preceding stage's output. Unset when that length is unknown. + optional uint64 declared_body_bytes = 9; +} + +// Request head after network-policy admission and before credentials. +message HttpRequestPreflightHead { + // Admitted destination and request target. Limited to 32 KiB encoded. + HttpRequestTarget target = 1; + // Request headers after all earlier stages' mutations, in wire order. + // Repeated names stay separate. Credential, routing, framing, and hop-by-hop + // headers are omitted, as is every header with a name that carries an + // OpenShell credential placeholder in any of its values. Limited to 128 + // lines and 64 KiB encoded. + repeated HttpHeader headers = 2; +} + +// Final upstream response head before delivery to the sandbox. +message HttpResponsePreflightHead { + // Admitted request target with a redacted query. Limited to 32 KiB encoded. + HttpRequestTarget target = 1; + // Final non-informational upstream status. Upgrades are not evaluated. + uint32 status_code = 2; + // Response headers after all earlier stages' mutations, in wire order. + // Credential, routing, and hop-by-hop headers are omitted. Limited to 128 + // lines and 64 KiB encoded. + repeated HttpHeader headers = 3; + // Headers of the request this response answers, as the request middleware + // chain forwarded them: after every request stage's mutations and before + // OpenShell adds credentials or makes its own forwarding changes, such as + // sending Accept-Encoding: identity. In wire order, with the omissions and + // limits of HttpRequestPreflightHead.headers. + repeated HttpHeader request_headers = 4; +} + +// Limits for one stage exchange. max_whole_body_bytes is 0 unless WHOLE_BODY +// is permitted, and max_chunk_bytes is 0 when no body mode is permitted. +// DUPLEX has no total deadline, and no total byte cap unless +// max_output_body_bytes is set. WHOLE_BODY has 2 minutes to receive, process, +// and deliver one body, and each body exchange is also bounded by the binding +// timeout. +message HttpBodyLimits { + // Largest input_chunk and output_chunk payload. + uint64 max_chunk_bytes = 1; + // Largest WHOLE_BODY input and replacement. + uint64 max_whole_body_bytes = 2; + // DUPLEX middleware-stall timeout. A DUPLEX stage fails when it does not + // accept the next input within this duration, or returns nothing within + // this duration after input_end. Time the sandbox or upstream takes to + // produce input does not count. OpenShell uses 30 seconds. + google.protobuf.Duration idle_timeout = 3; + // Largest request body OpenShell forwards after every stage, set when + // DUPLEX is permitted and OpenShell holds the complete output before it + // contacts the upstream, such as to sign it. Output beyond it fails the + // message with body_over_limit. Unset when the output streams without a + // total cap. + optional uint64 max_output_body_bytes = 4; +} + +// Preflight decision. To deny, return deny (requests) or block_delivery +// (responses) instead, including when the stage cannot inspect the message. +message HttpPreflightResult { + oneof decision { + // Let the message continue without this stage seeing its body. + HttpContinue continue_without_body = 1; + // Inspect the body in one permitted mode. + HttpInspect inspect = 2; + } + // Ordered head mutations, applied to this stage's input before begin. + // The next stage sees these and this stage's body-time mutations together. + // Only visible end-to-end headers may change. Credential, routing, framing, + // and hop-by-hop headers are protected, as are Content-Encoding, response + // Content-Range, and request headers omitted because they carry a credential + // placeholder. OpenShell does not update integrity headers such + // as ETag, Content-MD5, or Content-Digest when a body changes: a stage that + // changes the body removes them or writes new values, typically in its late + // head mutations. Limited to 64 + // operations, 32 KiB of name/value data, and 64 KiB encoded. + repeated HeaderMutation header_mutations = 3; + MiddlewareDiagnostics diagnostics = 4; +} + +// The body bypasses this stage but remains eligible for later stages. +message HttpContinue {} + +// Selects one permitted body mode. +message HttpInspect { + oneof mode { + HttpWholeBodyMode whole_body = 1; + HttpDuplexMode duplex = 2; + } +} + +message HttpWholeBodyMode { + // Positive and no larger than the offered max_whole_body_bytes. The cap + // applies separately to the input and the replacement. + uint64 max_body_bytes = 1; +} + +// DUPLEX makes the stage responsible for all output from accepted preflight. +message HttpDuplexMode {} + +// Starts the selected body lifecycle. +message HttpBegin { + // The preflight head plus this stage's preflight mutations. Earlier + // stages have already finalized these headers; later stages cannot change + // this stage's input. Mutations in whole_body_result or output_start apply + // on top of this head before the next stage receives its preflight. + repeated HttpHeader headers = 1; +} + +// WHOLE_BODY input: exactly one complete body, possibly empty. +message HttpWholeBody { + bytes data = 1; + // Trailers of a chunked message, in wire order. + repeated HttpHeader trailers = 2; +} + +// WHOLE_BODY terminal success. +message HttpWholeBodyResult { + oneof body { + // Keep the body. + HttpUnchanged unchanged = 1; + // Replace the body. Present empty bytes delete it. + bytes replacement = 2; + } + // Head mutations applied after this stage's preflight mutations, before + // the next stage receives its preflight. + repeated HeaderMutation header_mutations = 3; + // Ordered trailer mutations. A write must name a trailer the input already + // carries; removing an absent name does nothing. + repeated HeaderMutation trailer_mutations = 4; + MiddlewareDiagnostics diagnostics = 5; +} + +message HttpUnchanged {} + +message HttpInputChunk { + // Nonempty bytes without transfer framing. Boundaries carry no meaning. + bytes data = 1; +} + +message HttpInputEnd { + // Sent exactly once after the input, including for an empty body. Trailers + // of a chunked message, in wire order. + repeated HttpHeader trailers = 1; +} + +// Starts DUPLEX output. OpenShell commits the outgoing head when the last +// stage in the chain starts output, unless that output declares an empty body. +message HttpOutputStart { + // Head mutations applied after this stage's preflight mutations, before + // the next stage receives its preflight. + repeated HeaderMutation header_mutations = 1; + // When present, the output is exactly this many bytes, and OpenShell frames + // the outgoing message with Content-Length. Otherwise a streamed message is + // chunked. Content-Length framing cannot carry trailers, so OpenShell drops + // the trailers of a message it frames this way. Until every stage finishes, + // OpenShell holds back what would complete the message: the terminating + // chunk, the final byte of a declared length, or the whole head of a + // declared empty body. + optional uint64 output_body_bytes = 2; +} + +message HttpOutputChunk { + // Nonempty output bytes. Input and output chunk counts are independent. + bytes data = 1; +} + +// DUPLEX terminal success. Valid only after output_start and after input_end +// was consumed. +message HttpFinish { + // Ordered trailer mutations, under the rules of + // HttpWholeBodyResult.trailer_mutations. + repeated HeaderMutation trailer_mutations = 1; + MiddlewareDiagnostics diagnostics = 2; +} + +// Terminal denial at preflight or during body processing. Until a response +// head reaches the sandbox, OpenShell answers the sandbox with the canonical +// 403 middleware denial with the validated reason_code. A message already +// partly delivered stays incomplete: OpenShell withholds its terminating chunk +// or final byte, trailers, and any error body, and closes that connection. +// A stage that cannot inspect an HTTP message, such as a body in a format it +// does not understand, denies it with a reason_code that says so. +message HttpDenial { + MiddlewareDiagnostics diagnostics = 1; +} + +message MiddlewareDiagnostics { + // Service text, never copied to denied responses or security logs. Maximum + // 4 KiB. + string reason = 1; + // Optional stable audit-safe code that OpenShell may return to the sandbox + // with a denial. Codes start with a lowercase ASCII letter and contain only + // lowercase ASCII letters, digits, and underscores, at most 64 bytes. + string reason_code = 2; + // Up to 32 audit-safe findings, each limited to 4 KiB encoded. + repeated Finding findings = 3; + // Non-secret diagnostic metadata, limited to 64 entries and 32 KiB. + map metadata = 4; +} + +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // HttpRequestEvaluation contains one buffered HTTP request to evaluate. message HttpRequestEvaluation { + option deprecated = true; // Evaluation phase selected for this request. SupervisorMiddlewarePhase phase = 1; // Sandbox and request identity available to the supervisor. @@ -130,7 +540,13 @@ message HttpRequestEvaluation { string middleware_name = 7; } -// HttpHeader is one HTTP header line. +// HttpHeader is one HTTP header or trailer field. +// +// Values are valid UTF-8. OpenShell rejects a request whose head is not valid +// UTF-8 before any middleware runs. When a response head is not valid UTF-8, +// an HTTP hook response chain fails closed with failure_code +// middleware_cannot_inspect, and legacy response hooks follow on_error. HTTP +// hooks also fail a message whose trailers are not valid UTF-8. message HttpHeader { // Lowercased header name. string name = 1; @@ -138,10 +554,12 @@ message HttpHeader { string value = 2; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // One ordered response event. A stream starts with preflight, may continue with // body units ending in end_of_stream, may then include trailers, and may end // with one best-effort session_end. message HttpResponseEvent { + option deprecated = true; oneof event { // Initial response head and request context. HttpResponsePreflight preflight = 1; @@ -154,9 +572,11 @@ message HttpResponseEvent { } } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Each preflight, body, and trailers event requires one ordered result. // session_end has no result. message HttpResponseEventResult { + option deprecated = true; oneof result { // Result for preflight. HttpResponsePreflightResult preflight_result = 1; @@ -167,8 +587,10 @@ message HttpResponseEventResult { } } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // HttpResponsePreflight exposes the current final response head to one stage. message HttpResponsePreflight { + option deprecated = true; // Request identity. request_id links request and response evaluations. // Limited to 4 KiB encoded. RequestContext context = 1; @@ -204,10 +626,12 @@ message HttpResponsePreflight { repeated HttpResponseBodyMode permitted_body_modes = 8; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Selects skip, inspect, or block. Diagnostic fields apply to every action. // Invalid diagnostics make the entire result a middleware failure handled // according to on_error. message HttpResponsePreflightResult { + option deprecated = true; oneof action { // Deliver unchanged without invoking on_error. HttpResponsePreflightSkip skip = 1; @@ -228,9 +652,13 @@ message HttpResponsePreflightResult { map metadata = 6; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Ends this stage successfully without body inspection. -message HttpResponsePreflightSkip {} +message HttpResponsePreflightSkip { + option deprecated = true; +} +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Blocks delivery as a successful decision regardless of on_error. OpenShell // evaluates results in policy order. Once it accepts a valid block, it stops // later middleware evaluation and ends every still-writable opened stage with @@ -252,10 +680,14 @@ message HttpResponsePreflightSkip {} // After response commitment, including during STREAM_BYTES, OpenShell aborts // downstream delivery. It does not inject an error body, a terminating chunk, // or an error trailer. OpenShell does not reuse the upstream connection. -message HttpResponseBlockDelivery {} +message HttpResponseBlockDelivery { + option deprecated = true; +} +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Selects body inspection and response-header mutations. message HttpResponsePreflightInspect { + option deprecated = true; // Required mode from permitted_body_modes. Invalid values fail according to // on_error. HttpResponseBodyMode body_mode = 1; @@ -266,8 +698,10 @@ message HttpResponsePreflightInspect { repeated HeaderMutation header_mutations = 2; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Controls which response-body units a stage receives. enum HttpResponseBodyMode { + option deprecated = true; // Invalid value handled according to on_error. HTTP_RESPONSE_BODY_MODE_UNSPECIFIED = 0; // Inspect only the response head. @@ -284,9 +718,11 @@ enum HttpResponseBodyMode { HTTP_RESPONSE_BODY_MODE_STREAM_BYTES = 3; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // One normalized body unit. Boundaries have no transport or application // meaning. message HttpResponseBodyUnit { + option deprecated = true; // Contiguous and stage-local, starting at 1. uint64 sequence = 1; oneof payload { @@ -305,6 +741,7 @@ message HttpResponseBodyUnit { bool end_of_stream = 3; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Result for one body unit. Units are processed in lockstep; V1 does not // support ownership transfer or cross-unit retention. Diagnostic fields apply // to every action. Invalid diagnostics make the entire result a middleware @@ -312,6 +749,7 @@ message HttpResponseBodyUnit { // until it validates the result, so fail-open can continue from the last input // OpenShell still owns. message HttpResponseBodyResult { + option deprecated = true; // Must match the next unit. Zero, gaps, duplicates, and regressions fail. uint64 sequence = 1; // Exactly one explicit action is required. @@ -339,13 +777,18 @@ message HttpResponseBodyResult { map metadata = 7; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Preserves the input unit. -message HttpResponseBodyPassThrough {} +message HttpResponseBodyPassThrough { + option deprecated = true; +} +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Finalizes this unit and ends the stage. This stage receives no later body or // trailer events. The current and later units continue through other stages. // For WHOLE_BODY_BYTES, this equals its nested action. message HttpResponseBodySkipRemaining { + option deprecated = true; // Exactly one action for the current unit. oneof current { // Forward the current unit unchanged. @@ -355,8 +798,10 @@ message HttpResponseBodySkipRemaining { } } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Replaces the complete input unit. message HttpResponseBodyTransform { + option deprecated = true; // Required replacement, limited to max_payload_bytes. Present empty data // deletes the input unit. The replacement fully accounts for this input unit; // middleware must not retain input bytes for a later unit in V1. @@ -366,15 +811,18 @@ message HttpResponseBodyTransform { } } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // The current normalized response trailers in wire order. Repeated names stay // as separate fields. A stage that completes WHOLE_BODY_BYTES or STREAM_BYTES // receives exactly one trailers event after its final body result, including // when this set is empty. SKIP, HEADERS_ONLY, semantically bodyless responses, // and stages ended by block, failure, or skip_remaining receive no trailers. message HttpResponseTrailers { + option deprecated = true; repeated HttpHeader headers = 1; } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // Applies ordered trailer mutations atomically. An empty mutation list // preserves the current trailers. A write may target only a case-insensitive // name present in the trailers event; V1 cannot create a trailer name. Removal @@ -384,6 +832,7 @@ message HttpResponseTrailers { // mutations make the entire result a middleware failure handled according to // on_error. message HttpResponseTrailersResult { + option deprecated = true; // At most 64 operations, 32 KiB of validated name/value data, and 64 KiB // encoded are accepted. repeated HeaderMutation trailer_mutations = 1; @@ -456,8 +905,16 @@ message MiddlewareExchangeProtocolError {} // Supervisor operation selected for middleware evaluation. enum SupervisorMiddlewareOperation { SUPERVISOR_MIDDLEWARE_OPERATION_UNSPECIFIED = 0; + // An HTTP request at PRE_CREDENTIALS. OpenShell calls + // HttpHooks.EvaluateRequest when the service requires the + // openshell.supervisor-middleware.http-hooks capability, and the + // deprecated EvaluateHttpRequest otherwise. SUPERVISOR_MIDDLEWARE_OPERATION_HTTP_REQUEST = 1; SUPERVISOR_MIDDLEWARE_OPERATION_WEBSOCKET_MESSAGE = 2; + // A final HTTP response at PRE_RETURN. OpenShell calls + // HttpHooks.EvaluateResponse when the service requires the HTTP + // hooks capability, and the deprecated HttpResponsePreReturn.Evaluate + // otherwise. SUPERVISOR_MIDDLEWARE_OPERATION_HTTP_RESPONSE = 3; } @@ -587,7 +1044,11 @@ message RequestContext { string request_id = 1; // Sandbox id that originated the request. string sandbox_id = 2; - // Workload process that originated the request, when available. + // Workload process that opened the connection, when OpenShell resolved it. + // The identity is per connection: every request on a reused connection + // carries the process resolved when the connection was admitted. When the + // context would exceed its size limit, OpenShell drops the farthest + // ancestors first, then the whole process. Process originating_process = 3; // Sandbox name that originated the request. For display and logging only. // Names are workspace-scoped and may be reused for different sandbox @@ -619,7 +1080,7 @@ message HttpRequestTarget { message Process { // Executable path for the originating process. string binary = 1; - // Process id within the sandbox. + // Process id within the sandbox. 0 when unknown. uint32 pid = 2; // Executable paths for ancestor processes, nearest parent first. repeated string ancestors = 3; @@ -685,8 +1146,10 @@ message HeaderMutation { } } +// Deprecated: legacy HTTP hooks, removed in 0.2.0. // HttpRequestResult contains the decision and optional request mutations. message HttpRequestResult { + option deprecated = true; // Allow or deny decision for this request. Decision decision = 1; // Free-form service diagnostic. OpenShell does not relay this text into diff --git a/rfc/0009-supervisor-middleware/README.md b/rfc/0009-supervisor-middleware/README.md index 1f6eacde7b..f35f87723d 100644 --- a/rfc/0009-supervisor-middleware/README.md +++ b/rfc/0009-supervisor-middleware/README.md @@ -19,6 +19,7 @@ links: |------|------------|--------| | 2026-07-17 | [#2010](https://github.com/NVIDIA/OpenShell/issues/2010) | Added unary HTTP request middleware with built-in and operator-run services. | | 2026-07-28 | [#2428](https://github.com/NVIDIA/OpenShell/issues/2428) | Added WebSocket preflight and text-message evaluation, and aligned the middleware API names, limits, and diagnostics. | +| 2026-10-08 | [#2431](https://github.com/NVIDIA/OpenShell/issues/2431), [#3307](https://github.com/NVIDIA/OpenShell/issues/3307) | Added HTTP hooks (`HttpHooks.EvaluateRequest` and `HttpHooks.EvaluateResponse`), selected by a required extension capability, with buffered and streamed bodies, and deprecated legacy HTTP hooks. | ## Summary @@ -298,7 +299,22 @@ For WebSocket traffic, a service advertises `WEBSOCKET_MESSAGE/PRE_CREDENTIALS` Inspectable WebSocket text input and replacements share the 4 MiB platform cap. The operator's `max_payload_bytes` is the shared HTTP-body and WebSocket-text ceiling, further constrained by each operation binding's capability. It does not bound binary pass-through, which retains a separate raw-frame safety limit. Logical messages use a protobuf `oneof` with `string text` and `bytes binary` variants; results use an optional matching replacement `oneof`, whose presence also represents an empty replacement without a separate boolean. Protobuf decoding enforces UTF-8 for text, and OpenShell rejects replacement variants that would change the message type. A complete text message holds one process-wide admission permit for its entire chain; preflight fan-out holds one permit until every stage resolves. Permit waiting is backpressure and does not consume the per-message deadline. Per-stage timeouts are also bounded by a 30-second total chain budget, which applies to HTTP chains too. This bound controls concurrency and peak buffered inspection memory; it is not rate limiting. -The `originating_process` is the same identity OpenShell resolves on the egress path - the binary, pid, and ancestor chain it uses for binary-scoped network policy and OCSF audit. It is per-connection rather than strictly per-request and is optional. Middleware must treat missing process data as unavailable rather than as an authorization failure. The initial implementation leaves this field unset until reliable propagation is available. +The `originating_process` is the same identity OpenShell resolves on the egress path - the binary, pid, and ancestor chain it uses for binary-scoped network policy and OCSF audit. It is per-connection rather than strictly per-request and is optional. Middleware must treat missing process data as unavailable rather than as an authorization failure. OpenShell sets it on every hook that carries a `RequestContext` (legacy HTTP hooks, HTTP hooks, and WebSocket preflights) with `pid` 0, and drops the farthest ancestors when the context would exceed its 4 KiB limit. + +> **Update - HTTP hooks:** The following contract text adds a second HTTP contract, the HTTP hooks, and deprecates the unary `EvaluateHttpRequest` and lockstep `HttpResponsePreReturn` RPCs above. The [Supported Operations](../../docs/extensibility/supervisor-middleware/operations.mdx#http) guide describes the full contract. + +#### HTTP hooks + +- **RPCs.** `HttpHooks.EvaluateRequest` and `HttpHooks.EvaluateResponse` are bidirectional streams, each with its own event and result messages. One stream serves one stage for one HTTP message. +- **Lifecycle.** Each stream starts with a preflight. The stage then continues without the body, rejects, or inspects the body as one `WHOLE_BODY` message or as a `DUPLEX` stream. HTTP preflights handle HTTP messages only, so connections OpenShell cannot inspect follow the chain-level rule above. +- **Limits.** A `DUPLEX` stream has no total time or size limit, unless the route buffers request output before upstream contact. Then the preflight reports the route's own buffering limit as `max_output_body_bytes`, such as 10 MiB for SigV4 body signing. +- **Failure behavior.** HTTP hooks always fail closed. The gateway rejects `on_error: fail_open` for them. +- **Version selection.** A service selects HTTP hooks for all of its HTTP bindings by listing `openshell.supervisor-middleware.http-hooks` in its manifest's `extension.required_capabilities`, and keeps the `HTTP_REQUEST` and `HTTP_RESPONSE` operations. Peers that predate HTTP hooks do not support the capability, so they reject the service at `Describe` instead of calling the v1 RPCs. A capability listed only as supported would not have that effect, so the gateway rejects such a manifest. +- **One version per chain.** A service uses one HTTP hook version for all of its HTTP bindings. The gateway rejects policies whose overlapping selectors would put both hook versions in one request or response chain. +- **Deprecation.** `EvaluateHttpRequest`, `HttpResponsePreReturn`, and the messages only they use remain as legacy HTTP hooks. The protobuf contract marks them `deprecated` for removal in 0.2.0. +- **Content coding.** Neither hook version decodes content codings, and the sandbox controls `Accept-Encoding`. When an HTTP hook response stage may select a body mode, OpenShell sends `Accept-Encoding: identity` upstream, so the sandbox cannot make response bodies uninspectable by asking for compression. +- **Request headers in responses.** A response preflight carries the request headers as the request middleware chain forwarded them, recorded before OpenShell injects credentials or makes its own forwarding changes such as the `Accept-Encoding` rewrite. A response stage can use them without keeping per-request state. +- **Credential headers.** Like `Authorization`, a request header whose value carries a credential placeholder is a credential header. HTTP hooks never see it and cannot change it, so whether a stage sees a placeholder does not depend on the header that carries it. ### Relationship to RFC 0010 @@ -322,9 +338,9 @@ Intentional differences: ### Contract versioning -The middleware gRPC contract lives under a major-versioned protobuf package (`openshell.middleware.v1`), the same convention the compute-driver contract uses in [RFC 0001](../0001-core-architecture/README.md). Within a stable major version, changes stay additive and backward compatible - new fields, RPCs, operation phases, and manifest fields can be added - while breaking wire or semantic changes require a new major version. The research preview may still make intentional breaking changes before the contract is declared stable. +The middleware gRPC contract lives under a major-versioned protobuf package (`openshell.middleware.v1`), the same convention the compute-driver contract uses in [RFC 0001](../0001-core-architecture/README.md). Within a major version, changes are additive: new fields, RPCs, operation phases, and manifest fields can be added. When older peers cannot safely ignore an addition, an extension capability gates it: the service lists the capability in `required_capabilities`, so peers that predate it reject the service at `Describe` instead of misreading its manifest. Removing elements marked `deprecated` follows the [release stability policy](../0014-release-stability/README.md): a minor release may remove them with notice and migration guidance, and their names and field numbers are not reused. Other breaking wire or semantic changes require a new major version. The research preview may still make intentional breaking changes before the contract is declared stable. -The protobuf package is the wire-version handshake. `Describe` reports a diagnostic service name, implementation version, and stable binding IDs for supported hook points; it does not carry a second API-version field. Manifest validation is mandatory: if OpenShell cannot fetch the manifest, bindings conflict, a service claims the reserved `openshell/` namespace, or policy asks for an unsupported binding or invalid config, the gateway rejects the relevant configuration before traffic can depend on it. Runtime invocation failures are handled through `on_error` and use `fail_closed` by default. +The protobuf package is the wire-version handshake, and extension capabilities select optional contracts within it. `Describe` reports a diagnostic service name, implementation version, protocol metadata with capabilities, and stable binding IDs for supported hook points; it does not carry a second API-version field. The HTTP hooks are the first capability-gated contract: `openshell.supervisor-middleware.http-hooks` selects them for every HTTP binding of a service. Manifest validation is mandatory: if OpenShell cannot fetch the manifest, bindings conflict, a service claims the reserved `openshell/` namespace, or policy asks for an unsupported binding or invalid config, the gateway rejects the relevant configuration before traffic can depend on it. Runtime invocation failures are handled through `on_error` and use `fail_closed` by default. ### Registration and delivery