flow: the compatibility matrix, checked against the runtime in CI

CAPABILITIES.md

@@ -185,6 +185,7 @@ improvising or retrying when something is off.
 | Run an API step in a durable run: checkpoint the span after the save is sent, never resend or redo a save that may have landed, and reconcile it by reading the record back | In review ([#1358](https://github.com/OpenAdaptAI/OpenAdapt/pull/1358)) | Synthetic | [API_TIER.md](packages/flow/docs/API_TIER.md#in-a-durable-run) |
 | Run an admitted API step under a governed run authorization and the Standard or Regulated profile: the authorization names the API step by digest, the person who admits it names the right-record parameter, and the outcome reads `VERIFIED` | In review ([#1361](https://github.com/OpenAdaptAI/OpenAdapt/pull/1361)) | Synthetic | [API_TIER.md](packages/flow/docs/API_TIER.md#under-governance) |
 | Defer an API save's read-back with pipelined read-back, as for a GUI save, and keep self-repair's probes and resumed run on the repaired GUI step | In review ([#1362](https://github.com/OpenAdaptAI/OpenAdapt/pull/1362)) | Synthetic | [API_TIER.md](packages/flow/docs/API_TIER.md#with-parallel-workers-pipelined-read-back-and-self-repair) |
+| Say, for every feature under every run profile, whether it works or is refused and why, checked against the runtime in CI | In review ([#1364](https://github.com/OpenAdaptAI/OpenAdapt/pull/1364)) | Synthetic | [COMPATIBILITY.md](packages/flow/docs/COMPATIBILITY.md) |
 | Run a worklist on several workers at once, one signed-in session each, each item claimed once, with a stop on any worker stopping all | In review ([#1296](https://github.com/OpenAdaptAI/OpenAdapt/pull/1296)) | Synthetic patients on a local OpenEMR test copy | `--workers`, [THROUGHPUT.md](packages/flow/docs/THROUGHPUT.md) |
 | Read a save back while the next item runs (pipelined read-back, REST, FHIR, GraphQL or SQL record checks) | In review ([#1296](https://github.com/OpenAdaptAI/OpenAdapt/pull/1296)) | Synthetic | `--readback-window`, [THROUGHPUT.md](packages/flow/docs/THROUGHPUT.md) |
 | A faster speed profile that keeps every check (one insert per plain text field, shorter page-quiet wait) | In review ([#1296](https://github.com/OpenAdaptAI/OpenAdapt/pull/1296)) | Synthetic | `runtime.speed: fast`, [THROUGHPUT.md](packages/flow/docs/THROUGHPUT.md) |

llms-full.txt

@@ -541,6 +541,7 @@ improvising or retrying when something is off.
 | Run an API step in a durable run: checkpoint the span after the save is sent, never resend or redo a save that may have landed, and reconcile it by reading the record back | In review ([#1358](https://github.com/OpenAdaptAI/OpenAdapt/pull/1358)) | Synthetic | [API_TIER.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/API_TIER.md#in-a-durable-run) |
 | Run an admitted API step under a governed run authorization and the Standard or Regulated profile: the authorization names the API step by digest, the person who admits it names the right-record parameter, and the outcome reads `VERIFIED` | In review ([#1361](https://github.com/OpenAdaptAI/OpenAdapt/pull/1361)) | Synthetic | [API_TIER.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/API_TIER.md#under-governance) |
 | Defer an API save's read-back with pipelined read-back, as for a GUI save, and keep self-repair's probes and resumed run on the repaired GUI step | In review ([#1362](https://github.com/OpenAdaptAI/OpenAdapt/pull/1362)) | Synthetic | [API_TIER.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/API_TIER.md#with-parallel-workers-pipelined-read-back-and-self-repair) |
+| Say, for every feature under every run profile, whether it works or is refused and why, checked against the runtime in CI | In review ([#1364](https://github.com/OpenAdaptAI/OpenAdapt/pull/1364)) | Synthetic | [COMPATIBILITY.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/COMPATIBILITY.md) |
 | Run a worklist on several workers at once, one signed-in session each, each item claimed once, with a stop on any worker stopping all | In review ([#1296](https://github.com/OpenAdaptAI/OpenAdapt/pull/1296)) | Synthetic patients on a local OpenEMR test copy | `--workers`, [THROUGHPUT.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/THROUGHPUT.md) |
 | Read a save back while the next item runs (pipelined read-back, REST, FHIR, GraphQL or SQL record checks) | In review ([#1296](https://github.com/OpenAdaptAI/OpenAdapt/pull/1296)) | Synthetic | `--readback-window`, [THROUGHPUT.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/THROUGHPUT.md) |
 | A faster speed profile that keeps every check (one insert per plain text field, shorter page-quiet wait) | In review ([#1296](https://github.com/OpenAdaptAI/OpenAdapt/pull/1296)) | Synthetic | `runtime.speed: fast`, [THROUGHPUT.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/THROUGHPUT.md) |

packages/flow/docs/API_TIER.md

@@ -220,6 +220,9 @@ Not yet covered, so the GUI steps run with that reason:
   probes and resumed run use the GUI steps, because the repair and the
   person's approval are about the GUI step ([SELF_REPAIR.md](SELF_REPAIR.md)).
 
+Every feature under every run profile, and the API step with each other
+feature, is in [COMPATIBILITY.md](COMPATIBILITY.md).
+
 ## When it doesn't run
 
 The API step runs only in a plain linear run. It doesn't run, and the GUI

packages/flow/docs/COMPATIBILITY.md

@@ -0,0 +1,180 @@
+# Compatibility: features and run profiles
+
+<!-- Generated by scripts/build_compatibility_doc.py from
+     openadapt_flow/compatibility.py. Edit those, not this file. -->
+
+Each feature either works under a run profile or is refused with a
+named reason. Nothing falls back silently. When a run takes the slower or
+narrower path, it records why: an API step's `api_fallback`, a
+pipelined read-back's `inline_reason`, or the command's refusal before
+anything runs. `tests/test_compatibility_matrix.py` checks every cell
+against the runtime, and fails when a cell that should work falls back
+or a refusal gives another reason.
+
+## The widest combination today
+
+A Regulated, durable, governed run with per-item approval, on parallel
+workers, using the API step for an admitted save, is the combination
+where the most work together: each worker's item sends one request a
+person approved, checks it with the system-of-record read-back, and
+reads `VERIFIED`. In that run the read-back runs inline (durable), and
+self-repair runs through the Python API under the Standard contract.
+Under a signed qualification admission (what `openadapt-flow run`
+requires for Standard and Regulated actuation) the API step and
+self-repair are still refused, each with its reason below.
+
+## Profiles
+
+- **Default (gated).** `openadapt-flow run` with no `--profile`: a run authorization from the run gate, and per-item approval (`each_item`), the default approval mode.
+- **Durable.** `--durable`: a checkpoint after every verified step, and pause and resume from the last one.
+- **Governed.** The Standard or Regulated profile contract with a run authorization from the run gate: durable, an identity check on every save, and a record check that reads the system of record. This is how the Python API and the tutorial run it.
+- **Pinned.** `--pin-digest`: the run refuses any bundle but the one with that sealed content digest.
+- **Qualified.** A signed production qualification admission (`--qualification-authority-file`), which `openadapt-flow run` requires for Standard and Regulated actuation.
+- **Managed.** A Cloud-dispatched run (`--managed-dispatch-file`), whose run authorization comes from Cloud.
+
+## Features
+
+- **API step** ([API_TIER.md](API_TIER.md)). An admitted save compiled from network capture, run as one request in place of its GUI steps.
+- **Pipelined read-back** ([THROUGHPUT.md](THROUGHPUT.md)). `--readback-window`: an item's record check runs while the next item starts.
+- **Self-repair** ([SELF_REPAIR.md](SELF_REPAIR.md)). `--self-repair`: an agent proposes a fix for a halt on a changed screen; probes and a person's approval decide it.
+- **Standard changes** ([SELF_REPAIR.md](SELF_REPAIR.md#standard-changes-repairs-without-a-person)). Self-repair fixes of a class a person approved in advance, in a signed repair policy.
+- **Parallel workers** ([THROUGHPUT.md](THROUGHPUT.md)). `--workers N`: worklist items on several signed-in sessions at once.
+- **Multi-surface** ([MULTI_SURFACE.md](MULTI_SURFACE.md)). One program across several windows, tabs or apps.
+
+## At a glance
+
+| Feature | Default (gated) | Durable | Governed | Pinned | Qualified | Managed |
+| --- | --- | --- | --- | --- | --- | --- |
+| API step | Works | Works | Works | Works | Refused | Refused |
+| Pipelined read-back | Works | Refused | Refused | Works | Refused | Refused |
+| Self-repair | Works | Works | Works | Refused | Refused | Refused |
+| Standard changes | Works | Works | Works | Refused | Refused | Refused |
+| Parallel workers | Works | Works | Works | Works | Works | Refused |
+| Multi-surface | Works | Works | Works | Works | Works | Works |
+
+Every refusal names its reason (below), and a run or command that
+refuses says it in those words.
+
+## Features by profile
+
+### API step
+
+| Profile | Result |
+| --- | --- |
+| Default (gated) | Works: the person approves the exact request |
+| Durable | Works: a save that may have landed is never resent |
+| Governed | Works: needs `api admit --identity` |
+| Pinned | Works: the pin is checked before the run |
+| Qualified | Refused: a production qualification admission doesn't cover API steps yet, so the qualified GUI path runs |
+| Managed | Refused: a managed dispatch doesn't bind API steps yet |
+
+### Pipelined read-back
+
+| Profile | Result |
+| --- | --- |
+| Default (gated) | Works: GUI and API saves |
+| Durable | Refused: durable runs verify inline (the record check runs before the next item) |
+| Governed | Refused: durable runs verify inline (Standard and Regulated are durable) |
+| Pinned | Works: when the run isn't durable |
+| Qualified | Refused: durable runs verify inline (Standard and Regulated are durable) |
+| Managed | Refused: --workers and --readback-window apply to local runs, not a managed dispatch or a qualification case |
+
+### Self-repair
+
+| Profile | Result |
+| --- | --- |
+| Default (gated) | Works |
+| Durable | Works: self-repair always runs durable |
+| Governed | Works: Standard contract through the Python API |
+| Pinned | Refused: --pin-digest pins the original bundle |
+| Qualified | Refused: a signed qualification admission binds the original bundle; the approved bundle needs its own admission |
+| Managed | Refused: a managed dispatch binds the original bundle |
+
+### Standard changes
+
+| Profile | Result |
+| --- | --- |
+| Default (gated) | Works: needs a signed repair policy |
+| Durable | Works: needs a signed repair policy |
+| Governed | Works: needs a signed repair policy |
+| Pinned | Refused: --pin-digest pins the original bundle |
+| Qualified | Refused: a signed qualification admission binds the original bundle; the approved bundle needs its own admission |
+| Managed | Refused: a managed dispatch binds the original bundle |
+
+### Parallel workers
+
+| Profile | Result |
+| --- | --- |
+| Default (gated) | Works |
+| Durable | Works: each item has its own durable run |
+| Governed | Works: each item gets its own single-use authorization |
+| Pinned | Works |
+| Qualified | Works: the admission is checked for every item |
+| Managed | Refused: --workers and --readback-window apply to local runs, not a managed dispatch or a qualification case |
+
+### Multi-surface
+
+| Profile | Result |
+| --- | --- |
+| Default (gated) | Works: an API step inside it runs on the GUI (below) |
+| Durable | Works: resume re-binds each surface by identity |
+| Governed | Works |
+| Pinned | Works |
+| Qualified | Works: one environment boundary per surface |
+| Managed | Works: every backend surface must be bound |
+
+## The API step with the other features
+
+| With | Result |
+| --- | --- |
+| Parallel workers | Works: each item runs its own API step |
+| Pipelined read-back | Works: when the run isn't durable |
+| Self-repair | Refused: self-repair's probes and resumed run exercise the repaired GUI step, and the person approved that step, so its API step doesn't run (the first, ordinary run can use the API step) |
+| Standard changes | Refused: self-repair's probes and resumed run exercise the repaired GUI step, and the person approved that step, so its API step doesn't run (the repaired bundle's test-record run is GUI-only too) |
+| Multi-surface | Refused: a run across several windows or tabs uses the GUI steps |
+
+## Evidence
+
+Each cell names the tests that show it. They run in CI.
+
+- API step / Default (gated): `tests/test_api_step_approval.py::test_the_person_approves_the_exact_request_and_it_is_sent_once`, `tests/test_compatibility_matrix.py::test_api_step_runs_where_the_matrix_says`
+- API step / Durable: `tests/test_api_step_durable.py::test_a_durable_run_uses_the_api_step_and_checkpoints_after_it`, `tests/test_api_step_durable.py::test_an_uncertain_send_is_never_resent_by_resume`
+- API step / Governed: `tests/test_api_step_governed.py::test_the_default_profile_runs_the_admitted_api_step`
+- API step / Pinned: `tests/test_compatibility_matrix.py::test_api_step_runs_where_the_matrix_says`
+- API step / Qualified: `tests/test_compatibility_matrix.py::test_api_step_refuses_where_the_matrix_says`
+- API step / Managed: `tests/test_compatibility_matrix.py::test_api_step_refuses_where_the_matrix_says`
+- Pipelined read-back / Default (gated): `tests/test_readback_pipeline.py::test_confirmed_readback_settles_the_saved_report`, `tests/test_api_step_readback_repair.py::test_an_api_save_reads_back_while_the_next_item_runs`, `tests/test_compatibility_matrix.py::test_pipelined_readback_where_the_matrix_says`
+- Pipelined read-back / Durable: `tests/test_compatibility_matrix.py::test_pipelined_readback_where_the_matrix_says`
+- Pipelined read-back / Governed: `tests/test_compatibility_matrix.py::test_pipelined_readback_where_the_matrix_says`
+- Pipelined read-back / Pinned: `tests/test_compatibility_matrix.py::test_pipelined_readback_where_the_matrix_says`
+- Pipelined read-back / Qualified: `tests/test_compatibility_matrix.py::test_pipelined_readback_where_the_matrix_says`
+- Pipelined read-back / Managed: `tests/test_compatibility_matrix.py::test_the_worklist_pool_where_the_matrix_says`
+- Self-repair / Default (gated): `tests/test_self_repair_cli.py::test_run_self_repair_cli_resumes_after_one_phone_tap`, `tests/test_compatibility_matrix.py::test_self_repair_where_the_matrix_says`
+- Self-repair / Durable: `tests/test_self_repair.py::test_renamed_button_is_repaired_approved_on_the_phone_and_verified`
+- Self-repair / Governed: `tests/test_self_repair.py::test_renamed_button_is_repaired_approved_on_the_phone_and_verified`
+- Self-repair / Pinned: `tests/test_self_repair_cli.py::test_run_self_repair_refuses_a_pinned_bundle`, `tests/test_compatibility_matrix.py::test_self_repair_where_the_matrix_says`
+- Self-repair / Qualified: `tests/test_compatibility_matrix.py::test_self_repair_where_the_matrix_says`
+- Self-repair / Managed: `tests/test_compatibility_matrix.py::test_self_repair_where_the_matrix_says`
+- Standard changes / Default (gated): `tests/test_standard_change.py::test_relabelled_link_is_repaired_without_a_person`
+- Standard changes / Durable: `tests/test_standard_change.py::test_relabelled_link_is_repaired_without_a_person`
+- Standard changes / Governed: `tests/test_standard_change.py::test_relabelled_link_is_repaired_without_a_person`
+- Standard changes / Pinned: `tests/test_compatibility_matrix.py::test_self_repair_where_the_matrix_says`
+- Standard changes / Qualified: `tests/test_compatibility_matrix.py::test_self_repair_where_the_matrix_says`
+- Standard changes / Managed: `tests/test_compatibility_matrix.py::test_self_repair_where_the_matrix_says`
+- Parallel workers / Default (gated): `tests/test_cli_worklist_pool.py::test_two_workers_split_the_items_one_row_each`, `tests/test_compatibility_matrix.py::test_the_worklist_pool_where_the_matrix_says`
+- Parallel workers / Durable: `tests/test_compatibility_matrix.py::test_workers_run_governed_durable_api_items`
+- Parallel workers / Governed: `tests/test_compatibility_matrix.py::test_workers_run_governed_durable_api_items`
+- Parallel workers / Pinned: `tests/test_compatibility_matrix.py::test_the_worklist_pool_where_the_matrix_says`
+- Parallel workers / Qualified: `tests/test_compatibility_matrix.py::test_the_worklist_pool_where_the_matrix_says`
+- Parallel workers / Managed: `tests/test_compatibility_matrix.py::test_the_worklist_pool_where_the_matrix_says`
+- Multi-surface / Default (gated): `tests/test_multi_surface_governed.py::test_governed_durable_multi_surface_run_reaches_verified`
+- Multi-surface / Durable: `tests/test_multi_surface_identity.py::test_resume_rebinds_surfaces_by_identity`
+- Multi-surface / Governed: `tests/test_multi_surface_governed.py::test_governed_durable_multi_surface_run_reaches_verified`
+- Multi-surface / Pinned: `tests/test_multi_surface_governed.py::test_governed_durable_multi_surface_run_reaches_verified`
+- Multi-surface / Qualified: `tests/test_multi_surface_governed.py::test_case_evidence_needs_a_matching_observation_per_used_surface`
+- Multi-surface / Managed: `tests/test_multi_surface_contract.py::test_managed_dispatch_needs_every_backend_surface_bound`
+- API step / Parallel workers: `tests/test_compatibility_matrix.py::test_workers_run_governed_durable_api_items`
+- API step / Pipelined read-back: `tests/test_api_step_readback_repair.py::test_an_api_save_reads_back_while_the_next_item_runs`
+- API step / Self-repair: `tests/test_api_step_readback_repair.py::test_a_probe_or_resumed_run_never_sends_the_api_step`
+- API step / Standard changes: `tests/test_api_step_readback_repair.py::test_a_standard_change_test_record_run_is_gui_only`
+- API step / Multi-surface: `tests/test_compatibility_matrix.py::test_api_step_refuses_in_a_multi_surface_run`

packages/flow/openadapt_flow/__main__.py

@@ -3603,12 +3603,14 @@ def _cmd_run(args: argparse.Namespace) -> int:
         # Parallel worklist (docs/THROUGHPUT.md): every item gets its own
         # single-use authorization, bound to that item's one-row worklist and
         # otherwise identical to the run's (same gate, policy, approvals).
-        if dispatch_file or qualification_case is not None:
-            print(
-                "run REFUSED: --workers and --readback-window apply to local "
-                "runs, not a managed dispatch or a qualification case. Nothing "
-                "was executed."
-            )
+        from openadapt_flow.cli_worklist_pool import pool_profile_refusal
+
+        pool_refused = pool_profile_refusal(
+            managed=bool(dispatch_file),
+            qualification_case=qualification_case is not None,
+        )
+        if pool_refused is not None:
+            print(f"run REFUSED: {pool_refused}. Nothing was executed.")
             return 2
         try:
             relation = loop_relation(workflow)

packages/flow/openadapt_flow/cli_worklist_pool.py

@@ -60,6 +60,21 @@ def requested_pool(args: Any, cfg: Any) -> Optional[PoolPolicy]:
     )
 
 
+def pool_profile_refusal(*, managed: bool, qualification_case: bool) -> Optional[str]:
+    """Why ``--workers`` / ``--readback-window`` can't run here, or None.
+
+    Workers and a read-back window are for local runs. A managed dispatch and
+    a qualification case each bind one exact run.
+    """
+
+    if managed or qualification_case:
+        return (
+            "--workers and --readback-window apply to local runs, not a managed "
+            "dispatch or a qualification case"
+        )
+    return None
+
+
 def loop_relation(workflow: Any) -> str:
     """The relation of a single-LOOP program (the ``for-each`` shape).