feat(agent): teach a workflow over MCP, then call it with no model calls

.github/workflows/packages.yml

@@ -363,7 +363,7 @@ jobs:
           python-version: "3.12"
           cache: pip
       - name: Install
-        run: pip install -e ../types -e '.[dev]'
+        run: pip install -e ../types -e '.[dev,agent]'
       - name: Resolve Playwright version
         id: pw
         run: |
@@ -399,6 +399,7 @@ jobs:
             tests/e2e/test_structural_action.py \
             tests/e2e/test_compile_agent_e2e.py \
             tests/e2e/test_code_action_e2e.py \
+            tests/e2e/test_mcp_teach_e2e.py \
             --basetemp=runs/ci-critical --junitxml=runs/critical-junit.xml
       - name: Fail if any critical test was skipped or none ran
         run: |

.gitleaksignore

@@ -48,6 +48,10 @@ eb78575c688f06d7edc04afc69389aad10e1f5fa:packages/flow/tests/fixtures/terminal_v
 # branch commit that added them. The lines carry gitleaks:allow from the next commit on.
 3f0df2191a649540e79a621ac5406ebe2ff228e2:packages/flow/openadapt_flow/jit/authority.py:generic-api-key:101
 3f0df2191a649540e79a621ac5406ebe2ff228e2:packages/flow/openadapt_flow/jit/authority.py:generic-api-key:390
+# MCP teach (PR #1335): the generic-api-key rule matched the variable name `key_dir`
+# in a principal_from_key(...) call in two tests. No credential.
+e8ec96725c4451aa64cb97230c18d04b55d98503:packages/flow/tests/test_mcp_teach_surfaces.py:generic-api-key:147
+e8ec96725c4451aa64cb97230c18d04b55d98503:packages/flow/tests/e2e/test_mcp_teach_e2e.py:generic-api-key:122
 # PR #1367 (network capture redaction): the first commit's synthetic test values in
 # test_network_capture_redact_json.py; the next commit builds them at run time.
 52502adf407793c211fb3c0ac5aba37556206d8c:packages/flow/tests/test_network_capture_redact_json.py:generic-api-key:17

CAPABILITIES.md

@@ -21,7 +21,10 @@ shipping in the next release; a pull request number means it is being merged.
    the verified run into a program (`openadapt_flow.jit`, released;
    prompt-to-program `compile-agent`, landing; the agent can act through code
    instead of screenshots, [#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)). Agents can drive the recording
-   on web, native Windows, RDP and Citrix.
+   on web, native Windows, RDP and Citrix. An agent connected over MCP can
+   teach itself a new workflow and then call it as a tool
+   (`openadapt-agent serve --teach`,
+   [#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)).
 2. **Compiles to the fastest layer the app allows.** Page structure first (DOM,
    Windows UI Automation, macOS Accessibility), direct API calls where the app's
    own traffic allows them (`ApiActuator`, released; compiled from captured
@@ -218,6 +221,7 @@ improvising or retrying when something is off.
 | Code as action: the run-1 agent acts through one terminal tool (structure first, actions by role and name, a sandboxed read-only shell, screenshots on request); every action is a structured step checked against the compiled program | In review ([#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)) | Real app (local OpenEMR) and synthetic | `ClaudeAgent(vision="code")`, `CODE_AS_ACTION.md` ([#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)) |
 | Admit a draft by a person, an admitter key, or an agent under a person-signed policy, including self-admission when the policy allows it | Released | Unit tests | [JIT_ADMISSION.md](packages/flow/docs/JIT_ADMISSION.md) |
 | MCP server: your agent decides what to enter, OpenAdapt makes the entry and checks it | Released | Synthetic sandbox | `openadapt-agent`, [packages/agent](packages/agent/README.md) |
+| Teach over MCP: an agent does a new task once through OpenAdapt; the run is read back, compiled and admitted (a person confirms the record check; the agent admits only under a person-signed policy), and becomes a tool it calls with no model calls. Web, and Windows, macOS, Linux, RDP and Citrix with a person approving each save | In review ([#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)) | Synthetic (MockMed over MCP, and through the real RDP and Citrix backends) | `openadapt-agent serve --teach`, [packages/agent](packages/agent/README.md#teach-a-new-workflow-from-your-agent) |
 | Task ledger per job: rules locked when the job opens and enforced on every run, item status set only from OpenAdapt's own results, and yes-or-no questions a person answers on the phone or in Desktop, including when a run stops | In review ([#1332](https://github.com/OpenAdaptAI/OpenAdapt/pull/1332)) | Synthetic (30-item MockMed worklist, scripted agent, no model calls) | MCP tools `open_job`, `get_job`, `add_items`, `record_evidence`, `ask_user`, `get_answer`, `change_constraint`, `close_job`; [Track a long job](packages/agent/README.md#track-a-long-job) |
 | Compile an agent's first verified run from a prompt (`compile-agent`), and a code-as-action authoring agent | In review ([#1277](https://github.com/OpenAdaptAI/OpenAdapt/pull/1277), [#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)); the agent harness behind it runs on every backend, on main ([#1283](https://github.com/OpenAdaptAI/OpenAdapt/pull/1283)) | Synthetic | `compile-agent` |
 | Score harness optimizers (Harness², Meta-Harness) on GUI tasks with the record read-back as the reward, with full trace export | On main ([#1299](https://github.com/OpenAdaptAI/OpenAdapt/pull/1299), [#1300](https://github.com/OpenAdaptAI/OpenAdapt/pull/1300), [#1301](https://github.com/OpenAdaptAI/OpenAdapt/pull/1301)) | Synthetic (scripted proposer) | `integrations/harness2/` |
@@ -246,7 +250,7 @@ A test fails when a command has no file or this list is out of date.
 
 **Record and build a program:**
 
-- `compile-agent`
+- `compile-agent`: With `admit TASK_DIR`, a person confirms and admits a waiting draft, such as one an agent taught over MCP (`openadapt-agent serve --teach`).
 - `compile`
 - `compose`
 - `demo-record`

packages/agent/README.md

@@ -238,6 +238,8 @@ asyncio.run(main())
 
 The SDK starts the server with only a few environment variables. To pass others, such as `OPENADAPT_BUNDLE_KEY`, add `env=` to `StdioServerParameters`.
 
+To let your agent add workflows itself, see [Teach a new workflow from your agent](#teach-a-new-workflow-from-your-agent).
+
 [scripts/smoke_client.py](scripts/smoke_client.py) is a fuller example that runs every sandbox case. From a clone of this repository, run `pip install -e .`, then:
 
 ```bash
@@ -384,6 +386,46 @@ ChatGPT and Claude in a web browser can't start a server on your computer. To re
 
 It claims the job, polls openadapt.ai over outgoing HTTPS, and asks you in the terminal before a chat account may drive it. Don't confuse it with the launcher's `openadapt connect`, which pairs a computer with OpenAdapt Cloud. [docs/MAILBOX_CLI.md](docs/MAILBOX_CLI.md) has the details.
 
+### Teach a new workflow from your agent
+
+`serve --teach` lets your agent add a workflow it doesn't have yet, without leaving the conversation. It does the task once through OpenAdapt; OpenAdapt checks the record it saved, compiles the run, and once the run is admitted your agent calls it as a tool. Every later call runs the compiled program with no model calls and reads the save back.
+
+```bash
+claude mcp add --scope user openadapt-teach -- uvx --python 3.12 --from 'openadapt-agent[tutorial]' openadapt-agent serve --teach verify.yaml --url https://emr-test.example.org --runs-dir ~/.openadapt/runs
+```
+
+`verify.yaml` needs only a `verifier:` section that reads the app's records (`rest`, `sql`, `fhir`, `graphql` or `file`), the same file `openadapt-flow record --verify` uses. The first run makes a real entry in the app, so point `--url` at a test copy with test records.
+
+Your agent then:
+
+1. Calls `teach_start` with the task (`{note}` for each input), this run's `inputs`, the `record_fields` the saved record must have (such as its patient id), and `on_screen`, the record's identity as the screen shows it (a name and an MRN).
+2. Calls `teach_observe` and `teach_act` until the task is done. On the web, `teach_observe` lists the page as short lines with a ref for each control, and before every save OpenAdapt checks that the record's identity is shown next to the control. A save on the wrong record is refused and ends the run.
+3. Calls `teach_finish`. OpenAdapt compiles the run and reads the app's records: exactly one record must have appeared, with every `record_fields` value and every input. The result is the draft: its steps, its inputs, the proposed record check, and what still needs a person.
+4. Calls `teach_confirm_checks`. Your MCP client shows a person the proposed check, and only their answer there, with their name, confirms it. Your agent has no argument that confirms it.
+5. Calls `teach_admit` with `canary_inputs`, the values each canary saves. A canary is a real save on the same record, so use values that are safe there.
+
+`teach_admit` admits only under the rules in [JIT_ADMISSION.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/JIT_ADMISSION.md). The operator starts the server with `--teach-key agent.pem --teach-policy signed-policy.json`: the key of the agent registered with `openadapt-flow jit register --role agent`, and an admission policy a person signed with `openadapt-flow jit sign-policy`. The server records the run as that agent, so admitting it is self-admission, which the policy must allow (`allow_self_admission`). Without a key and a policy, `teach_admit` refuses and returns the command a person runs at a terminal instead:
+
+```bash
+openadapt-flow compile-agent admit <task dir> --admit-as "<your name>" --canary-param note="Canary test note"
+```
+
+Once the program is admitted, it is in `list_workflows` and `run_workflow` at once and has its own `taught_<name>` tool, and the server sends `notifications/tools/list_changed` to clients that support it. A server started later lists every program taught in its `--teach-dir`.
+
+It works on a web app (`--url`) and on the app a deployment file's `backend:` section names (`--config deploy.yaml`): a native Windows, macOS or Linux window, or a local RDP or Citrix client window. Off the web, OpenAdapt can't read which record a form shows, so a person approves, in your client's form, every action that might save: any click except on a text field, a check box, a tab or a control that plainly navigates (Open, New, Cancel), any click at x,y, Enter, typing outside a text field, and shortcuts. Each answer covers that one action on the screen as it was; if anything changes first, the action isn't done. A client without forms can't save. RDP and Citrix have no control tree, and the Windows, macOS and Linux backends don't list controls yet, so off the web your agent needs `--teach-screens scrubbed` to see the screen, and clicks x,y pixels. Admission there fails closed for now: a pixel-only click carries no checked record identity, so the canary is refused and says why. Network RDP (`rdp_host`) is refused.
+
+| Flag | What it does |
+| --- | --- |
+| `--teach VERIFY_YAML` | Adds `teach_start`, `teach_observe`, `teach_act`, `teach_finish`, `teach_abort`, `teach_status`, `teach_confirm_checks` and `teach_admit`, and turns on runs of taught workflows. |
+| `--teach-dir DIR` | Where drafts and taught programs live (default `RUNS_DIR/teach`). |
+| `--teach-key PEM`, `--teach-policy JSON` | The registered agent's encrypted key (passphrase in `OPENADAPT_JIT_KEY_PASSPHRASE`) and the signed admission policy. Only an agent key under a policy; an admitter key admits from CI with `openadapt-flow jit admit`. |
+| `--key-dir DIR` | The JIT key directory (default `OPENADAPT_JIT_KEY_DIR` or `~/.openadapt/jit`). |
+| `--teach-hold` | On the web, a person also approves every save in the first run. Without it, your agent names the identity the save gate checks (`on_screen`); a value every form shows proves nothing, and then only the record check after the run and the person confirming it catch a wrong record. |
+| `--teach-screens scrubbed` | Adds `teach_screen`: the app's screen with identifiers scrubbed, and nothing shared if scrubbing fails. `unscrubbed` skips scrubbing. Off by default. |
+| `--mode production --bundles DIR` | Serves your other workflows next to the taught ones. |
+
+[COMPILE_AGENT.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/COMPILE_AGENT.md) describes the pipeline underneath: the harness, the record check, the canaries and the files it keeps.
+
 ## Track a long job
 
 On long work, agents lose track of the rules they were given, miss information that arrives partway through, guess instead of asking, and skip checking their work ([OSWorld 2.0](https://arxiv.org/abs/2606.29537)). The task ledger keeps that state on the server, so your agent reads it instead of rebuilding it from its own history. It's on whenever runs or guarded saves are; `serve --no-ledger` turns it off.

packages/agent/docs/DESIGN.md

@@ -219,6 +219,7 @@ repeats the text. Every declared input stays required unless
 | `--mode production --bundles DIR` (or `--allow-run`) | production | the above plus `run_workflow` and deprecated `run_<opaque-id>` |
 | `--mode attended --bundles DIR` (or `--allow-run --allow-attended-actions`) | attended | the above plus Reject, Teach, Escalate; Continue and Skip with a qualified `--config` |
 | `--bundles DIR --allow-attended-actions` (older flags, no runs) | attended, read-only | the read-only tools plus the same decisions on paused runs |
+| `--teach VERIFY_YAML` with `--url` or `--config` | production (taught programs) | the production tools plus `teach_start`, `teach_observe`, `teach_act`, `teach_finish`, `teach_abort`, `teach_status`, `teach_confirm_checks`, `teach_admit`, `teach_screen` (with `--teach-screens`), and `taught_<name>` for each admitted program |
 
 The read-only tools return PHI-safe projections: workflow cards, opaque
 ids, availability, status, and count or boolean metrics. They do not
@@ -227,6 +228,14 @@ observed text, local paths, subprocess output, or exception text. The
 attention tools use the same boundary plus typed categories, artifact
 ids, and non-authorizing capability metadata.
 
+The teach tools are the exception. While the agent does run 1 of a task,
+`teach_observe` and `teach_act` return what it needs to find the right record:
+on the web, the page's text and controls, scrubbed by
+`openadapt_flow.privacy` except for the values the agent put in the task. Run
+1 makes a real entry, so the operator points it at a test copy. The draft
+report, the record check and the admission return the same kind of facts as
+the other tools, plus the proposed record check's field names and values.
+
 The deprecated per-workflow `run_<opaque-id>` tools keep their schemas and
 legacy `status` field and add the contract fields. They take no
 `request_id`, so their descriptions say a retry can write twice. Missing

packages/flow/docs/BACKEND_PARITY.md

@@ -99,6 +99,7 @@ drives an app).
 | `effects.kind: desktop_readback` ([#1280](https://github.com/OpenAdaptAI/OpenAdapt/pull/1280)) | refused: "a web record page uses effects.kind: ui_readback" | tested (UI Automation, WinForms stand-in) | works (OCR; Accessibility tree not read yet) | works (OCR; AT-SPI tree not read yet) | tested (network RDP, OCR) | works (OCR) |
 | `openadapt-agent serve --authoring` (pinning a window) | tested | tested | tested (frontmost window only) | **refused** | tested | tested |
 | `openadapt-agent` run tools | tested | works (surface from `--config`) | works | works | works | works |
+| `openadapt-agent serve --teach` (record, check, compile, admit over MCP) | tested (`tests/e2e/test_mcp_teach_e2e.py`) | works (a person approves every save) | works (same) | works (same; needs `linux_allow_physical_input`) | tested; admission refused: pixel identity | tested; admission refused: pixel identity |
 
 - `SurfaceApp`: `tests/test_jit_surface_app.py` (every kind: validation,
   origin binding, Citrix readiness refusal) and
@@ -128,6 +129,16 @@ drives an app).
   `--authoring --config deploy.yaml --authoring-from-config` for every
   surface except network RDP, which it refuses (each run's own session would
   disconnect the recording's).
+- `serve --teach`: `tests/test_mcp_teach_surfaces.py` teaches MockMed over
+  MCP through the real `RemoteDisplayBackend` and `CitrixWorkspaceBackend`:
+  run 1 records, passes its record check and a person confirms it; the canary
+  is refused with the identity-coverage reason above, so nothing is admitted.
+  `tests/test_compile_agent_surface_harness.py` covers the surface harness on
+  a native window (what is gated, nothing performed when refused). Off the
+  web the harness can't read which record a form shows, so a person approves
+  every gated action in the client's form. Native backends list no controls
+  yet (the `authoring_tree()` gap below), so the agent reads the screen
+  (`--teach-screens`) and clicks pixels there too.
 - An agent clicks at window coordinates. Native Linux refuses coordinate
   input unless the section sets `linux_allow_physical_input: true`
   ("coordinate input is disabled for native Linux"), so agent-driven
@@ -165,5 +176,5 @@ are not app URLs.
 | `route` on a desktop surface: no test | [#1264](https://github.com/OpenAdaptAI/OpenAdapt/pull/1264) | [#1288](https://github.com/OpenAdaptAI/OpenAdapt/pull/1288) |
 | `induce --probe-on` on a desktop test copy: no test | [#1276](https://github.com/OpenAdaptAI/OpenAdapt/pull/1276) | [#1289](https://github.com/OpenAdaptAI/OpenAdapt/pull/1289) |
 | `run --self-repair` refuses non-web: "only browser runs can be repaired for now" | [#1265](https://github.com/OpenAdaptAI/OpenAdapt/pull/1265) | refused with a reason; desktop support after it merges |
-| `compile-agent` takes `--backend`/`--config` but refuses every non-web target (exit 2, before anything runs); its observe step and save gate read the DOM | [#1277](https://github.com/OpenAdaptAI/OpenAdapt/pull/1277) | refused with a reason; desktop support (via `jit.SurfaceApp`) after it merges |
+| `compile-agent` takes `--backend`/`--config` but refuses every non-web target (exit 2, before anything runs); its Claude agent reads the DOM listing | [#1277](https://github.com/OpenAdaptAI/OpenAdapt/pull/1277) | refused with a reason; the same pipeline runs on every surface through `serve --teach` (`FirstRunCompiler(surface=...)` with the surface harness) |
 | The GTK3 AT-SPI qualification job is in `packages/flow/.github/`, which GitHub does not run in the monorepo | `.github/workflows/` | follow-up |