diff --git a/CAPABILITIES.md b/CAPABILITIES.md index c9125ded3..1ec2ac947 100644 --- a/CAPABILITIES.md +++ b/CAPABILITIES.md @@ -78,7 +78,11 @@ shipping in the next release; a pull request number means it is being merged. | Learns branching programs from several demonstrations, from repeated halts, and from many background runs. | `induce`, `teach-branch`, [SHADOW_MINING.md](packages/capture/docs/SHADOW_MINING.md), [LEARN_FROM_MANY_RUNS.md](packages/flow/docs/LEARN_FROM_MANY_RUNS.md) | | Chains programs across surfaces, and one program can switch surfaces, tabs and popups mid-run. | `compose`, `process`, [#1197](https://github.com/OpenAdaptAI/OpenAdapt/pull/1197) | | Uses page structure first on web, Windows, macOS and Linux, with pixels as the fallback for remote desktops. Benchmarks labelled "vision-only" ran a deliberately pixel-only arm. | [RESOLUTION_LADDER.md](packages/flow/docs/RESOLUTION_LADDER.md) | -| Runs steps as direct API calls; compiling them from captured network traffic is being merged. | `runtime/actuators/api.py`, [#1269](https://github.com/OpenAdaptAI/OpenAdapt/pull/1269) | +| Runs steps as direct API calls, compiled from the network traffic captured while recording. | `api`, [#1269](https://github.com/OpenAdaptAI/OpenAdapt/pull/1269) | +| Builds the record check itself. After a save, the app usually reads the record back to show it; `scaffold-verifier` finds that read in the captured traffic or a read-only database, builds the check from it and tries to break it before a person reviews it. No API or SQL needs to be written by hand. | [EFFECT_KIT.md](packages/flow/docs/EFFECT_KIT.md#a-record-check-built-from-the-apps-own-reads-scaffold-verifier) | +| Checks a save on desktop, RDP and Citrix apps with no API at all, by looking the record up in the app's own screens in a separate session. | [EFFECT_KIT.md](packages/flow/docs/EFFECT_KIT.md#separate-session-desktop-read-back-desktop_readback) | +| Keeps pixel-path runs working across display scaling from 100% to 175% and font smoothing on or off, and stops rather than saves when the record changed. | [benchmark/pixel_scale](packages/flow/benchmark/pixel_scale/README.md) | +| Lists every change made before a stop, and reverses them with undo steps a person approves. | `undo`, [AFTER_A_STOP.md](packages/flow/docs/AFTER_A_STOP.md) | | Repairs itself when the app changes: governed repair, agent-demonstrated fixes, fleet patches, and self-repair probed up to the save. | [REPAIR_LIFECYCLE.md](packages/flow/docs/REPAIR_LIFECYCLE.md) | | Compiles a computer-use agent's verified run, with no human demonstration needed. | [JIT.md](packages/flow/docs/JIT.md) | | The agent that does a task once can act through code, not a screenshot-and-click loop: it reads the page or accessibility tree, acts by role and name, keeps notes with a sandboxed read-only shell, and asks for a screenshot only when it needs one. It is arm 7 of the scoreboard; its figures are quoted from there once its rows are in. | `CODE_AS_ACTION.md` in [#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305) | diff --git a/packages/flow/README.md b/packages/flow/README.md index 24d148f7d..89174db88 100644 --- a/packages/flow/README.md +++ b/packages/flow/README.md @@ -232,8 +232,13 @@ A production run reports success only after it reads back every write it declared. Each workflow needs two things: 1. A description of what each write changes. `openadapt-flow - scaffold-verifier rec` drafts one from the recording's write steps. The - draft checks nothing until a person edits it. + scaffold-verifier rec` builds it from the app's own reads: after a save, + the app usually reads the record back to show it, and that read, in the + captured network traffic or a read-only database, becomes the check. The + command tries to break the check before a person reviews it + ([docs/EFFECT_KIT.md](docs/EFFECT_KIT.md#a-record-check-built-from-the-apps-own-reads-scaffold-verifier)). + When it finds no such read, the draft it writes checks nothing until a + person edits it. 2. A second way to read the record. A database connection isn't required. A read-only REST, FHIR, or GraphQL API works, and so does a read-only SQL query, a file, report, or email the app exports, or, when you have none