Skip to content
This repository was archived by the owner on Oct 17, 2026. It is now read-only.

feat(authoring): let the local agent drive recording on Windows, RDP, and Citrix - #77

Merged
abrichr merged 1 commit into
mainfrom
agent/native-agent-recording
Oct 17, 2026
Merged

abrichr merged 1 commit into
mainfrom
agent/native-agent-recording

Conversation

@abrichr

@abrichr abrichr commented Oct 17, 2026

Copy link
Copy Markdown
Member

serve --authoring forced CoachOnlySession for windows/rdp/citrix (authoring.py:136-137 and :675-681 on main), and the README said a person has to do the recording there.

New flags pin a driver through Flow's build_backend:

  • --windows-agent-url (token from OAFLOW_AGENT_TOKEN). Connects only to a Windows agent server the operator started. It never spawns one.
  • --remote-window / --remote-window-title / --citrix: the local client window, screen-only.
  • --authoring-screens off|scrubbed|unscrubbed (default off). Adds a screen tool. scrubbed uses the repair scrubber and shares nothing if scrubbing fails.

Unpinned Windows stays coach-only. The hosted mailbox client stays coach-only, because hosted clicks are node_id only. A compat path keeps pinned Windows/RDP/Citrix observations working with openadapt-types releases that still have the coach-only rule.

Depends on OpenAdaptAI/openadapt-flow#527 (tests/test_authoring_native.py skips until a Flow release has SCREEN_ONLY_KINDS) and OpenAdaptAI/openadapt-types#47.
Local: 468 passed, 10 skipped against the released Flow. The 10 native tests pass against the Flow branch. ruff is clean.

… and Citrix

serve --authoring turned every native Windows, RDP, and Citrix session
into a CoachOnlySession, so the agent could only tell the person what to
click. New flags pin a driver through Flow's build_backend, the factory
replay uses:

- --windows-agent-url: WindowsBackend over a Flow Windows agent server the
  operator already started (token from OAFLOW_AGENT_TOKEN).
- --remote-window / --remote-window-title / --citrix: the local RDP or
  Citrix client window, screen-only.
- --authoring-screens scrubbed|unscrubbed: an operator-only switch that
  adds a screen tool returning the pinned frame, scrubbed through the
  repair scrubber unless the operator chose unscrubbed. Off by default.

Windows with nothing pinned is still coach-only, and the hosted mailbox
client is unchanged. Observe drops any tree for rdp/citrix and keeps a
pinned Windows observation when an older openadapt-types still has the
coach-only rule.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
abrichr merged commit 428759e into main Oct 17, 2026
9 checks passed
@abrichr
abrichr deleted the agent/native-agent-recording branch October 17, 2026 03:38
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant