[thymikee]

Summary

Follow-up to #3699: Apple XCUITest and Limrun iOS select recognized alert labels by ranked preference and refuse ties/unknowns. While Using outranks Once; Open/Done/Confirm… and Keep Browsing remain recognized.

Nine files. Android ordinary-label effects are intentional and tested; grant-id policy is unchanged. The separate macOS AX helper, capture, activation, deadlines, and wire contracts are unchanged. Shared choice fixtures do not prove capture parity.

Maintainer feedback addressed: move TS golden tests to contracts, add all 28 adjacent rank boundaries in both orders (120 fixture cases), remove the test-only Swift helper, preserve #3627 safety/canary tests, and correct the macOS docs claim.

Validation

Head 80c8328df6fa387f29f874e597fc5578d73211bb; base 620228b8ca67a69a108fa4fc5041fdd88880b792. Relevant upstream #3824 required rebase; range-diff preserves the original commit and attribution; #3705 retained.

• Frozen install/build, repository format, XCTest selection, iOS/macOS builds passed.
• Focused Vitest: 179 passed. Swift golden XCTest: passed. Ordering mutation: 131 pass → 4 fail/127 pass → 131 pass. Swift Open removal: three golden assertions failed.
• Fresh independent Sol review: no findings, static only.
• Direct pnpm check:affected --run --head 80c8328df6fa387f29f874e597fc5578d73211bb: exit 0, receipt run-Qag7Om, exactCommit=true, clean initial/final trees; 4,662 related tests passed, 12 checks passed/2 deduplicated/10 skipped.

Live evidence confirms candidate iOS location/ATT/refusal and Android Done. Source provenance and owned-resource cleanup verified. CI is green, including Apple smoke on maintainer-triggered attempt 2; the original timeout cause remains unproven. Separate diagnostics follow-up tracks it. tvOS/visionOS builds and live Limrun remain unverified. Old fe2f429 evidence validates only that SHA.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-20 14:31 UTC

[github-actions]

Size Report

Metric	Base	Current	Diff
Installed (including dependencies)	5.45 MB	5.45 MB	+443 B
Package (unpacked)	5.45 MB	5.45 MB	+443 B
Package (download)	1.65 MB	1.65 MB	+216 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	38.8 ms	38.5 ms	-0.3 ms
CLI --help	76.7 ms	77.3 ms	+0.6 ms

[thymikee]

The code looks right at fe2f429, but the changed iOS runner route has not run on a device yet. The 18 checks are green, and drafts skip the device lanes, so CI does not cover this route. No conflicts.

The runner now chooses a different button on the local iOS alert route (daemon, runAppleAlert, chooseAlertButton, buttonIndex) in RunnerAlertButtonChoice.swift. It also changes the shared vocabulary that Android uses in alert-detection.ts. Only unit and golden tests ran, and the PR body says no live run happened. Unit fixtures cannot show that XCTest exposes the labels the table expects, for example whether "Allow While Using App" is read verbatim, or whether extra actionable elements create ties. Please build the runner from fe2f429 and run it on an iOS simulator through three states. First, run alert accept on a location sheet (Allow Once / Allow While Using App / Don't Allow) and show authorizedWhenInUse from the app status or a simctl/TCC read. Second, run alert dismiss on the ATT prompt and show TCC kTCCServiceUserTracking with auth_value=0. Third, run alert accept on an alert with a lone unknown button, or tied labels, and show error.code ALERT_BUTTON_NOT_FOUND with details.dispatched: no, then show alert get still reports the alert. Then run one Android emulator dialog with Cancel/Done (or Keep Browsing/Open) and show alert accept taps Done/Open.

Not blocking, take or leave: the new bullet in commands.md lists macOS under the ranked preferences, but alert.ts sends macOS alerts to the helper, which keeps a last-button fallback, so consider dropping macOS or saying "the Apple XCUITest runner (iOS, tvOS, visionOS simulators and devices)"; the hand-written cases in RunnerTests+AlertButtonChoiceTests.swift repeat the fixture cases, and isAcceptButton is now only called by its own test, so you could delete both and keep the end-to-end canary; and the TS golden test in alert-detection.test.ts would fit better in contracts/src/alert-contract.test.ts against selectRankedAlertButtonByRole, with adjacent-pair ordering cases or the ordered pattern lists in the fixture, since today swapping two non-adjacent patterns on one side would still pass.

I did not run the Swift or Vitest suites, and the claim that the base fails rests on reading the old code by hand. I could not check whether real alert roots expose extra actionable elements that would now tie and refuse where base picked the first match. tvOS and visionOS builds were not run, and Limrun live behavior was not checked. #3699 is unreleased, so Limrun has no compatibility obligation.

Before merge, please post the live iOS simulator proof from the fe2f429 runner (location accept picks While Using, ATT dismiss leaves auth_value 0, lone-unknown accept refuses with dispatched: no), plus the one Android dialog run.

[thymikee]

Reply to #3825 (comment)

Addressed the selection/test ownership feedback at 80c8328df6fa387f29f874e597fc5578d73211bb (base 620228b8ca67a69a108fa4fc5041fdd88880b792).

• The TS golden table now runs in contracts/src/alert-contract.test.ts against production classifyAlertButtonRole and selectRankedAlertButtonByRole. Limrun capture/selection seam tests remain.
• Added every adjacent preference pair in both orders: 56 cases across all 28 boundaries, 120 shared fixture cases total. In an isolated checkout at this head, the contract suite passed 131 tests; swapping nonadjacent Save/Done ranks failed four new adjacency cases (127 passed); restoring production passed 131 again. This directly closes the ordering coverage hole.
• Removed the test-only isAcceptButton and its direct-only test. Retained the handwritten iOS: alert dismiss taps "Allow" on the App Tracking Transparency prompt #3627 safety cases as independent vocabulary assertions, including untranslated/unknown decline refusal, popup-marker scoping, and known-label behavior; retained both iOS end-to-end canaries. With the direct-only test deleted, removing Swift's Open preference fails the retained golden XCTest with three assertions (open and both Open/Allow Once orderings). No important safety canary was deleted.
• Corrected commands.md to name the Apple XCUITest route for iOS/tvOS/visionOS and Limrun iOS. The production macOS AX helper and Android grant-id policy remain unchanged.

Validation at this head: frozen install/build and repository-wide format passed; focused Vitest: 3 files / 179 tests passed; XCTest selection passed; iOS and macOS runner builds passed; macOS golden XCTest: 1 test passed over all 120 cases. The first host invocation used a wrong xctestrun filename and exited 70 with no bundles; the corrected observed generated filename executed successfully. Fresh independent Sol medium review found no findings, static only. Upstream #3824 changed the retained fixture-existence canaries, so I rebased; range-diff shows the original alert commit unchanged, attribution retained, and #3705 remains in ancestry. Old fe2f429 results/receipt remain attributed only to fe2f429.

The requested live candidate-route proof remains unresolved. A bounded T3 capability recheck reports local agent-device 0.21.25 installed/running (candidate package 0.21.29-dev); device_open exposes only the pinned installed launcher and device/session flags, with no candidate-source/runner selection. The explicit no-source/xctestrun/DerivedData/environment override constraint remains in force. Consequently I did not allocate/open a simulator, drive the foreign Android emulator, provision cloud resources, replace installed tooling, or present an installed-version run as candidate evidence. No owned device/session cleanup is pending.

Still required before merge: candidate local iOS daemon → runAppleAlert → chooseAlertButton → buttonIndex evidence for location accept + authorizedWhenInUse/TCC, ATT dismiss + kTCCServiceUserTracking auth_value=0, unknown/tied accept + ALERT_BUTTON_NOT_FOUND/dispatched: no + still-visible alert get; and candidate Android Cancel/Done or Keep Browsing/Open accept with raw JSON/NDJSON, statuses, source attribution, and android-helper backend/helperVersion match. tvOS/visionOS builds and live Limrun remain unchecked. Fixtures and builds do not satisfy this live requirement. Undrafting is explicitly authorized to allow native CI; this is not a merge-ready claim.

Direct published-head gate:

pnpm check:affected --run --head 80c8328df6fa387f29f874e597fc5578d73211bb
exit=0
receipt=run-Qag7Om
status=passed exactCommit=true checkoutChanged=false
initial.status="" final.status=""
12 passed / 2 deduplicated / 10 skipped / 0 not-run
vitest-related: 4662 passed, 0 failed, 0 skipped

Receipt: .tmp/validation/run-Qag7Om/receipt.json in this worktree. Native/device CI, coverage and provider integration remain GitHub obligations. Marking the existing PR ready for review by explicit request; not asserting merge readiness.

[thymikee]

Apple CI diagnosis at 80c8328df6fa387f29f874e597fc5578d73211bb: job 118539508658 failed during public prepare ios-runner, after the iOS build and four selected generic XCTest regressions passed. macOS, Android and Linux smoke tests passed. Later iOS scenarios were skipped.

The error is COMMAND_FAILED for xcrun simctl list devices -j, timeoutMs: 15000, with reported exitCode: 0, empty stderr and truncated inventory stdout. Artifact 12253545187 contains request 6856718ae0b6d2fa: initial target resolution took 13,540 ms, a later cached resolution 0 ms; the session events file records total prepare duration 44,362 ms. These files were independently matched byte-for-byte against the downloaded artifact. They do not contain subprocess spawn/deadline/exit/close ordering, so the precise failing phase and cause remain unproven. A timeout plus exit code 0 is possible in the shared process wrapper; it does not establish an infrastructure fault or exonerate this PR. No CI retry or timeout change has been made.

Separately, the user has now authorized an isolated candidate-build override through the T3 launcher. The requested candidate iOS location/ATT/refusal and Android dialog verification will proceed with owned resources and preserved raw evidence. Until that proof and Apple CI are resolved, this PR remains not merge-ready.

[thymikee]

Live verification requested in #3825 (comment) is now complete at exact published head 80c8328df6fa387f29f874e597fc5578d73211bb, base 620228b8ca67a69a108fa4fc5041fdd88880b792. Rechecked before and after execution; source stayed clean, with no production edits or commits.

All four scenarios passed using one owned iOS 27.0 simulator and one owned Android 15 emulator, with disposable native apps.

Provenance: After frozen pnpm install --frozen-lockfile and pnpm build (both exit 0), built the candidate XCTest runner and Android helpers (exit 0). Every interaction used the exact T3 launcher /Users/michal/.t3/userdata/device/bin/agent-device and its original config/session/platform/device flags. Authorized, supported --daemon-base-url / --daemon-auth-token overrides selected a foreground candidate daemon with isolated state; the iOS external-artifact environment settings selected the candidate runner. Installed CLI 0.21.25 served only as the request gateway, not candidate execution evidence. Candidate daemon metadata reported 0.21.29-dev, codeOrigin: checkout; its recorded and independently recomputed bundle-graph signature both were graph:64:a39bec1a45fa17db89e5fc1a1341aa82ce1c6323. The installed-on-owned-simulator XCTest binary matched the built binary SHA-256: e28f9e16aac88f475204cda9a34d14afbac1ee102a7ca80314af7f1c28c53361.

iOS location: alert get exit 0, raw response excerpt:

{"items":["Allow Once","Allow While Using App","Don’t Allow"]}

alert accept exit 0. Candidate runner log:

AGENT_DEVICE_RUNNER_ALERT_ACTIVATION action=accept label=Allow While Using App

The app's real CLLocationManager delegate callback recorded {"event":"locationCallback","location":4} (authorizedWhenInUse), also visible in the subsequent snapshot. This exercised runAppleAlert → chooseAlertButton → buttonIndex.

iOS ATT: actual prompt, alert get exit 0:

{"items":["Ask App Not to Track","Allow"]}

alert dismiss exit 0; runner activation label was Ask App Not to Track. The app callback recorded {"event":"attCallback:2","att":2} (denied). Read-only, own-bundle-scoped TCC query exited 0:

[{"service":"kTCCServiceUserTracking","client":"com.callstack.pr3825.ad4.proof","auth_value":0,"auth_reason":2}]

ATT was never accepted; no privacy reset was used.

iOS unknown refusal: freshly relaunched fixture; alert get exit 0 exposed {"message":"Unknown Choice","items":["Quasar"]}. alert accept exited 1:

{"success":false,"error":{"code":"ALERT_BUTTON_NOT_FOUND","details":{"runnerErrorCode":"ALERT_BUTTON_NOT_FOUND","dispatched":"no","requestId":"ca6274281086e45a"}}}

Subsequent alert get exit 0 still exposed the same title and Quasar. The real app event log ended at unknownShown, without its unknownActivated callback; runner command completed ok=0, with no alert activation event for that request. No independent tap-counter claim.

Android ordinary dialog: candidate alert get exit 0 exposed {"buttons":["CANCEL","DONE"],"source":"native-dialog"}. alert accept exit 0 returned {"handled":true,"button":"DONE"}; the app's real callback file recorded {"event":"Done"}. Snapshot before and after reported {"backend":"android-helper","helperVersion":"0.21.29-dev"}, matching the candidate package. Candidate daemon provenance above covers execution of the changed Android TypeScript selector.

Raw JSON/exits, request NDJSON, event/runner logs, app callbacks, scoped TCC read, fixture sources, build logs and hashes are preserved locally under /private/tmp/pr3825-live-ad4/ (manifest.json, files.sha256.json, raw/, preserved-sessions/). Snippets above are excerpts, not full responses. Initial overlay-path failure and disposable app scene-lifecycle crash are preserved too; corrected setup produced the results above. No macOS/tvOS/visionOS or Limrun live claim.

Cleanup verified: both original sessions closed; candidate daemon exited 0 via its supported graceful SIGTERM after the launcher rejected daemon stop --clean (launcher requires session, daemon command refuses it). Recorded daemon and runner PIDs are gone, daemon record/lock removed, T3 streams closed, owned simulator and AVD shut down/deleted. Foreign emulator 5554 remains running and untouched; launcher and installed tools unchanged.

This supplies the requested live alert proof. It does not resolve the separate red Apple CI prepare job described in #3825 (comment). No CI retry, timeout change, broad gate, or merge-ready claim.

[thymikee]

The earlier findings from fe2f429 are fixed at 80c8328, and I found no new problems in the alert button choice change. I'm relying on your live proof in comment 6287842818 at this head. I did not re-run the Swift golden XCTest, the Vitest suites or the ordering mutation. tvOS and visionOS builds and live Limrun behavior are not verified, as you noted.

Smoke Tests iOS failed in the ios-runner device inventory step, where xcrun simctl list devices -j timed out after 15000 ms with truncated output. The XCTest step before it passed, and macOS, Android and Linux smoke passed. This diff does not touch simctl inventory, target resolution or runner prepare, so I think the failure is unrelated, but I have not proven the cause. There are no conflicts. Please rerun Smoke Tests iOS. It must go green before merge.

[thymikee]

Verified handoff at unchanged head 80c8328df6fa387f29f874e597fc5578d73211bb: current checks are 17 successful, 4 skipped and 1 neutral, with no pending/failed checks. Apple attempt 2 / job 118603208188 passed both iOS and macOS phases; GitHub records thymikee as the triggering actor. No retry was requested by this agent. The original inventory timeout remains unexplained; this success is not causal clearance.

The maintainer's findings are addressed, all four requested candidate live scenarios are published, the direct exact-head gate passed, no unresolved review threads exist, and ready-for-human is present. The PR is undrafted and mergeable. Handing this PR back for human judgment/merge; no merge or approval performed.

The separately authorized readiness/process diagnostics follow-up continues independently and will not alter this alert PR. tvOS/visionOS builds and live Limrun remain unverified as disclosed.