diff --git a/.changeset/nightwatch-run-all.md b/.changeset/nightwatch-run-all.md new file mode 100644 index 00000000..a80fe4f4 --- /dev/null +++ b/.changeset/nightwatch-run-all.md @@ -0,0 +1,7 @@ +--- +'@wdio/nightwatch-devtools': patch +'@wdio/devtools-backend': patch +'@wdio/devtools-app': patch +--- + +Enable Run all for Nightwatch, which the dashboard disabled although the backend could already run the whole config. Run all now replays the selection the run was launched with (spec paths, `--tag`, `--group`, `--env` and the like) rather than every spec in the config, and a rerun in between does not narrow it. diff --git a/.changeset/selenium-run-all-launch-command.md b/.changeset/selenium-run-all-launch-command.md new file mode 100644 index 00000000..7a098058 --- /dev/null +++ b/.changeset/selenium-run-all-launch-command.md @@ -0,0 +1,5 @@ +--- +'@wdio/selenium-devtools': patch +--- + +Keep Run all on the original command after a targeted rerun. A rerun's process was started with a name filter, and Run all used to be built from that process's own arguments, so it replayed the filter; it now inherits the command the dashboard's first run published. Run all also no longer comes out as `npm run npx` when the tests were started with `npx` or `pnpm exec`, and a spec path or filter containing a space now reaches the relaunched run as one argument. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 4bb9a40a..73d17608 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -29,6 +29,10 @@ - [ ] `script` (page-injected runtime) - [ ] `trace` (Trace mode) +## Capability matrix + +- [ ] `packages/conformance/src/features.ts` updated, and tables regenerated with `pnpm docs:capabilities`, if this adds or changes user-facing behaviour + ## Notes for reviewers [//]: # (Anything non-obvious: design trade-offs, alternatives you ruled out, follow-ups. If this PR touches more than one adapter, say why the shared logic doesn't live in `core`.) diff --git a/CLAUDE.md b/CLAUDE.md index 533471c9..608e5dc0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,6 +29,7 @@ Run from repo root unless noted. | `pnpm demo:wdio` / `pnpm demo:nightwatch` / `pnpm demo:selenium` | Run the per-framework example projects. Useful for manual verification of UI or runtime changes. **No package pins a browser driver**, deliberately: a pinned `chromedriver` rots against whatever Chrome the developer has, and because pnpm puts the package's `node_modules/.bin` on `PATH`, Selenium Manager finds it there, prefers it over resolving, and on a major mismatch only *warns* before returning it anyway. The session then fails to start. With nothing on `PATH` both adapters resolve a matching driver (Nightwatch's Chrome service builder sets `requiresDriverBinary: false` and passes an unset `server_path` through to the same resolver), and nightwatch declares `chromedriver` an optional peer. | | `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and drive the Clock app that ships with every Android system image (keying a duration on the timer keypad, never starting a timer), so a native example needs no `.apk`. `DEVTOOLS_MOBILE_PLATFORM=ios` runs the iOS spec instead — all four adapters — which drives Settings because the simulator ships no Clock, from a separate spec per platform rather than a branch. The simulator is chosen by udid and defaults to whichever is already booted, and an unmatched `IOS_DEVICE_NAME` is refused: naming one that does not exist makes the XCUITest driver create and boot it, every run, rather than fail. That refusal and the booted-simulator preflight are **local** policy — `xcrun simctl` enumerates local simulators and nothing else — so `IOS_UDID` and a non-local `APPIUM_HOST` both bypass them, or a real device and a cloud grid would be refused a run they were correctly configured for. `DEVTOOLS_MOBILE=web` drives the device's own browser on both platforms — Chrome on Android, Safari on iOS, which the XCUITest driver serves without a chromedriver. `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches; `DEVTOOLS_MODE=trace` flips any demo to trace mode. | | `pnpm dev` | Run all packages in parallel dev mode. | +| `pnpm docs:capabilities` | Render the per-adapter capability tables from `packages/conformance/src/features.ts` into `README.md`; `--website ` also renders the docs-site tables, `--check` fails instead of writing, and `--parity` lists every feature some adapter lacks plus the evidence counts. | | `python3 packages/selenium-devtools-py/scripts/changes.py next-version` | The version a Python-adapter release would publish, from the fragments pending in `changes/`. `check --base ` is the CI gate; `apply` is what the release runs. | `selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:mocha:allure` / `:jest` / `:cucumber`. @@ -48,6 +49,7 @@ Defined in root `tsconfig.json`: | `@wdio/devtools-script` / `*` | `packages/script/src/...` | | `@wdio/devtools-service` / `*` | `packages/service/src/...` | | `@wdio/selenium-devtools` / `*` | `packages/selenium-devtools/src/...` | +| `@wdio/nightwatch-devtools/*` | `packages/nightwatch-devtools/src/*` (subpaths only, for type imports such as the option types) | | `@wdio/devtools-shared` / `*` | `packages/shared/src/...` | | `@wdio/devtools-core` / `*` | `packages/core/src/...` | | `@wdio/devtools-trace` / `*` | `packages/trace/src/...` | @@ -259,6 +261,7 @@ Neither is hand-versioned: both assemble the version and the changelog at releas ### Documentation - User-facing docs live in two places that must stay in sync: this repo's `README.md` (+ per-package READMEs) and the **WebdriverIO devtools webpage** (`website/docs/devtools/**` in the `webdriverio/webdriverio` repo — e.g. `wdio/TraceMode.md`). When a change adds, removes, or alters user-facing behavior (a new option, CLI, flag, output, or workflow), update the README here **and** mirror it to the matching webpage doc in the same change. A docs PR that updates only one side isn't complete. +- The capability tables (the README's support, dashboard and mobile tables, the docs site's `CrossFrameworkSupport.md` and `MobileTesting.md`) are generated, never edited by hand. `packages/conformance/src/features.ts` declares every user-facing feature with a cell per adapter, typed as a complete map over the conformance suite's adapter ids, so a new adapter does not compile until each feature declares it. A partial or unsupported cell carries a required reason, which renders as the footnote. A supported cell carries its evidence: conformance rules the adapter has no known gap against (rendered "✅ tested"), a test file in that adapter's own tests or a shared package's (`core`, `trace`, `backend`), or why it is checked by hand. The cell builders and rule groups are in `capability-helpers.ts` beside it, so `features.ts` stays the list alone. Two coverage checks keep a new feature from bypassing the list: every user-facing option (`capability-options.ts`, keyed on the real option types of the shared base and each JS adapter, plus the Python `enable()` keywords, pytest options and user-facing `DEVTOOLS_*` variables read from source by a test) names a feature id, typed so a misspelt id fails to compile, or says why it is not one; and every conformance rule backs a tested cell or is listed in `INTERNAL_RULES` with a reason. A test fails when the README tables are out of date; the docs-site copy is refreshed with `pnpm docs:capabilities --website ` in the same change. --- @@ -269,7 +272,7 @@ Documented divergences from the conventions above. They exist today as debt to b ### Architecture - `replaceCommand` has two semantics — Selenium mutates in place (preserves `_id`/`id` for chained calls); Nightwatch splices and reissues. Both call the same `core/suite-helpers` factories; the storage strategy stays adapter-specific because runner integrations differ. Could be unified by parameterizing the policy if the divergence ever causes a real problem. -- `patchNodeAssert` (via `core/assert-patcher`) is now wired in all three adapters, default-on behind each adapter's `captureAssertions` option (opt out with `captureAssertions: false`). Framework matcher libraries differ: Service taps expect-webdriverio's `beforeAssertion`/`afterAssertion` hooks so passing+failing matchers render as `expect.*` actions (mechanism in the assert-capture entry below); Nightwatch native `assert`/`verify` and Selenium's `node:assert` also surface passing+failing rows via their reconcile/patch paths. The remaining gap is Selenium's jest-style `expect()` (and chai): jest/vitest expose no pass+fail assertion hook, so only failing matchers surface there. +- `patchNodeAssert` (via `core/assert-patcher`) is now wired in all three adapters, default-on behind each adapter's `captureAssertions` option (opt out with `captureAssertions: false`). Framework matcher libraries differ: Service taps expect-webdriverio's `beforeAssertion`/`afterAssertion` hooks so passing+failing matchers render as `expect.*` actions (mechanism in the assert-capture entry below); Nightwatch native `assert`/`verify` and Selenium's `node:assert` also surface passing+failing rows via their reconcile/patch paths. The remaining gap is Selenium's jest-style `expect()` (and chai): jest/vitest expose no assertion hook, so those matchers produce no rows at all, and a failing one only marks its test failed. Nightwatch's own `expect` API is not wrapped either, so it produces no rows. - BiDi is auto-attached in Service and Selenium; Nightwatch is opt-in via `bidi: true` and requires `webSocketUrl: true` in capabilities. - **Network has three sources, and the page's Resource Timing is the floor.** BiDi (all four adapters) and Nightwatch's Chrome performance log are driver-side and carry headers and status. The page collector carries two: its patched `fetch`/`XMLHttpRequest` (the only source with bodies) and the document's Resource Timing buffer (`script/src/collectors/resourceTiming.ts`): every load with timing phases and size, a status on Chromium only, never headers. Resource Timing is the only network source a browser without BiDi has, which is Safari and every Appium session (Appium's BiDi reference lists `session.*` alone). The collector reads the buffer at construction, which is what covers loads made before a `