Repository navigation
Import benchmarks-website under bench/ - #112
Conversation
The develop tree of vortex-data/benchmarks-website at 827dc55, unmodified. Its 39-commit history stays in that repository, which is archived with a pointer here after the cutover. git-subtree-dir: bench git-subtree-split: 827dc55fa492d5861cef30667b045aad4523d6a3 Signed-off-by: Will Manning <will@willmanning.io>
Merge the squashed develop tree of vortex-data/benchmarks-website at 827dc55 into bench/. Signed-off-by: Will Manning <will@willmanning.io>
Next.js looks for a PostCSS config in the app directory and every parent, so inside this repository bench/web picked up the marketing site's Tailwind config at the root and failed with "Cannot find module '@tailwindcss/postcss'". The new config reproduces the plugins Next.js applies when it finds no config. A build with it produces byte-identical CSS to a build with no config at all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bench pinned next 15.5.19, which has critical and high advisories, and dependency-review would block this import on them. Move next and eslint-config-next to 15.5.27, the newest 15.5 release past the 14-day cooldown. It fixes GHSA-p293-qw3h-jr36, GHSA-2xp9-vwfh-vxw4 and the 15.5.21 advisories. Two remain until 15.5.28 clears the cooldown on 2026-10-28 at 17:36 UTC: GHSA-2464-fhr4-w9xw (critical, image optimizer) and GHSA-p3m5-59mp-jv5r (high). bench doesn't use next/image, and Vercel serves /_next/image itself. Override next's exact postcss pin to the patched 8.5 line, and force sharp onto 0.35.5, which next's own range already accepts. Add pnpm's minimumReleaseAge (14 days), matching the Bun and Renovate cooldowns at the root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Add a changed-paths action that reports whether a push or pull request touches the marketing site, bench, or both. ci.yml gates its build and audit jobs on it with `if:` rather than an `on.paths` filter, because "Lint, build, verify" is a required check: a skipped job reports success, while a workflow that never starts leaves the check pending and blocks the merge. Dependency review, the secret scan and REUSE still run on every change. Lighthouse isn't required, so a plain paths-ignore filter is enough there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Rename them with a bench- prefix and adapt them to this repository: - Trigger on main instead of develop. bench-ci gates its job with the changed-paths action, so it can become a required check; bench-deploy filters on bench/**. - Pin every action by SHA and run on ubuntu-24.04. setup-uv comes from astral-sh directly rather than through a wrapper action, and configure-aws-credentials moves to v6.3.0, the pin the vortex monorepo uses for the same AWS account. - Install the Vercel CLI (64.0.2) from bench/vercel-cli's lockfile instead of `npm install -g vercel@latest`, so the job holding VERCEL_TOKEN never resolves a freshly published package. - Read VERCEL_TOKEN from the bench-preview or bench-production environment. The environment replaces the hand-rolled GitHub Deployment calls, and bench-production can be limited to main. The deploy job stays off until BENCH_VERCEL_PROJECT_ID is set. - Post the preview URL with the sticky-comment action under its own header. `gh pr comment --edit-last` would overwrite the Lighthouse comment, which the same bot posts. - Use the root .nvmrc for Node and drop persisted checkout credentials. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bench's source files carry SPDX headers, but its JSON, YAML and .gitignore files can't, and its four logo files had no annotation. Map the config to Apache-2.0, and add the logos and icons to the Vortex brand assets, which are CC-BY-4.0 with the trademark note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
The root tsconfig includes **/*.ts and Biome includes **, so both would check bench's code under the marketing site's rules. With bench/ in scope, `tsc --noEmit` reports 268 errors from bench's files. bench keeps its own tsconfig, ESLint and Prettier until it moves to Biome. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Set the `vortex` project's Ignored Build Step in vercel.json. It skips the build when nothing outside bench/ and the bench workflows changed since the branch's last successful deployment. With nothing to compare against, it builds. bench deploys to its own Vercel project from GitHub Actions, so rebuilding the marketing site for its commits only republishes the same site. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
CLAUDE.md now describes the two-app layout, bench's pnpm equivalents of the supply-chain layers, and its advisories at import. Renovate gives bench's updates their own branches and doesn't automerge them. Platform automerge waits only for required checks, and bench CI isn't one yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
- provision.sh takes a branch for each role. The schema role now trusts vortex-data/vortex.dev on main; the ingest role still trusts vortex-data/vortex on develop. The trust test pins both. AWS changes only when someone applies the rendered policy, as a cutover step. - The build-SHA footer linked to commits in benchmarks-website. Deploys now build from vortex.dev commits, so it links there. - Docs: workflow file names, main as the production branch, where the Vercel variables and token live, and a note on the setup runbook that it describes the standalone repository. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Section 7.b.iv of the Vortex Technical Charter receives and licenses all documentation under CC-BY-4.0, but bench's 13 Markdown files carried Apache-2.0 SPDX headers from the standalone repository. A file's own header wins over REUSE.toml, so correct the headers themselves. Code keeps Apache-2.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Dependency ReviewThe following issues were found:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
🔦 Lighthouse audit
Run |
Dependency review reads pnpm lockfiles without telling dev dependencies from runtime ones, so the high advisories in bench's test and lint tooling block this PR too. Refresh @grpc/grpc-js, brace-expansion, js-yaml and source-map-js within their parents' ranges, all to releases past the 14-day cooldown. vite needs a direct devDependency (^8.3.4). It is vitest's peer, and pnpm kept the vulnerable locked 8.0.16 even with an override in place. undici 7.30.1 is the one dev fix left; it clears the cooldown on 2026-10-29 at 14:18 UTC. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Dependency review flagged 13 packages in the CLI's lockfile, among them a critical in tar. The CLI and its framework builders pin those versions exactly, and patching undici alone would mean a 5 to 6 major inside the CLI's HTTP client. A committed lockfile would leave every bench PR red on advisories that nothing in this repository can fix. Install it with `npm install -g vercel@<exact>`, never `@latest`, and set `--before` to 14 days ago so its whole tree gets the cooldown. A Renovate custom manager bumps VERCEL_CLI_VERSION through its `# renovate:` comment, and the bench rule now also covers the bench workflows, so a CLI bump is not automerged before a deploy has exercised it. CLAUDE.md records the accepted advisories and how to re-audit them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
|
Bench preview for |
The cutover steps moved from the import PR's description to #113. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
…zation' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: Will Manning <will@willmanning.io>
GitHub Actions
Error
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…ean-ef8f04 # Conflicts: # CLAUDE.md
CodeQL flagged js/bad-code-sanitization because the font loader built its inline script by interpolating a value. The Copilot fix escaped that value, but its helper doubles every backslash after JSON.stringify. That breaks the escapes JSON.stringify adds, so a value containing a quote would end the string early. Here the value is a constant URL, so nothing was exploitable. Write the URL into the script instead, so the script is a constant and nothing needs sanitizing. The emitted script is byte-identical to the one before the Copilot fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
main's GitHub Actions update (#115) moved the other workflows to v7.1.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c11d57935
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!acquired) { | ||
| throw new GroupCacheFillInProgressError(); | ||
| } | ||
| return await fill(); |
There was a problem hiding this comment.
When BENCH_DB_POOL_MAX=1, which is a valid value for the documented connection-limit setting, this function holds the pool's only client for the advisory lock while fill() calls collectGroups through getPool().query(). Those queries wait forever for a second client, while the lock client is released only after they finish, so cold landing-page and group-cache requests hang; either enforce a minimum pool size of two or execute the fill through the locked client.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pre-existing in the imported bench code, not changed by the move. BENCH_DB_POOL_MAX isn't set on the Vercel project, so the pool uses the default of 8 and this is latent. Tracked as a follow-up in #113.
🤖 Addressed by Claude Code
| existing_rule=$(aws ec2 describe-security-groups --group-ids "$DB_SG_ID" \ | ||
| --query 'SecurityGroups[0].IpPermissions[?FromPort==`5432`]' --output json) |
There was a problem hiding this comment.
When the named security group already has any permission beginning at port 5432—for example TCP 5432 restricted to an office CIDR—this query returns a nonempty array and skips the required public rule, leaving the public RDS endpoint unreachable from GitHub or Vercel. AWS represents the protocol, both range endpoints, and CIDR separately, and its documented example checks from-port, to-port, and cidr together (AWS CLI reference); verify TCP, both ports equal to 5432, and an IpRanges[].CidrIp of 0.0.0.0/0 before treating the rule as present.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pre-existing in the one-time bootstrap script, not changed by the move; the live security group already has the public rule. Tracked as a follow-up in #113.
🤖 Addressed by Claude Code
Both bench workflows read the Node version from .nvmrc. The changed-paths action counted a change to it as site-only, so a Node bump could merge without running bench's build and tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
summary.ts still said the compression-size query used a JSON field lookup that tolerates databases from before migration 009. bench #29 replaced that lookup with a plain column read because it timed out in production, so the query now requires migration 009. The web README said the deploy workflow probes CDN caching after each deploy. No workflow has ever done that, so the README now describes the manual check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
next 15.5.28 fixes GHSA-2464-fhr4-w9xw (critical) and GHSA-p3m5-59mp-jv5r, the last advisories that dependency review flags. undici 7.30.1 fixes the remaining dev-only advisories. Both are inside the 14-day cooldown, so pnpm-workspace.yaml exempts exactly these versions, as approved on 2026-10-28. Remove the exemptions after 2026-10-29 14:18 UTC, when both are past the window. pnpm audit for bench/web now reports no high or critical advisories. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b3ba969f98
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| paths: | ||
| - 'bench/**' | ||
| - '.github/workflows/bench-deploy.yml' |
There was a problem hiding this comment.
When a PR or push changes only .nvmrc, these path filters prevent the bench deployment workflow from starting, even though its Setup Node step reads that file and the bench Vercel project has Git integration disabled. The bench CI now validates such a change, but no preview is produced and production remains built with the previous Node version until an unrelated bench/** change triggers another deployment; include .nvmrc in both trigger path lists.
Useful? React with 👍 / 👎.
| aws rds create-db-proxy \ | ||
| --db-proxy-name "$DB_PROXY_NAME" \ | ||
| --engine-family POSTGRESQL \ | ||
| --auth "AuthScheme=SECRETS,SecretArn=${DB_MASTER_SECRET_ARN},IAMAuth=REQUIRED" \ |
There was a problem hiding this comment.
When VPC connectivity is added and the operator points BENCH_DB_HOST at this advertised Vercel-reader proxy, the shipped bench_read configuration cannot authenticate: the web app supports a static bench_read password, while this proxy both requires IAM client authentication and associates only the master-user secret. The AWS CLI reference defines IAMAuth=REQUIRED as requiring IAM authentication for connections to the proxy, but the README explicitly says the Vercel runtime has no AWS credentials and bench_read has no rds_iam grant. Configure password client authentication with a bench_read secret, or stop directing the reader to this proxy.
Useful? React with 👍 / 👎.
| if uv run --no-project scripts/migrate-schema.py status; then | ||
| echo "dry_run: no drift — migration set matches the ledger." | ||
| else | ||
| echo "dry_run: drift detected (pending or orphaned migrations) — re-run without dry_run to apply." | ||
| fi |
There was a problem hiding this comment.
When uv run fails for an operational reason—such as a TLS/database connection error, missing ledger permission, or dependency resolution failure—the if treats the same nonzero status as ordinary schema drift, prints the drift message, and then lets the step exit successfully. A manually requested dry run can therefore report green even though it never inspected the database; give drift a distinct exit code and only suppress that code while propagating command failures.
Useful? React with 👍 / 👎.
| // the filter dropdown rather than failing an otherwise renderable chart. | ||
| const [chart, universe] = await Promise.all([ | ||
| getChart(slug, singleSearchParam(sp.n)), | ||
| collectFilterUniverse().catch(() => undefined), |
There was a problem hiding this comment.
When a chart page misses the HTML CDN cache—for example for a new chart slug or query-string variant—this direct collector bypasses the existing cachedFilterUniverse() wrapper and issues both universe-discovery queries against Postgres on every render. The universe is already tagged for ingest-driven invalidation and does not depend on the chart window, so using the cached wrapper here avoids unnecessary database work while preserving the existing optional-chrome .catch() behavior.
Useful? React with 👍 / 👎.
Move
vortex-data/benchmarks-website(bench.vortex.dev) into this repository, underbench/. This is stage 1 of #113. The issue has the plan, the cutover steps and the reasons for the decisions.The first commit imports bench's
developbranch at 827dc55 without changes. The other commits move bench's workflows to.github/workflows/bench-*.yml, and run the CI for each app only when its files change. They also fix two build problems that the import caused, patch dependency advisories, and update licenses and paths.The PR also moves bench to
next15.5.28 andundici7.30.1, which removes all of its high and critical advisories. Both versions are exempt from the 14-day cooldown until it passes; #113 has the removal step.🤖 Generated with Claude Code