Skip to content

ci: validate every PR and check declared plugin component paths exist - #582

Open
akushonkamen wants to merge 2 commits into
cursor:mainfrom
akushonkamen:fix/531-validator-component-paths
Open

akushonkamen wants to merge 2 commits into
cursor:mainfrom
akushonkamen:fix/531-validator-component-paths

Conversation

@akushonkamen

@akushonkamen akushonkamen commented Oct 12, 2026 •

Copy link
Copy Markdown

Fixes #531.

Why

At 1b6d44f (this PR's base), the CI setup checks the two manifests and nothing else:

  • scripts/validate-plugins.mjs calls existsSync only three times (lines 37, 57, 65), and all three are on the manifests themselves: marketplace.json, the plugin's source directory, and its plugin.json. A plugin can declare skills / agents / rules / commands / hooks / mcpServers / logo paths that do not exist and still pass schema validation and CI — the paths fail silently at load time in Cursor.
  • .github/workflows/validate-plugins.yml gates the job behind a paths: filter listing three globs (marketplace.json, **/plugin.json, schemas/**), so a PR that only touches SKILL.md, hooks.json, mcp.json, agents or scripts runs no CI at all.

Issue #531 files the pieces separately ("filing as one issue so the pieces can be picked off separately"). This PR takes the first two pieces of the suggested shape: drop the paths: filter (per the issue author: "broaden to the whole tree (or drop the filter); the job takes seconds"), and add path-existence checking for declared component fields. Pieces 3–6 of the suggested shape (frontmatter parsing, ${VAR} placeholders vs variables, unlisted plugin directories, a lockfile with npm ci) are left for follow-up PRs.

Change

Piece 1 — .github/workflows/validate-plugins.yml: remove the paths: filter under pull_request:, so every pull request runs the validation job. The job takes seconds. No other change to the workflow (the npm install --no-save ajv ajv-formats line is untouched; the lockfile piece is issue-suggested shape #2, out of scope here).

Piece 2 — scripts/validate-plugins.mjs: add a checkComponentPath() helper and apply it to every declared component path:

  • skills, agents, rules, commands (string or array entries)
  • hooks and mcpServers when declared as path strings (string or array entries)
  • logo when declared as a string

Check rules, matching the schema's allowed shapes:

  • A plain path relative to the plugin root must exist.
  • A path with a trailing slash must be a directory (statSync).
  • A glob pattern (contains *, ?, {, }, [, ]) must have an existing static base directory (the path prefix up to the first glob-containing segment), so a typo'd base like docs/**/*.md with no docs/ is caught.
  • Absolute URLs (allowed for logo) and inline hooks / mcpServers objects are skipped.

No new dependencies — the check uses fs/path only.

Verification

  • Full marketplace, before and after: timeout 60 node scripts/validate-plugins.mjs → All plugins validated successfully., exit 0 at the base commit 1b6d44f and at this PR's HEAD. All 100 plugins currently in the marketplace pass; zero failures, so the new check introduces no false positives.
  • Broken fixtures on a separate copy (fixture marketplace + a plugin declaring missing paths), 6/6 red:
    • missing skills/ directory, missing logo, missing hooks file, missing mcp.json → each fails with Plugin "<name>": <field> path "<path>" does not exist, exit 1
    • glob docs/**/*.md with no docs/ base → skills glob "docs/**/*.md" matches nothing — base directory "docs" does not exist, exit 1
    • "README.md/" (trailing slash on a file) → skills path "README.md/" is not a directory, exit 1
  • Skip cases, 2/2 green: logo as an https:// URL and inline hooks / mcpServers objects → All plugins validated successfully., exit 0.
  • Pre-existing checks unchanged, still red: a plugin whose source directory is missing → source directory "..." does not exist, exit 1; a plugin.json failing schema (version as number) plus a marketplace/plugin name mismatch → both errors reported, exit 1.
  • Workflow syntax: actionlint .github/workflows/validate-plugins.yml → exit 0.

AI disclosure

This fix was developed with an AI coding agent (Claude) and human-reviewed; the claim comment on #531 discloses the same.


Note

Low Risk
CI-only validation and workflow trigger changes; no production runtime or data-path behavior.

Overview
Every pull request now runs the plugin validation workflow by removing the paths: filter on pull_request in validate-plugins.yml, so changes to skills, hooks, or other plugin assets are no longer skipped when manifests are untouched.

scripts/validate-plugins.mjs adds filesystem checks after JSON schema validation: declared paths for skills, agents, rules, commands, string hooks / mcpServers, and logo must exist under the plugin root. Globs only require the static prefix directory to exist; trailing-slash paths must be directories; https:// (and similar) logos are ignored.

Fixes the gap where broken component paths could pass CI and only fail at Cursor load time (#531).

Reviewed by Cursor Bugbot for commit 5ba81f0. Bugbot is set up for automated code reviews on this repo. Configure here.

The paths filter only listed the two manifest files and schemas, so a
PR that only touches SKILL.md, hooks.json, mcp.json, agents or scripts
ran no CI at all. The job takes seconds; drop the filter so every PR is
validated.

Piece 1 of cursor#531.

Signed-off-by: akushonkamen <akushonkamen@163.com>
The validator only called existsSync on the manifests themselves
(marketplace.json, plugin dir, plugin.json), so a plugin could declare
skills/agents/rules/commands/hooks/mcpServers/logo paths that do not
exist and pass CI. Check each declared path: plain paths must exist
(directory when declared with a trailing slash), glob patterns must
have an existing static base directory, and absolute URLs (allowed for
logo) plus inline hooks/mcpServers objects are skipped.

Validated green against all 100 plugins currently in the marketplace.

Piece 2 of cursor#531.

Signed-off-by: akushonkamen <akushonkamen@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI and validator cover manifests only: skills, hooks, mcp.json, agents and the script test suites are unchecked

1 participant