diff --git a/AGENTS.md b/AGENTS.md index 6248a71..7278b1d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -94,6 +94,11 @@ prints advisory observations (which never fail the build) when a project's decla automatic: the prerelease channel is withheld once a stable release (GitHub release or NuGet version) catches up with the prerelease line, and returns with the next prerelease — never delete the declaration to stop publishing it. + - A page's slug is its path from the docs root (ADR 0011): a topic folder stays a path segment + (`schemas/Custom-Rules.md` is published at `/docs//schemas/custom-rules/`), a flat corpus + keeps the single-segment slug it always had, and every relative link resolves against the page + that contains it. A project whose docs are reorganised records the old→new slug in + `docs.redirects`, which the build turns into redirects; a redirect never shadows a real page. - Agent Skills a project publishes are declared in `skills:` (ADR 0010) at `skills//SKILL.md`; the sync mirrors the whole directory to `/skills///` and each project page renders the same install block diff --git a/README.md b/README.md index 33276e2..19898d4 100644 --- a/README.md +++ b/README.md @@ -270,8 +270,15 @@ docs (in-repo `docs/`, GitHub wikis via a shallow clone, or the README), then: the conventional `index.md`/`Home.md`/`readme.md`, otherwise the repository README when `readmeAsIndex: true`, - converts GitHub alert blockquotes to Starlight asides, -- rewrites relative links and images so they resolve inside the portal, -- honours `_Sidebar.md` ordering when the source provides it. +- rewrites relative links and images so they resolve inside the portal, resolving + each link against the page that contains it (which is what GitHub does), +- honours `_Sidebar.md` ordering when the source provides it, +- derives each page's slug from its path relative to the docs root (ADR 0011), so + a topic folder stays a path segment — `schemas/Custom-Rules.md` is published at + `/docs//schemas/custom-rules/` — while a flat corpus keeps the + single-segment slug it always had. Each segment is reduced to lower-case + letters, digits and dashes, so any folder name publishes a URL that needs no + percent-encoding (`API v2 (beta)/` → `api-v2-beta/`). ### Documentation landing pages @@ -299,6 +306,25 @@ Each documentation page shows its owning project, lifecycle status, staleness (last review older than 365 days, configurable in `src/lib/docs/staleness.ts`), and an edit link to the source repository. +### Moving a documentation page + +A page's folder is part of its URL, so reorganising a docs tree changes published +URLs. Declare the move once in the project catalogue and the build writes a +meta-refresh redirect from the old slug to the new one: + +```yaml +docs: + redirects: + custom-rules: schemas/custom-rules +``` + +Redirects never overwrite a real page, and a pair is skipped while the channel has +not published the new address — so while a `live` channel is pinned to a tag from +before the move, nothing happens; at promotion each redirect appears (ADR 0011). +`docsPage` deep links are the exception: the catalogue keeps them a single path +segment, so a page the catalogue deep-links either stays at the docs root or moves +together with its deep link. + ### Live and prerelease documentation A project whose default branch is ahead of its latest release can publish two diff --git a/docs/decisions/0011-documentation-folders.md b/docs/decisions/0011-documentation-folders.md new file mode 100644 index 0000000..7ff3ad6 --- /dev/null +++ b/docs/decisions/0011-documentation-folders.md @@ -0,0 +1,113 @@ +# 0011 — Documentation folders keep their place in the URL + +- **Status:** Accepted +- **Date:** 2026-03-11 +- **Supersedes:** none + +## Context + +Documentation used to be one flat directory per project, so a page's slug was its +file name. That works for a dozen pages and fails at thirty: a project cannot +group its pages by topic without the grouping existing only in the sidebar, and a +reader cannot tell from an address that `custom-rules` is part of validation +rather than a top-level concern. + +Letting a project nest its pages exposed two assumptions that were invisible while +every corpus was flat: + +1. **Slugs ignored the folder.** `slugifyDocFile` folded the whole path into one + dash-joined slug, so `schemas/Custom-Rules.md` would publish at + `/docs/zodsharp/schemas-custom-rules/` — the URL says the tree has a folder and + then hides it. +2. **Links resolved from the docs root.** `rewriteDocMarkdown` matched a link by + its *file name* against a set of root-relative slugs. A page in `schemas/` + linking `../concepts/Core-Concepts.md` matched nothing, so the link degraded + into a GitHub blob URL computed from the root — pointing at a path that does not + exist. + +## Decision + +### 1. A page's folder is a path segment of its slug + +`docSlug(path)` reduces each segment to lower-case ASCII letters, digits and inner +dashes and joins them with `/`, so `schemas/Custom-Rules.md` publishes at +`/docs//schemas/custom-rules/` and the mirror writes +`docs//schemas/custom-rules.md`. A folder is therefore supported without +restriction — `Topic Folder`, `Q&A` and `v2.guides` become `topic-folder`, `q-a` +and `v2-guides` — and only the file's own `.md`/`.mdx` extension is dropped. +`isUrlSafeDocSlug` is the contract every published slug must satisfy: non-empty +segments, no leading or trailing dash, nothing that would need percent-encoding. A +folder or file name that reduces to nothing (`___/Page.md`) fails the sync with a +`DocsValidationError` naming the source path, rather than publishing a page at an +address nobody can write down. + +A flat corpus is the single-segment case of the same rule, which is why **no +existing URL moves**: `Getting-Started.md` is still `getting-started`, and a +project that never nests needs no changes. + +### 2. Links resolve against the page that contains them + +`DocLinkContext.pagePath` carries the page's doc-root-relative path, and a link is +resolved from that page's directory — the way GitHub and a browser resolve it. +A bare file name still resolves by name, so a flat corpus and a link written before +the folders keep working. Repository-path fallbacks resolve from the page's +*directory in the repository*, so `../../src/Foo` from a nested page still lands on +the right GitHub blob. + +`` placeholders now accept `/`, and `resolveDocRelativeLink` counts the +hops between two pages with `path.posix.relative` instead of assuming one `..`, so +`schemas/custom-rules` reaches `serialization/jsonschema-export` at `../../`. + +### 3. A move is declared, and a redirect needs both ends + +A page's folder is part of its URL, so reorganising a tree changes published URLs. +`docs.redirects` maps a former slug to its current one and the build writes a +meta-refresh redirect for each pair — but only when the pair is real in the build +at hand. `writeStaticRedirect` refuses to overwrite an existing file, so a redirect +never shadows a page that owns its URL; and `writeDocsPageRedirects` skips a pair +whose destination the channel has not published yet. + +That second check is what makes a declaration timing-safe across a promotion. A +project that nests its pages changes every slug at once, while the `live` channel +keeps reading the last release tag until the next release is promoted: the old +slug is still a real page, so nothing is written. For a page that only exists in +the newer tree, the new address is unpublished too, so forwarding the reader would +be forwarding into a 404 — the post-build link crawl fails the build on exactly +that, which is how this was found. Both halves activate per page, at promotion. + +### 4. `docsPage` stays a single segment + +The catalogue's deep-link slug keeps its `^[a-z0-9-]+$` shape and resolves against +the default channel. A deep link must therefore keep working across two trees that +can differ (a tag and a branch), so a page the catalogue deep-links either stays at +the docs root or moves together with its deep link. Widening the schema would only +move the failure to the moment a channel's tree diverges. + +## Consequences + +- The published URL mirrors the source tree, so a page's address says where it + lives; moving a page between folders is a URL change and needs a `docs.redirects` + entry. +- Sibling folders are reached with `../../`; the previous assumption of a single + `..` is gone, and the recursion through `starlight-links-validator` is what + catches a mistake in the built site. +- Documentation written for a flat corpus is unaffected in both slug and link + terms, so this landed without touching another project's catalogue. +- ZodSharp's own move is the first user of `docs.redirects`: 30 entries that stay + inert until the release that carries the folders is promoted to `live`. + +## Alternatives considered + +- **Fold the folder into the slug** (`schemas-custom-rules`). One-line change, but + the URL then advertises a hierarchy it does not express, and `/` is what a + reader expects to see. Rejected. +- **Keep resolving links from the docs root.** Requires every cross-folder link to + be written root-relative, which breaks on GitHub and in any editor that resolves + relative paths. Rejected in favour of resolving from the page. +- **Redirect every nested page from its flattened slug automatically.** Would have + avoided the `docs.redirects` map, but it also invents URLs for pages that never + had them (`/docs//setup/` for a page that was always `guides/setup`) and + cannot express a rename. Rejected in favour of an explicit map. +- **Allow `/` in `docsPage`.** Rejected: see decision 4 — a deep link that exists + in one channel's tree and not the other fails the build either way, and the + schema is the cheapest place to make that impossible. diff --git a/src/config/docs-redirects-integration.ts b/src/config/docs-redirects-integration.ts index c85fa7b..d6f2a71 100644 --- a/src/config/docs-redirects-integration.ts +++ b/src/config/docs-redirects-integration.ts @@ -3,18 +3,23 @@ import type { AstroIntegration } from 'astro'; import { fileURLToPath } from 'node:url'; import { readDocsManifest } from '../src/lib/docs/aggregate'; -import { writeRetiredChannelRedirects, writeRetiredPageRedirects } from '../src/lib/docs/redirects'; +import { + writeDocsPageRedirects, + writeRetiredChannelRedirects, + writeRetiredPageRedirects, +} from '../src/lib/docs/redirects'; import { loadProjects } from '../src/lib/manifest/load'; /** - * Redirects for retired documentation channels and moved pages (ADR 0009, - * ADR 0010). + * Redirects for retired documentation channels, moved documentation pages and the + * site's own moved pages (ADR 0009, ADR 0010). * * A project's `prerelease` channel is withheld automatically once a stable - * release catches up with it, and `/skills/` moved to the project pages, so - * neither set of URLs may 404. purview.dev is served from GitHub Pages, which - * only serves files, so each retired URL is written as a meta-refresh page - * pointing at its new home. + * release catches up with it, a project may declare the slugs a reorganisation + * moved (`docs.redirects`), and `/skills/` moved to the project pages, so none of + * those URLs may 404. purview.dev is served from GitHub Pages, which only serves + * files, so each retired URL is written as a meta-refresh page pointing at its new + * home. * * Runs at `astro:build:done` beside the discovery integration. It deliberately * writes outside Astro's route list, so the redirects never enter the sitemap, @@ -28,11 +33,13 @@ export default function docsRedirectsIntegration(): AstroIntegration { hooks: { 'astro:build:done': ({ dir, logger }) => { const dist = fileURLToPath(dir); - const retired = writeRetiredChannelRedirects(dist, loadProjects(), readDocsManifest()); - const moved = writeRetiredPageRedirects(dist); - if (retired.length + moved.length > 0) { + const projects = loadProjects(); + const retired = writeRetiredChannelRedirects(dist, projects, readDocsManifest()); + const moved = writeDocsPageRedirects(dist, projects); + const relocated = writeRetiredPageRedirects(dist); + if (retired.length + moved.length + relocated.length > 0) { logger.info( - `Redirects: ${retired.length} retired documentation URL(s) and ${moved.length} moved page(s) now point at their new home.`, + `Redirects: ${retired.length} retired documentation URL(s), ${moved.length} moved documentation page(s) and ${relocated.length} moved site page(s) now point at their new home.`, ); } }, diff --git a/src/src/data/projects/zodsharp.yml b/src/src/data/projects/zodsharp.yml index daf0872..8bea64f 100644 --- a/src/src/data/projects/zodsharp.yml +++ b/src/src/data/projects/zodsharp.yml @@ -43,6 +43,41 @@ docs: ref: latest-stable prerelease: ref: default + # The documentation was reorganised into topic folders, which makes each page's folder part of its published URL. + # Every page that moved keeps its old flat URL working: the site writes a meta-refresh redirect for each pair once + # the default channel publishes that page's new address (and only while the old slug is no longer a page), so these + # entries stay dormant while `live` still reads a tag that predates the move and activate per page at promotion. + redirects: + core-concepts: concepts/core-concepts + composition-and-transforms: concepts/composition-and-transforms + string-validation: schemas/string-validation + number-validation: schemas/number-validation + object-validation: schemas/object-validation + arrays-and-other-schemas: schemas/arrays-and-other-schemas + unions-and-discriminated-unions: schemas/unions-and-discriminated-unions + validation-rules-reference: schemas/validation-rules-reference + custom-rules: schemas/custom-rules + value-objects-integration: schemas/value-objects-integration + compiled-validators-and-caching: schemas/compiled-validators-and-caching + source-generator: generator/source-generator + source-generator-dataannotations: generator/source-generator-dataannotations + source-generator-diagnostics: generator/source-generator-diagnostics + jsonschema-export: serialization/jsonschema-export + jsonschema-import: serialization/jsonschema-import + systemtextjson-integration: serialization/systemtextjson-integration + newtonsoftjson-integration: serialization/newtonsoftjson-integration + microsoft-extensions-validation: hosting/microsoft-extensions-validation + openapi-integration: hosting/openapi-integration + blazor-integration: hosting/blazor-integration + http-integration: hosting/http-integration + dependency-injection: hosting/dependency-injection + localization: hosting/localization + guarantees-and-limitations: quality/guarantees-and-limitations + performance: quality/performance + trimming-and-aot: quality/trimming-and-aot + cross-platform-interop: quality/cross-platform-interop + contributing: development/contributing + release-flow: development/release-flow targetFrameworks: - net8.0 - net9.0 diff --git a/src/src/lib/docs/aggregate.ts b/src/src/lib/docs/aggregate.ts index 36722e2..176bdaf 100644 --- a/src/src/lib/docs/aggregate.ts +++ b/src/src/lib/docs/aggregate.ts @@ -20,7 +20,13 @@ import { normalizeDocumentHeadings, renderFrontmatter, } from './frontmatter'; -import { extractHeadingSlugs, rewriteDocMarkdown, resolveDoclinks, slugifyDocFile } from './links'; +import { + docSlug, + extractHeadingSlugs, + isUrlSafeDocSlug, + rewriteDocMarkdown, + resolveDoclinks, +} from './links'; import { stripMkdocsMarkup } from './mkdocs'; export const DOCS_OUTPUT_DIR = resolve('src/content/docs'); @@ -361,7 +367,7 @@ export function selectDocsRootFile( ); } - const rootSourceSlug = slugifyDocFile(relativeToDocRoot(index.path, docRoot)); + const rootSourceSlug = docSlug(relativeToDocRoot(index.path, docRoot)); return { index, regular: rawFiles.filter((raw) => raw !== index), @@ -379,11 +385,27 @@ async function buildPage( sourceDir: string, slug: string, raw: RawSourceFile, + pagePath: string, order: number, knownSlugs: Set, headingSlugs: Map>, slugAliases?: Map, ): Promise { + // A slug becomes the published URL, so it must survive as an address verbatim. A folder or file name that reduces + // to nothing (`___/Page.md`) leaves an empty path segment, which would otherwise publish a page at a URL nobody can + // write down. + if (!isUrlSafeDocSlug(slug)) { + throw new DocsValidationError( + [ + `Invalid documentation path in ${project.id}`, + ` "${raw.path}" produces the slug "${slug}", which cannot be published without encoding.`, + '', + 'Remediation: name the file, and every folder above it, with letters, digits and dashes only, so each path', + 'segment survives slugification and the published address mirrors the source tree.', + ].join('\n'), + ); + } + const branch = channel.ref; const imageBase: string | null = project.docs?.source === 'wiki' @@ -398,6 +420,7 @@ async function buildPage( branch, knownSlugs, sourcePath: sourceDir, + pagePath, imageBase, headingSlugs, slugAliases, @@ -501,7 +524,8 @@ async function collectGithubPathChannel( ? parseWikiSidebar((await fetchRawFile(project.repository, ref, sidebarPath))?.content) : []; const orderBySidebar = new Map(); - sidebarOrder.forEach((name, index) => orderBySidebar.set(slugifyDocFile(name), index)); + // A sidebar entry is a path from the docs root, so it slugifies to the page slug - folders included. + sidebarOrder.forEach((name, index) => orderBySidebar.set(docSlug(name), index)); let selectedRoot: { index: RawSourceFile; @@ -542,13 +566,13 @@ async function collectGithubPathChannel( const knownSlugs = new Set(['index']); for (const raw of selectedRoot.regular) { - knownSlugs.add(slugifyDocFile(relativeToDocRoot(raw.path, docRoot))); + knownSlugs.add(docSlug(relativeToDocRoot(raw.path, docRoot))); } const headingSlugs = new Map>(); for (const raw of [...selectedRoot.regular, selectedRoot.index]) { const slug = - raw === selectedRoot.index ? 'index' : slugifyDocFile(relativeToDocRoot(raw.path, docRoot)); + raw === selectedRoot.index ? 'index' : docSlug(relativeToDocRoot(raw.path, docRoot)); headingSlugs.set(slug, extractHeadingSlugs(stripMkdocsMarkup(raw.content))); } @@ -560,6 +584,7 @@ async function collectGithubPathChannel( layout.path, 'index', selectedRoot.index, + relativeToDocRoot(selectedRoot.index.path, docRoot), -1, knownSlugs, headingSlugs, @@ -568,7 +593,7 @@ async function collectGithubPathChannel( ); for (const raw of selectedRoot.regular) { const relative = relativeToDocRoot(raw.path, docRoot); - const slug = slugifyDocFile(relative); + const slug = docSlug(relative); const explicitOrder = layout.order?.indexOf(slug); const order = explicitOrder !== undefined && explicitOrder >= 0 @@ -581,6 +606,7 @@ async function collectGithubPathChannel( layout.path, slug, raw, + relative, order, knownSlugs, headingSlugs, @@ -613,7 +639,8 @@ async function collectWikiChannel( } const orderBySidebar = new Map(); - sidebarOrder.forEach((name, index) => orderBySidebar.set(slugifyDocFile(name), index)); + // A sidebar entry is a path from the docs root, so it slugifies to the page slug - folders included. + sidebarOrder.forEach((name, index) => orderBySidebar.set(docSlug(name), index)); const rawFiles: RawSourceFile[] = candidates.map((candidate) => ({ path: candidate.name, @@ -624,12 +651,12 @@ async function collectWikiChannel( const knownSlugs = new Set(['index']); for (const raw of selectedRoot.regular) { - knownSlugs.add(slugifyDocFile(raw.path)); + knownSlugs.add(docSlug(raw.path)); } const headingSlugs = new Map>(); for (const raw of [...selectedRoot.regular, selectedRoot.index]) { - const slug = raw === selectedRoot.index ? 'index' : slugifyDocFile(raw.path); + const slug = raw === selectedRoot.index ? 'index' : docSlug(raw.path); headingSlugs.set(slug, extractHeadingSlugs(stripMkdocsMarkup(raw.content))); } @@ -641,6 +668,7 @@ async function collectWikiChannel( 'docs/wiki', 'index', selectedRoot.index, + selectedRoot.index.path, -1, knownSlugs, headingSlugs, @@ -648,7 +676,7 @@ async function collectWikiChannel( ), ); for (const raw of selectedRoot.regular) { - const slug = slugifyDocFile(raw.path); + const slug = docSlug(raw.path); const order = orderBySidebar.get(slug) ?? 1000; pages.push( await buildPage( @@ -657,6 +685,7 @@ async function collectWikiChannel( 'docs/wiki', slug, raw, + raw.path, order, knownSlugs, headingSlugs, @@ -684,6 +713,7 @@ async function collectReadmeChannel( '', 'index', readme, + '', -1, new Set(['index']), new Map([['index', extractHeadingSlugs(stripMkdocsMarkup(readme.content))]]), diff --git a/src/src/lib/docs/links.ts b/src/src/lib/docs/links.ts index 2abf763..84e806e 100644 --- a/src/src/lib/docs/links.ts +++ b/src/src/lib/docs/links.ts @@ -1,3 +1,5 @@ +import { posix } from 'node:path'; + /** * Deterministic transforms applied to aggregated documentation so the source * markdown renders correctly inside Starlight and never leaks build paths. @@ -6,14 +8,62 @@ const DOC_EXTENSIONS = new Set(['md', 'mdx']); const IMAGE_EXTENSIONS = new Set(['png', 'jpg', 'jpeg', 'gif', 'svg', 'webp', 'avif']); +/** Reduce one path segment to lower-case letters, digits and dashes, with no leading or trailing dash. */ +function reduceSegment(segment: string): string { + return segment + .replace(/[^a-zA-Z0-9-]+/g, '-') + .replace(/^-+|-+$/g, '') + .toLowerCase(); +} + /** Slugify a wiki/doc file name into a Starlight route slug. */ export function slugifyDocFile(fileName: string): string { const parts = fileName.split('.'); const base = parts.length > 1 ? parts.slice(0, -1).join('.') : fileName; - return base - .replace(/[^a-zA-Z0-9-]+/g, '-') - .replace(/^-+|-+$/g, '') - .toLowerCase(); + return reduceSegment(base); +} + +/** + * Slug for a page from its path relative to the docs root. + * + * A flat corpus (`Getting-Started.md`) keeps the single-segment slug it always + * had, so no published URL moves. A page in a topic folder keeps the folder as a + * path segment (`schemas/Custom-Rules.md` → `schemas/custom-rules`), so the + * published URL mirrors the source tree and the reader can tell where a page + * lives from its address. Each segment is slugified independently, which is why + * the folder survives instead of being folded into the file name. + * + * Folders are therefore supported without restriction, including names that would + * need encoding in an address (`Topic Folder`, `Q&A`, `v2.guides`): every segment + * is reduced to lower-case ASCII letters, digits and inner dashes, and only the + * file's own `.md`/`.mdx` extension is dropped. `isUrlSafeDocSlug` is the contract + * a slug has to satisfy before it is published. + */ +export function docSlug(relativePath: string): string { + const segments = relativePath + .split(/[/\\]/) + .filter((segment) => segment !== '' && segment !== '.'); + return segments + .map((segment, index) => + reduceSegment(index === segments.length - 1 ? segment.replace(/\.(md|mdx)$/i, '') : segment), + ) + .join('/'); +} + +/** A slug segment as this sync publishes it: no character needs percent-encoding. */ +const URL_SAFE_SEGMENT = /^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$/; + +/** + * Whether a page slug can be used in a published URL verbatim. + * + * Every `/`-separated segment must be non-empty and made of lower-case ASCII + * letters, digits and inner dashes only, so a folder can be nested as deeply as a + * project likes without any address needing percent-encoding. A path whose folder + * or file name reduces to nothing (`___/Page.md`) produces an empty segment, which + * is the case this rejects. + */ +export function isUrlSafeDocSlug(slug: string): boolean { + return slug !== '' && slug.split('/').every((segment) => URL_SAFE_SEGMENT.test(segment)); } export interface DocLinkContext { @@ -27,6 +77,12 @@ export interface DocLinkContext { knownSlugs: Set; /** Source path of the doc directory within the repository (e.g. "docs/wiki"). */ sourcePath: string; + /** + * Source path of the page being rewritten, relative to the docs root + * (`schemas/Custom-Rules.md`), so relative links resolve against the page they + * appear on rather than against the docs root. + */ + pagePath?: string; /** Absolute base URL for relative image references, or null to leave them relative. */ imageBase: string | null; /** Lowercased heading slugs per page slug, used to validate link fragments. */ @@ -46,13 +102,26 @@ function isExternal(target: string): boolean { return /^[a-z][a-z0-9+.-]*:/i.test(target) || target.startsWith('//') || target.startsWith('/'); } -/** Compute a base-relative href from one doc slug to another. */ +/** + * Compute a base-relative href from one doc slug to another. + * + * A slug is also the page's URL directory (`schemas/custom-rules` is served at + * `/docs//schemas/custom-rules/`), so the number of `..` hops depends on + * how deep the page is: a page in a folder reaches a sibling folder with `../../`, + * while a flat corpus keeps the single `../` it always had. An `index` page is + * its own directory, which is why it is trimmed before the comparison. + */ export function resolveDocRelativeLink(fromSlug: string, toSlug: string): string { - const target = toSlug === 'index' || toSlug === '' ? '' : `${toSlug}/`; - if (fromSlug === 'index' || fromSlug === '') { - return target === '' ? './' : target; + const relative = posix.relative(pageDirectory(fromSlug), pageDirectory(toSlug)); + return relative === '' || relative === '.' ? './' : `${relative}/`; +} + +/** The URL directory a page slug occupies: the slug itself, except for an `index` page. */ +function pageDirectory(slug: string): string { + if (slug === '' || slug === 'index') { + return '.'; } - return target === '' ? '../' : `../${target}`; + return slug.endsWith('/index') ? slug.slice(0, -'/index'.length) || '.' : slug; } function extensionOf(path: string): string { @@ -84,6 +153,12 @@ export function resolveRepoPath(sourcePath: string, relative: string): string { return out.join('/'); } +/** The directory part of a `/`-separated path (`schemas/Foo.md` → `schemas`). */ +function directoryOf(path: string): string { + const index = path.lastIndexOf('/'); + return index === -1 ? '' : path.slice(0, index); +} + /** Lowercased heading slugs for a markdown document (used for anchor checks). */ export function extractHeadingSlugs(markdown: string): Set { const headings = new Set(); @@ -148,22 +223,34 @@ export function rewriteDocMarkdown(markdown: string, context: DocLinkContext): s } const extension = extensionOf(path).toLowerCase(); const baseName = path.split(/[/\\]/).pop() ?? path; - const candidateSlug = slugifyDocFile(baseName.replace(/\.(md|mdx)$/i, '')); - - if ( - DOC_EXTENSIONS.has(extension) || - context.knownSlugs.has(candidateSlug) || - context.slugAliases?.has(candidateSlug) - ) { - const candidate = context.knownSlugs.has(candidateSlug) - ? candidateSlug - : slugifyDocFile(baseName); - const slug = context.slugAliases?.get(candidate) ?? candidate; + // A link resolves against the page it appears on, exactly as it does on + // GitHub: a page in `schemas/` names a sibling by file name and anything else + // by its path from there (`../concepts/Core-Concepts.md`). A bare file name + // also still resolves by name, so a flat corpus keeps working unchanged. + const pageDir = context.pagePath + ? directoryOf(resolveRepoPath(context.sourcePath, context.pagePath)) + : context.sourcePath; + const linkPath = resolveRepoPath(pageDir, path); + const docRelativePath = linkPath.startsWith(`${context.sourcePath}/`) + ? linkPath.slice(context.sourcePath.length + 1) + : linkPath; + const candidates = path.includes('/') + ? [docSlug(docRelativePath)] + : [docSlug(docRelativePath), slugifyDocFile(baseName.replace(/\.(md|mdx)$/i, ''))]; + const matched = candidates.find( + (candidate) => context.knownSlugs.has(candidate) || context.slugAliases?.has(candidate), + ); + + if (DOC_EXTENSIONS.has(extension) || matched !== undefined) { + const slug = + matched === undefined + ? (candidates[0] ?? '') + : (context.slugAliases?.get(matched) ?? matched); if (context.knownSlugs.has(slug)) { const rendered = isImage ? `![${label}]` : `[${label}]`; return `${rendered}(${usableHash(hash, slug, context)})`; } - const fallback = `https://github.com/${context.repository}/blob/${context.branch}/${resolveRepoPath(context.sourcePath, path)}${hash}`; + const fallback = `https://github.com/${context.repository}/blob/${context.branch}/${linkPath}${hash}`; const rendered = isImage ? `![${label}]` : `[${label}]`; return `${rendered}(${fallback})`; } @@ -179,7 +266,7 @@ export function rewriteDocMarkdown(markdown: string, context: DocLinkContext): s // Any other relative link (e.g. `src/src/Foo`, `../README`) is a // repository path. Rewrite it to an absolute GitHub blob URL so it is // never treated as a broken site-internal link. - const fallback = `https://github.com/${context.repository}/blob/${context.branch}/${resolveRepoPath(context.sourcePath, path)}${hash}`; + const fallback = `https://github.com/${context.repository}/blob/${context.branch}/${linkPath}${hash}`; return `[${label}](${fallback})`; } @@ -189,7 +276,8 @@ export function rewriteDocMarkdown(markdown: string, context: DocLinkContext): s /** Replace `` placeholders with base-relative doc links. */ export function resolveDoclinks(markdown: string, fromSlug: string): string { - return markdown.replace(//g, (_, slug: string) => + // Slugs may carry folder segments (`schemas/custom-rules`), so the placeholder accepts `/` as well. + return markdown.replace(//g, (_, slug: string) => resolveDocRelativeLink(fromSlug, slug), ); } diff --git a/src/src/lib/docs/redirects.ts b/src/src/lib/docs/redirects.ts index a5a970e..d7d759d 100644 --- a/src/src/lib/docs/redirects.ts +++ b/src/src/lib/docs/redirects.ts @@ -97,6 +97,67 @@ export function retiredChannelRedirects( return redirects; } +/** + * URLs a project declared it moved (`docs.redirects`), mapped to their new home. + * + * Unlike retired channels, a moved page has no derivable destination: the old slug + * and the new one share nothing, so the mapping is declared in the project + * catalogue. `writeStaticRedirect` never overwrites a file, which is what makes + * the declaration timing-safe - while the `live` channel still reads an older tag, + * the old slug is a real page and the redirect is simply not written. + */ +export function docsPageRedirects(projects: readonly ResolvedProject[]): DocChannelRedirect[] { + const redirects: DocChannelRedirect[] = []; + for (const project of projects) { + if (project.docs?.source !== 'github-path' || project.status === 'archived') { + continue; + } + const moved = project.docs.redirects ?? {}; + for (const [from, to] of Object.entries(moved)) { + redirects.push({ + from: `/docs/${project.id}/${from}/`, + to: `/docs/${project.id}/${to}/`, + title: 'This documentation page moved…', + message: 'The page now lives at its new address.', + }); + } + } + return redirects; +} + +/** + * Write the moved-page redirects into a built site. Returns the paths written, relative to `dist`. + * + * A redirect is written only when its destination is published *and* its source is not: + * + * - the source check is `writeStaticRedirect`'s - a real page owns its URL, so a redirect never shadows one; + * - the destination check is this function's. A project that nests its pages changes every slug at once, but the + * default channel keeps reading the last release tag until the next release is promoted (ADR 0009), so the new + * address can be unpublished while the declaration is already in source. Forwarding a reader into a 404 is worse + * than leaving the old URL alone, and the post-build link crawl says so - which is how this was found. + */ +export function writeDocsPageRedirects( + dist: string, + projects: readonly ResolvedProject[], +): string[] { + const written: string[] = []; + for (const redirect of docsPageRedirects(projects)) { + if (!isPublished(dist, redirect.to)) { + continue; + } + const relative = writeStaticRedirect(dist, redirect); + if (relative) { + written.push(relative); + } + } + return written; +} + +/** Whether a site path was written by the build as a page (`/a/b/` → `a/b/index.html`). */ +function isPublished(dist: string, path: string): boolean { + return existsSync(join(dist, ...path.split('/').filter(Boolean), 'index.html')); +} + function escapeHtml(value: string): string { return value .replace(/&/g, '&') diff --git a/src/src/lib/manifest/schema.ts b/src/src/lib/manifest/schema.ts index 0277040..74d809c 100644 --- a/src/src/lib/manifest/schema.ts +++ b/src/src/lib/manifest/schema.ts @@ -104,6 +104,16 @@ const docsPathSchema = z.object({ order: z.array(z.string()).optional(), exclude: z.array(z.string()).optional(), channels: docsChannelsSchema.optional(), + /** + * Slugs that moved, as `previous: current`. + * + * Reorganising a documentation tree changes published URLs, and a reader's + * bookmark should not 404 because of it. The site writes a meta-refresh + * redirect for every pair; a redirect never shadows a real page, so an entry + * stays inert while the old slug is still published — for example on a `live` + * channel pinned to an older tag, which activates it at promotion. + */ + redirects: z.record(z.string().min(1), z.string().min(1)).optional(), }); /** diff --git a/src/tests/unit/docs-redirects.test.ts b/src/tests/unit/docs-redirects.test.ts index 278448b..0624e97 100644 --- a/src/tests/unit/docs-redirects.test.ts +++ b/src/tests/unit/docs-redirects.test.ts @@ -7,8 +7,10 @@ import { dirname, join } from 'node:path'; import { DOCS_MANIFEST_SCHEMA } from '../../src/lib/docs/aggregate'; import { + docsPageRedirects, renderChannelRedirect, retiredChannelRedirects, + writeDocsPageRedirects, writeRetiredChannelRedirects, writeRetiredPageRedirects, } from '../../src/lib/docs/redirects'; @@ -134,3 +136,81 @@ describe('retired documentation channel redirects', () => { } }); }); + +describe('moved documentation page redirects', () => { + test('every nested page keeps the flat URL it had before the folders', () => { + // ZodSharp's documentation moved into topic folders, which made each folder part of the page's URL. The + // former flat address of every moved page is its file name alone, so the declared map must cover each one. + const zodsharp = projects.find((project) => project.id === 'zodsharp'); + const moved = zodsharp?.docs?.source === 'github-path' ? (zodsharp.docs.redirects ?? {}) : {}; + const nested = Object.values(moved).filter((slug) => slug.includes('/')); + + expect(nested.length).toBe(Object.keys(moved).length); + expect(moved['performance']).toBe('quality/performance'); + expect(moved['custom-rules']).toBe('schemas/custom-rules'); + + for (const [from, to] of Object.entries(moved)) { + // The former slug is the page's file name, slugified. + expect(from).toBe(to.slice(to.lastIndexOf('/') + 1)); + } + + expect(docsPageRedirects(projects)).toContainEqual({ + from: '/docs/zodsharp/performance/', + to: '/docs/zodsharp/quality/performance/', + title: 'This documentation page moved…', + message: 'The page now lives at its new address.', + }); + }); + + test('a moved page is written as a redirect once its new address is published', () => { + const dist = mkdtempSync(join(tmpdir(), 'purview-redirects-')); + try { + // The destination exists in this channel, the old slug does not: the redirect is written. + const target = join(dist, 'docs', 'zodsharp', 'quality', 'performance', 'index.html'); + mkdirSync(dirname(target), { recursive: true }); + writeFileSync(target, 'real page', 'utf8'); + + const written = writeDocsPageRedirects(dist, projects); + expect(written).toContain(join('docs', 'zodsharp', 'performance', 'index.html')); + + const html = readFileSync( + join(dist, 'docs', 'zodsharp', 'performance', 'index.html'), + 'utf8', + ); + expect(html).toContain('url=/docs/zodsharp/quality/performance/'); + expect(html).toContain('name="robots" content="noindex, follow"'); + + // A second run writes nothing: the redirects are created once. + expect(writeDocsPageRedirects(dist, projects)).toEqual([]); + } finally { + rmSync(dist, { recursive: true, force: true }); + } + }); + + test('a moved page is left alone while its new address is unpublished', () => { + // This is the promotion lag: the folders are in source, but the default channel still reads the release tag from + // before the move, so `quality/performance/` is not published yet. Forwarding into a 404 - or shadowing the live + // page at the old slug - would both be wrong, so nothing is written. + const dist = mkdtempSync(join(tmpdir(), 'purview-redirects-')); + try { + const livePage = join(dist, 'docs', 'zodsharp', 'performance', 'index.html'); + mkdirSync(dirname(livePage), { recursive: true }); + writeFileSync(livePage, 'live page', 'utf8'); + + expect(writeDocsPageRedirects(dist, projects)).toEqual([]); + expect(readFileSync(livePage, 'utf8')).toBe('live page'); + + // The same page, once the release that carries the folders is promoted: the old slug is gone and the new + // address is published, so the redirect appears. + rmSync(livePage); + const target = join(dist, 'docs', 'zodsharp', 'quality', 'performance', 'index.html'); + mkdirSync(dirname(target), { recursive: true }); + writeFileSync(target, 'real page', 'utf8'); + expect(writeDocsPageRedirects(dist, projects)).toEqual([ + join('docs', 'zodsharp', 'performance', 'index.html'), + ]); + } finally { + rmSync(dist, { recursive: true, force: true }); + } + }); +}); diff --git a/src/tests/unit/docs.test.ts b/src/tests/unit/docs.test.ts index b12ba6b..7d3ddb7 100644 --- a/src/tests/unit/docs.test.ts +++ b/src/tests/unit/docs.test.ts @@ -23,7 +23,9 @@ import { renderFrontmatter, } from '../../src/lib/docs/frontmatter'; import { + docSlug, extractHeadingSlugs, + isUrlSafeDocSlug, resolveDocRelativeLink, resolveRepoPath, rewriteDocMarkdown, @@ -59,6 +61,54 @@ describe('slugifyDocFile', () => { }); }); +describe('docSlug', () => { + test('leaves a flat corpus flat, so no published URL moves', () => { + expect(docSlug('Getting-Started.md')).toBe('getting-started'); + expect(docSlug('Getting-Started')).toBe('getting-started'); + }); + + test('keeps each topic folder as a path segment', () => { + expect(docSlug('schemas/Custom-Rules.md')).toBe('schemas/custom-rules'); + expect(docSlug('serialization/SystemTextJson-Integration.md')).toBe( + 'serialization/systemtextjson-integration', + ); + expect(docSlug('/nested//deeper/Page.md')).toBe('nested/deeper/page'); + }); + + test('reduces a folder that would need encoding to a URL-safe segment', () => { + expect(docSlug('Topic Folder/Page Name.md')).toBe('topic-folder/page-name'); + expect(docSlug('Q&A/Dot.Net.md')).toBe('q-a/dot-net'); + expect(docSlug('Ünïcode/Page.md')).toBe('n-code/page'); + expect(docSlug('API v2 (beta)/Page.md')).toBe('api-v2-beta/page'); + }); +}); + +describe('isUrlSafeDocSlug', () => { + test('accepts the slugs the sync emits, folders included', () => { + expect(isUrlSafeDocSlug('index')).toBe(true); + expect(isUrlSafeDocSlug('custom-rules')).toBe(true); + expect(isUrlSafeDocSlug('schemas/custom-rules')).toBe(true); + expect(isUrlSafeDocSlug('a/b/c/page-2')).toBe(true); + }); + + test('rejects a slug that would need an address to be encoded', () => { + expect(isUrlSafeDocSlug('')).toBe(false); + expect(isUrlSafeDocSlug('schemas/')).toBe(false); + expect(isUrlSafeDocSlug('a//b')).toBe(false); + expect(isUrlSafeDocSlug('/page')).toBe(false); + expect(isUrlSafeDocSlug('-page')).toBe(false); + expect(isUrlSafeDocSlug('page-')).toBe(false); + expect(isUrlSafeDocSlug('Page')).toBe(false); + expect(isUrlSafeDocSlug('page name')).toBe(false); + }); + + test('catches a folder whose name reduces to nothing', () => { + // `___` slugifies to an empty segment, which would publish the page at a URL that cannot be written down. + expect(docSlug('___/Page.md')).toBe('/page'); + expect(isUrlSafeDocSlug(docSlug('___/Page.md'))).toBe(false); + }); +}); + describe('front matter', () => { const markdown = [ '# Getting Started', @@ -270,6 +320,52 @@ describe('link rewriting', () => { }); }); +describe('link rewriting inside a topic folder', () => { + const nested = (overrides: Partial = {}) => + context({ + sourcePath: 'docs/wiki', + pagePath: 'schemas/Custom-Rules.md', + knownSlugs: new Set([ + 'index', + 'schemas/custom-rules', + 'schemas/string-validation', + 'concepts/core-concepts', + 'serialization/jsonschema-export', + ]), + ...overrides, + }); + + test('resolves a sibling by file name', () => { + expect(rewriteDocMarkdown('[String](String-Validation.md)', nested())).toBe( + '[String]()', + ); + }); + + test('resolves a link that climbs out of the folder', () => { + expect(rewriteDocMarkdown('[Core](../concepts/Core-Concepts.md)', nested())).toBe( + '[Core]()', + ); + expect(rewriteDocMarkdown('[Json](../serialization/JsonSchema-Export.md)', nested())).toBe( + '[Json]()', + ); + }); + + test('reaches the landing page through its slug alias', () => { + const output = rewriteDocMarkdown( + '[Start](../Getting-Started.md)', + nested({ slugAliases: new Map([['getting-started', 'index']]) }), + ); + expect(output).toBe('[Start]()'); + }); + + test('rewrites a page-relative repository path to a GitHub blob URL', () => { + // Three levels up from `docs/wiki/schemas/` is the repository root. + expect(rewriteDocMarkdown('[Source](../../../src/src/Z.cs)', nested())).toBe( + '[Source](https://github.com/purview-dev/demo/blob/main/src/src/Z.cs)', + ); + }); +}); + describe('docs root page selection', () => { test('allows a configured non-index root when no conventional root exists', () => { const selected = selectDocsRootFile( @@ -357,6 +453,19 @@ describe('resolveDocRelativeLink', () => { expect(resolveDocRelativeLink('getting-started', 'index')).toBe('../'); expect(resolveDocRelativeLink('index', 'index')).toBe('./'); }); + + test('counts the hops a topic folder adds', () => { + // A nested page is served at `/docs//schemas/custom-rules/`, so reaching a sibling folder is two + // levels up, not one. + expect(resolveDocRelativeLink('schemas/custom-rules', 'serialization/jsonschema-export')).toBe( + '../../serialization/jsonschema-export/', + ); + expect(resolveDocRelativeLink('schemas/custom-rules', 'schemas/string-validation')).toBe( + '../string-validation/', + ); + expect(resolveDocRelativeLink('schemas/custom-rules', 'index')).toBe('../../'); + expect(resolveDocRelativeLink('index', 'schemas/custom-rules')).toBe('schemas/custom-rules/'); + }); }); describe('resolveRepoPath', () => {