diff --git a/.changeset/quiet-tools-search.md b/.changeset/quiet-tools-search.md new file mode 100644 index 00000000..365a6818 --- /dev/null +++ b/.changeset/quiet-tools-search.md @@ -0,0 +1,5 @@ +--- +'@transloadit/mcp-server': minor +--- + +Add `transloadit_list_docs` and `transloadit_get_doc_help` for discovering public documentation and reading bounded Markdown. Their discovery/help naming matches the existing Robot tools; optional search and API, SDK, FAQ, and Robot scopes cover broader integration guidance. diff --git a/docs/prompts/2026-11-12-mcp-docs-review.md b/docs/prompts/2026-11-12-mcp-docs-review.md new file mode 100644 index 00000000..fb7d3e57 --- /dev/null +++ b/docs/prompts/2026-11-12-mcp-docs-review.md @@ -0,0 +1,15 @@ +# MCP documentation tools — PR #472 review follow-up + +- [x] Address kvz’s September 23 review: use `transloadit_list_docs` and `transloadit_get_doc_help`, matching the existing Robot discovery/help names. Keep the existing Robot tools unchanged; use their optional `search` input convention. +- [x] Explain the overlap: Robot tools return structured Step parameter references; documentation tools cover API authentication/endpoints, SDK setup, FAQs, integration concepts and full Robot prose. Update descriptions, README and the release changeset. +- [x] Rebase onto current main (`80251a8`), resolving the tool registry and README conflicts without reverting authentication metadata or Builtin discovery behavior. +- [x] Preserve the earlier scoped-index and bounded-stream review fixes. Retire the rollout fallback now that scoped indexes are deployed: missing/oversized indexes return errors instead of silently incomplete results. +- [x] Validate today’s live indexes and fix observed duplicate URLs, missing overview links and poor multi-term ranking. Add deterministic regression coverage and MCP client-level naming, anonymous access, listing, help and sanitized-error tests. + +Live checks on November 12, 2026 at 14:08 UTC passed for API/authentication, FAQ/assembly expiry, SDK/Node.js, Robot/resize and all/authentication. Each expected page ranked first and was fetched as bounded Markdown. The FAQ page was returned whole (455 characters); longer pages were truncated at 1,500 characters. Unfiltered discovery returned 20 distinct pages; `/docs/` resolved to the `/docs.md` overview. + +The MCP workspace typecheck, all 363 MCP unit tests, root `yarn check`, and both stdio integration tests pass. A real in-memory MCP client also listed and fetched the live authentication guide without account credentials. CI is recorded in the PR’s validation section. + +Independent review limitation: `council-review` is not installed. A read-only Claude CLI review was attempted, but its OAuth session is expired and could not be refreshed. No independent review is claimed. + +GitHub prohibits force pushes to the existing PR branch. The changes were rebased locally and validated; publication retains the previous PR head as a merge parent, preserving that history without changing the tested implementation. diff --git a/packages/mcp-server/CHANGELOG.md b/packages/mcp-server/CHANGELOG.md index 944348f7..3e5a194d 100644 --- a/packages/mcp-server/CHANGELOG.md +++ b/packages/mcp-server/CHANGELOG.md @@ -1,5 +1,13 @@ # @transloadit/mcp-server +## 0.8.7 + +### Patch Changes + +- 17ea841: Storage requests (`getStoredAsset`, `listStoredAssets`, `moveStoredAsset` and `deleteStoredAsset`) now sign their HTTP method and URL path into `params.auth` as `method` and `path`, as the Transloadit API requires for Storage requests. +- Updated dependencies [17ea841] + - @transloadit/node@6.8.0 + ## 0.8.6 ### Patch Changes diff --git a/packages/mcp-server/README.md b/packages/mcp-server/README.md index ee42e407..9415c65a 100644 --- a/packages/mcp-server/README.md +++ b/packages/mcp-server/README.md @@ -350,6 +350,8 @@ The JSON config accepts the same keys as `createTransloaditMcpHttpHandler()`, in | Tool | Account | OAuth scopes | Notes | | ---------------------------------------- | ------- | ------------------------------------ | ----------------------------------------------- | | `transloadit_lint_assembly_instructions` | no | none | read-only | +| `transloadit_list_docs` | no | none | read-only; public documentation | +| `transloadit_get_doc_help` | no | none | read-only; public documentation | | `transloadit_list_robots` | no | none | read-only | | `transloadit_get_robot_help` | no | none | read-only | | `transloadit_create_assembly` | yes | `assemblies:write`, `templates:read` | destructive, open-world (URL imports), widget | @@ -412,6 +414,19 @@ and null for older Builtins without a declared schema. Workspace Templates omit `transloadit_get_profile` returns `{ id, name?, nickname? }` for the Workspace behind the current credentials, derived from the Workspace's own Assemblies or Templates. +### Documentation and Robot help + +| Discovery | Help | Coverage | +| --- | --- | --- | +| `transloadit_list_robots` | `transloadit_get_robot_help` | Structured Robot summaries, required/optional parameters and examples for writing Assembly Steps. | +| `transloadit_list_docs` | `transloadit_get_doc_help` | Public Markdown documentation for API authentication and endpoints, SDK setup, FAQs, integration concepts and full Robot articles. | + +The existing Robot tools keep their names and behavior. Documentation tools add the broader guides and prose that a Robot parameter reference does not cover. Like `transloadit_list_robots`, `transloadit_list_docs` accepts an optional `search` filter. It also accepts `scope` (`all`, `api`, `faq`, `robots`, `sdks`) and `limit` (1–20, default 8); omitting `search` lists pages alphabetically. Search prioritizes pages matching more of the search terms and returns each Markdown URL once. + +`transloadit_get_doc_help` accepts `path_or_url` from a listing or a public documentation path, plus `max_chars` (1,000–100,000, default 50,000). It returns `{ title, url, markdown, truncated }`. It reads the response as a bounded stream, rejects non-documentation URLs and redirects, and reports temporary documentation failures without returning upstream diagnostics. + +The tools use the live `/docs/llms.txt` and scoped API/FAQ/Robot/SDK indexes. Unavailable or oversized indexes produce an explicit error instead of silently searching an incomplete root catalog. They fetch only public documentation and need no account credentials in caller-credentials mode; hosted deployments still apply the OAuth policy described above. + ### Result widget `transloadit_create_assembly` and `transloadit_wait_for_assembly` link the MCP Apps resource diff --git a/packages/mcp-server/package.json b/packages/mcp-server/package.json index e3e26c83..eb383721 100644 --- a/packages/mcp-server/package.json +++ b/packages/mcp-server/package.json @@ -1,6 +1,6 @@ { "name": "@transloadit/mcp-server", - "version": "0.8.6", + "version": "0.8.7", "description": "Transloadit MCP server", "keywords": [ "mcp", @@ -63,7 +63,7 @@ }, "dependencies": { "@modelcontextprotocol/sdk": "^1.29.0", - "@transloadit/node": "^6.7.0", + "@transloadit/node": "^6.8.0", "@transloadit/sev-logger": "^0.1.9", "@transloadit/zod": "^7.5.0", "ajv": "^8.17.1", diff --git a/packages/mcp-server/server.json b/packages/mcp-server/server.json index 6092a84b..9610ba67 100644 --- a/packages/mcp-server/server.json +++ b/packages/mcp-server/server.json @@ -3,7 +3,7 @@ "name": "io.github.transloadit/mcp-server", "title": "Transloadit Media Processing", "description": "Process video, audio, images and documents with Transloadit Robots. Connect by URL with OAuth.", - "version": "0.8.6", + "version": "0.8.7", "websiteUrl": "https://transloadit.com/docs/sdks/mcp-server/", "icons": [ { @@ -21,7 +21,7 @@ { "registryType": "npm", "identifier": "@transloadit/mcp-server", - "version": "0.8.6", + "version": "0.8.7", "runtimeHint": "npx", "packageArguments": [ { diff --git a/packages/mcp-server/src/docs.ts b/packages/mcp-server/src/docs.ts new file mode 100644 index 00000000..00da2295 --- /dev/null +++ b/packages/mcp-server/src/docs.ts @@ -0,0 +1,203 @@ +export type DocsScope = 'all' | 'api' | 'faq' | 'robots' | 'sdks' + +export interface DocsSearchResult { + title: string + url: string + description?: string +} + +export interface DocsPage { + title: string + url: string + markdown: string + truncated: boolean +} + +const docsOrigin = 'https://transloadit.com' +const docsFetchTimeoutMs = 10_000 +const docsIndexMaxChars = 500_000 +const docsIndexUrls: Record = { + all: [ + `${docsOrigin}/docs/llms.txt`, + `${docsOrigin}/docs/api/llms.txt`, + `${docsOrigin}/docs/faq/llms.txt`, + `${docsOrigin}/docs/robots/llms.txt`, + `${docsOrigin}/docs/sdks/llms.txt`, + ], + api: [`${docsOrigin}/docs/api/llms.txt`], + faq: [`${docsOrigin}/docs/faq/llms.txt`], + robots: [`${docsOrigin}/docs/robots/llms.txt`], + sdks: [`${docsOrigin}/docs/sdks/llms.txt`], +} +const markdownLinkPattern = /^- \[([^\]]+)\]\(([^)]+)\)(?::\s*(.*))?$/u + +function normalizeDocsUrl(value: string): URL | undefined { + if (!URL.canParse(value, docsOrigin)) return + + const url = new URL(value, docsOrigin) + if (url.origin !== docsOrigin || url.username || url.password) return + url.hash = '' + url.search = '' + if (url.pathname === '/llms.txt') return url + if (['/docs', '/docs/', '/docs.md'].includes(url.pathname)) { + url.pathname = '/docs.md' + return url + } + if (!url.pathname.startsWith('/docs/')) return + + if (!url.pathname.endsWith('.md') && !url.pathname.endsWith('.txt')) { + url.pathname = `${url.pathname.replace(/\/$/u, '')}.md` + } + return url +} + +interface DocsTextResponse { + text: string + truncated: boolean +} + +async function readBoundedResponseText( + response: Response, + maxChars: number, +): Promise { + if (response.body === null) { + return { text: '', truncated: false } + } + + const decoder = new TextDecoder() + const reader = response.body.getReader() + let text = '' + + while (true) { + const { done, value } = await reader.read() + if (done) { + text += decoder.decode() + return { + text: text.slice(0, maxChars), + truncated: text.length > maxChars, + } + } + + text += decoder.decode(value, { stream: true }) + if (text.length <= maxChars) continue + + await reader.cancel() + return { text: text.slice(0, maxChars), truncated: true } + } +} + +async function fetchDocsText( + url: string, + maxChars: number, + fetcher: typeof fetch, +): Promise { + const response = await fetcher(url, { + headers: { Accept: 'text/markdown, text/plain;q=0.9' }, + redirect: 'error', + signal: AbortSignal.timeout(docsFetchTimeoutMs), + }) + if (!response.ok) { + throw new Error(`Documentation request failed with HTTP ${response.status}.`) + } + const contentType = response.headers.get('content-type') ?? '' + if (!contentType.includes('text/markdown') && !contentType.includes('text/plain')) { + throw new Error(`Documentation request returned unsupported content type: ${contentType}.`) + } + return readBoundedResponseText(response, maxChars) +} + +function parseIndex(index: string): DocsSearchResult[] { + const results = new Map() + for (const line of index.split('\n')) { + const match = markdownLinkPattern.exec(line.trim()) + if (match === null) continue + + const [, title, rawUrl, description] = match + if (title === undefined || rawUrl === undefined) continue + const url = normalizeDocsUrl(rawUrl) + if (url === undefined || !url.pathname.endsWith('.md')) continue + if (!results.has(url.href)) results.set(url.href, { title, url: url.href, description }) + } + return [...results.values()] +} + +function matchesScope(result: DocsSearchResult, scope: DocsScope): boolean { + if (scope === 'all') return true + return new URL(result.url).pathname.startsWith(`/docs/${scope}/`) +} + +function getSearchScore( + result: DocsSearchResult, + terms: string[], +): { matches: number; weight: number } { + const title = result.title.toLowerCase() + const description = result.description?.toLowerCase() ?? '' + const url = result.url.toLowerCase() + + let matches = 0 + let weight = 0 + for (const term of terms) { + const score = title.includes(term) + ? 5 + : description.includes(term) + ? 2 + : url.includes(term) + ? 1 + : 0 + if (score === 0) continue + matches++ + weight += score + } + return { matches, weight } +} + +/** List public documentation pages, optionally ranked by search text, without account credentials. */ +export async function listTransloaditDocs( + search: string | undefined, + scope: DocsScope, + limit: number, + fetcher: typeof fetch = fetch, +): Promise { + const terms = search?.toLowerCase().split(/\s+/u).filter(Boolean) ?? [] + // The scoped indexes are deployed. Fail explicitly if one is unavailable rather than + // silently replacing it with the incomplete root index and presenting missing results. + const indexResponses = await Promise.all( + docsIndexUrls[scope].map((url) => fetchDocsText(url, docsIndexMaxChars, fetcher)), + ) + if (indexResponses.some(({ truncated }) => truncated)) { + throw new Error('A documentation index exceeded the maximum supported size.') + } + const index = indexResponses.map(({ text }) => text).join('\n') + + return parseIndex(index) + .filter((result) => matchesScope(result, scope)) + .map((result) => ({ result, score: getSearchScore(result, terms) })) + .filter(({ score }) => terms.length === 0 || score.matches > 0) + .sort( + (left, right) => + right.score.matches - left.score.matches || + right.score.weight - left.score.weight || + left.result.title.localeCompare(right.result.title), + ) + .slice(0, limit) + .map(({ result }) => result) +} + +/** Fetch one public Transloadit documentation page as bounded Markdown. */ +export async function getTransloaditDocHelp( + pathOrUrl: string, + maxChars: number, + fetcher: typeof fetch = fetch, +): Promise { + const url = normalizeDocsUrl(pathOrUrl) + if (url === undefined) return + + const { text: markdown, truncated } = await fetchDocsText(url.toString(), maxChars, fetcher) + const title = markdown.match(/^#\s+(.+)$/mu)?.[1] ?? url.pathname + return { + title, + url: url.toString(), + markdown, + truncated, + } +} diff --git a/packages/mcp-server/src/server.ts b/packages/mcp-server/src/server.ts index 1652be2e..ee9c877c 100644 --- a/packages/mcp-server/src/server.ts +++ b/packages/mcp-server/src/server.ts @@ -37,6 +37,7 @@ import { selectBuiltinInputUrls, validateBuiltinFields, } from './builtin-shortcuts.ts' +import { getTransloaditDocHelp, listTransloaditDocs } from './docs.ts' import { buildBearerChallenge, extractBearerToken } from './http-helpers.ts' import { buildRedactor, getDiagnosticsLogger } from './logger.ts' import { assertServerOptions } from './options.ts' @@ -173,6 +174,44 @@ const toolMessageSchema = z.object({ path: z.string().optional(), }) +const listDocsInputSchema = z.object({ + search: z.string().trim().min(2).optional(), + scope: z.enum(['all', 'api', 'faq', 'robots', 'sdks']).default('all'), + limit: z.number().int().min(1).max(20).default(8), +}) + +const listDocsOutputSchema = z.object({ + status: z.enum(['ok', 'error']), + results: z + .array( + z.object({ + title: z.string(), + url: z.string(), + description: z.string().optional(), + }), + ) + .optional(), + errors: z.array(toolMessageSchema).optional(), +}) + +const getDocHelpInputSchema = z.object({ + path_or_url: z.string().min(1), + max_chars: z.number().int().min(1_000).max(100_000).default(50_000), +}) + +const getDocHelpOutputSchema = z.object({ + status: z.enum(['ok', 'error']), + document: z + .object({ + title: z.string(), + url: z.string(), + markdown: z.string(), + truncated: z.boolean(), + }) + .optional(), + errors: z.array(toolMessageSchema).optional(), +}) + const listRobotsInputSchema = z.object({ category: z.string().optional(), search: z.string().optional(), @@ -1365,6 +1404,46 @@ export const createTransloaditMcpServer = ( } // Builtin templates supersede the old golden template tool; no legacy alias by design. + register( + 'transloadit_list_docs', + listDocsInputSchema, + listDocsOutputSchema, + async ({ search, scope, limit }) => { + try { + const results = await listTransloaditDocs(search, scope, limit) + return buildToolResponse({ status: 'ok', results }) + } catch { + return buildToolError( + 'mcp_docs_unavailable', + 'The Transloadit documentation index is temporarily unavailable.', + ) + } + }, + ) + + register( + 'transloadit_get_doc_help', + getDocHelpInputSchema, + getDocHelpOutputSchema, + async ({ path_or_url, max_chars }) => { + try { + const document = await getTransloaditDocHelp(path_or_url, max_chars) + if (document === undefined) { + return buildToolError( + 'mcp_invalid_docs_url', + 'Provide a public path under /docs or a transloadit.com documentation URL.', + ) + } + return buildToolResponse({ status: 'ok', document }) + } catch { + return buildToolError( + 'mcp_docs_unavailable', + 'The requested Transloadit documentation page is temporarily unavailable.', + ) + } + }, + ) + register( 'transloadit_lint_assembly_instructions', lintAssemblyInputSchema, diff --git a/packages/mcp-server/src/tool-metadata.ts b/packages/mcp-server/src/tool-metadata.ts index 01f394a8..ae635479 100644 --- a/packages/mcp-server/src/tool-metadata.ts +++ b/packages/mcp-server/src/tool-metadata.ts @@ -15,6 +15,8 @@ export type ToolSecurityScheme = { type: 'noauth' } | { type: 'oauth2'; scopes: export const toolNames = [ 'transloadit_lint_assembly_instructions', + 'transloadit_list_docs', + 'transloadit_get_doc_help', 'transloadit_create_assembly', 'transloadit_get_assembly_status', 'transloadit_wait_for_assembly', @@ -92,6 +94,22 @@ export const widgetToolNames: readonly ToolName[] = [ * server and the static server card so the two never drift. */ const genericToolMetadata: Record<(typeof toolNames)[number], ToolMetadata> = { + transloadit_list_docs: { + title: 'List Transloadit documentation', + description: + 'List public documentation pages about the API, SDKs, FAQs, Robots and integration concepts, optionally filtered by search text and scope. Returns titles, Markdown URLs and summaries, without fetching page bodies.', + annotations: readOnly, + access: 'public', + scopes: [], + }, + transloadit_get_doc_help: { + title: 'Get documentation help', + description: + 'Read one public documentation page as bounded Markdown for API authentication, SDK setup, troubleshooting or integration guidance. Accepts a /docs path or transloadit.com documentation URL. Robot parameter help is a separate structured reference.', + annotations: readOnly, + access: 'public', + scopes: [], + }, transloadit_lint_assembly_instructions: { title: 'Lint Assembly Instructions', description: diff --git a/packages/mcp-server/test/e2e/stdio.test.ts b/packages/mcp-server/test/e2e/stdio.test.ts index b2ded9bc..c797827c 100644 --- a/packages/mcp-server/test/e2e/stdio.test.ts +++ b/packages/mcp-server/test/e2e/stdio.test.ts @@ -20,6 +20,8 @@ describe('mcp-server stdio', { timeout: 20000 }, () => { const toolNames = toolsResult.tools.map((tool) => tool.name) expect(toolNames).toContain('transloadit_lint_assembly_instructions') + expect(toolNames).toContain('transloadit_list_docs') + expect(toolNames).toContain('transloadit_get_doc_help') const result = await client.callTool({ name: 'transloadit_lint_assembly_instructions', diff --git a/packages/mcp-server/test/unit/docs-tools.test.ts b/packages/mcp-server/test/unit/docs-tools.test.ts new file mode 100644 index 00000000..ee37dc91 --- /dev/null +++ b/packages/mcp-server/test/unit/docs-tools.test.ts @@ -0,0 +1,125 @@ +import { Client } from '@modelcontextprotocol/sdk/client/index.js' +import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +import { createTransloaditMcpServer } from '../../src/server.ts' +import { parseToolPayload } from '../e2e/mcp-client.ts' + +describe('MCP documentation tools', () => { + let client: Client + let server: ReturnType + + beforeEach(async () => { + server = createTransloaditMcpServer({}) + client = new Client({ name: 'docs-tools-test', version: '1.0.0' }) + const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair() + await Promise.all([server.connect(serverTransport), client.connect(clientTransport)]) + }) + + afterEach(async () => { + vi.unstubAllGlobals() + await client.close() + await server.close() + }) + + it('exposes matching discovery/help names alongside the existing Robot tools without account credentials', async () => { + const { tools } = await client.listTools() + expect(tools.map(({ name }) => name)).toEqual( + expect.arrayContaining([ + 'transloadit_list_docs', + 'transloadit_get_doc_help', + 'transloadit_list_robots', + 'transloadit_get_robot_help', + ]), + ) + expect( + tools.find(({ name }) => name === 'transloadit_list_docs')?._meta?.securitySchemes, + ).toEqual([{ type: 'noauth' }]) + expect( + tools.find(({ name }) => name === 'transloadit_get_doc_help')?._meta?.securitySchemes, + ).toEqual([{ type: 'noauth' }]) + }) + + it('lists a scope without a search term and returns its public Markdown links', async () => { + const fetchMock = vi + .fn() + .mockResolvedValue( + new Response( + '- [Authentication](https://transloadit.com/docs/api/authentication.md): Sign API requests.', + ), + ) + vi.stubGlobal('fetch', fetchMock) + const result = await client.callTool({ + name: 'transloadit_list_docs', + arguments: { scope: 'api' }, + }) + expect(parseToolPayload(result)).toMatchObject({ + status: 'ok', + results: [ + { title: 'Authentication', url: 'https://transloadit.com/docs/api/authentication.md' }, + ], + }) + expect(fetchMock).toHaveBeenCalledExactlyOnceWith( + 'https://transloadit.com/docs/api/llms.txt', + expect.objectContaining({ headers: { Accept: 'text/markdown, text/plain;q=0.9' } }), + ) + }) + + it('filters discovered pages with the same search input name as list_robots', async () => { + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValue( + new Response( + '- [Authentication](https://transloadit.com/docs/api/authentication.md): Sign API requests.', + ), + ), + ) + const result = await client.callTool({ + name: 'transloadit_list_docs', + arguments: { scope: 'api', search: 'something absent' }, + }) + expect(parseToolPayload(result)).toEqual({ status: 'ok', results: [] }) + }) + + it('returns bounded Markdown through the renamed help tool', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue(new Response(`# Authentication\n${'x'.repeat(2000)}`)), + ) + const result = await client.callTool({ + name: 'transloadit_get_doc_help', + arguments: { path_or_url: '/docs/api/authentication/', max_chars: 1000 }, + }) + expect(parseToolPayload(result)).toEqual({ + status: 'ok', + document: { + title: 'Authentication', + url: 'https://transloadit.com/docs/api/authentication.md', + markdown: `# Authentication\n${'x'.repeat(983)}`, + truncated: true, + }, + }) + }) + + it('sanitizes unavailable indexes instead of returning upstream response bodies', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue(new Response('private upstream diagnostic', { status: 503 })), + ) + const result = await client.callTool({ + name: 'transloadit_list_docs', + arguments: { scope: 'api' }, + }) + expect(parseToolPayload(result)).toEqual({ + status: 'error', + errors: [ + { + code: 'mcp_docs_unavailable', + message: 'The Transloadit documentation index is temporarily unavailable.', + }, + ], + }) + }) +}) diff --git a/packages/mcp-server/test/unit/docs.test.ts b/packages/mcp-server/test/unit/docs.test.ts new file mode 100644 index 00000000..ecfbbccf --- /dev/null +++ b/packages/mcp-server/test/unit/docs.test.ts @@ -0,0 +1,183 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' + +import { getTransloaditDocHelp, listTransloaditDocs } from '../../src/docs.ts' + +afterEach(() => { + vi.unstubAllGlobals() +}) + +describe('MCP documentation retrieval', () => { + it('searches and scopes the public documentation index', async () => { + const fetchMock = vi + .fn() + .mockResolvedValue( + new Response( + [ + '# Transloadit', + '- [Resize images](https://transloadit.com/docs/robots/image-resize.md): Resize and crop images.', + '- [Authentication](https://transloadit.com/docs/api/authentication.md): Sign API requests.', + '- [External](https://example.com/docs/image.md): Must not be returned.', + ].join('\n'), + { status: 200 }, + ), + ) + vi.stubGlobal('fetch', fetchMock) + + const results = await listTransloaditDocs('resize image', 'robots', 5) + + expect(fetchMock).toHaveBeenCalledWith( + 'https://transloadit.com/docs/robots/llms.txt', + expect.anything(), + ) + expect(results).toEqual([ + { + title: 'Resize images', + url: 'https://transloadit.com/docs/robots/image-resize.md', + description: 'Resize and crop images.', + }, + ]) + }) + + it('reports a missing scoped index instead of silently searching an incomplete root index', async () => { + const fetchMock = vi.fn().mockResolvedValue(new Response('', { status: 404 })) + vi.stubGlobal('fetch', fetchMock) + + await expect(listTransloaditDocs('authentication', 'api', 5)).rejects.toThrow('HTTP 404') + expect(fetchMock).toHaveBeenCalledExactlyOnceWith( + 'https://transloadit.com/docs/api/llms.txt', + expect.anything(), + ) + }) + + it.each(['api', 'faq', 'robots', 'sdks'] as const)('reads the %s scoped index', async (scope) => { + const fetchMock = vi.fn().mockResolvedValue(new Response('')) + await listTransloaditDocs(undefined, scope, 8, fetchMock) + expect(fetchMock).toHaveBeenCalledExactlyOnceWith( + `https://transloadit.com/docs/${scope}/llms.txt`, + expect.objectContaining({ redirect: 'error' }), + ) + }) + + it('deduplicates today’s overlapping indexes and omits links to other indexes', async () => { + const fetchMock = vi + .fn() + .mockImplementation( + () => + new Response( + [ + '- [Documentation overview](https://transloadit.com/docs.md): Choose an integration path.', + '- [Authentication](https://transloadit.com/docs/api/authentication.md): Sign API requests.', + '- [Authentication duplicate](https://transloadit.com/docs/api/authentication.md): Sign API requests.', + '- [API endpoints](https://transloadit.com/docs/api/llms.txt): Endpoint catalog.', + ].join('\n'), + ), + ) + const results = await listTransloaditDocs(undefined, 'all', 8, fetchMock) + expect(fetchMock).toHaveBeenCalledTimes(5) + expect(results.map(({ title, url }) => ({ title, url }))).toEqual([ + { title: 'Authentication', url: 'https://transloadit.com/docs/api/authentication.md' }, + { title: 'Documentation overview', url: 'https://transloadit.com/docs.md' }, + ]) + }) + + it('ranks results matching both search terms before a title matching only one', async () => { + const fetchMock = vi + .fn() + .mockResolvedValue( + new Response( + [ + '- [Are Assembly IDs secure?](https://transloadit.com/docs/faq/uuid.md): Assembly identifiers.', + '- [Are there limits for upload time or processing time?](https://transloadit.com/docs/faq/assembly-expiry.md): An Assembly expires after its time limit.', + ].join('\n'), + ), + ) + const results = await listTransloaditDocs('assembly expiry', 'faq', 1, fetchMock) + expect(results[0]?.url).toBe('https://transloadit.com/docs/faq/assembly-expiry.md') + }) + + it('rejects an oversized index instead of presenting incomplete results', async () => { + const fetchMock = vi.fn().mockResolvedValue(new Response('x'.repeat(500_001))) + await expect(listTransloaditDocs(undefined, 'faq', 8, fetchMock)).rejects.toThrow( + 'index exceeded', + ) + }) + + it('normalizes the documentation overview linked by today’s index', async () => { + const fetchMock = vi.fn().mockResolvedValue(new Response('# Documentation\n')) + expect(await getTransloaditDocHelp('/docs/', 1000, fetchMock)).toMatchObject({ + url: 'https://transloadit.com/docs.md', + title: 'Documentation', + truncated: false, + }) + expect(fetchMock).toHaveBeenCalledExactlyOnceWith( + 'https://transloadit.com/docs.md', + expect.anything(), + ) + }) + + it('normalizes an HTML docs URL and returns bounded Markdown', async () => { + const fetchMock = vi.fn().mockResolvedValue( + new Response('# Image resize\n\nDetailed documentation.', { + status: 200, + }), + ) + vi.stubGlobal('fetch', fetchMock) + + const page = await getTransloaditDocHelp( + 'https://transloadit.com/docs/robots/image-resize/', + 20, + ) + + expect(fetchMock).toHaveBeenCalledWith( + 'https://transloadit.com/docs/robots/image-resize.md', + expect.objectContaining({ + headers: { Accept: 'text/markdown, text/plain;q=0.9' }, + }), + ) + expect(page).toEqual({ + title: 'Image resize', + url: 'https://transloadit.com/docs/robots/image-resize.md', + markdown: '# Image resize\n\nDeta', + truncated: true, + }) + }) + + it('streams and truncates oversized Markdown without buffering the full response', async () => { + const response = new Response(`# Large page\n\n${'x'.repeat(2_000)}`, { + headers: { 'Content-Type': 'text/markdown' }, + status: 200, + }) + const text = vi.spyOn(response, 'text') + const fetchMock = vi.fn().mockResolvedValue(response) + vi.stubGlobal('fetch', fetchMock) + + const page = await getTransloaditDocHelp('/docs/large-page/', 1_000) + + expect(page?.markdown).toHaveLength(1_000) + expect(page?.truncated).toBe(true) + expect(text).not.toHaveBeenCalled() + }) + + it('rejects a successful HTML response instead of returning it as Markdown', async () => { + const fetchMock = vi.fn().mockResolvedValue( + new Response('Error', { + headers: { 'Content-Type': 'text/html' }, + status: 200, + }), + ) + vi.stubGlobal('fetch', fetchMock) + + await expect(getTransloaditDocHelp('/docs/error/', 1_000)).rejects.toThrow( + 'unsupported content type', + ) + }) + + it('rejects non-Transloadit and non-documentation URLs without fetching', async () => { + const fetchMock = vi.fn() + vi.stubGlobal('fetch', fetchMock) + + expect(await getTransloaditDocHelp('https://example.com/docs/secrets/', 1000)).toBeUndefined() + expect(await getTransloaditDocHelp('https://transloadit.com/pricing/', 1000)).toBeUndefined() + expect(fetchMock).not.toHaveBeenCalled() + }) +}) diff --git a/packages/node/CHANGELOG.md b/packages/node/CHANGELOG.md index 4e4c20e4..38d76f13 100644 --- a/packages/node/CHANGELOG.md +++ b/packages/node/CHANGELOG.md @@ -1,5 +1,11 @@ # @transloadit/node +## 6.8.0 + +### Minor Changes + +- 17ea841: Storage requests (`getStoredAsset`, `listStoredAssets`, `moveStoredAsset` and `deleteStoredAsset`) now sign their HTTP method and URL path into `params.auth` as `method` and `path`, as the Transloadit API requires for Storage requests. + ## 6.7.0 ### Minor Changes diff --git a/packages/node/package.json b/packages/node/package.json index 0c68aa06..9770dd6c 100644 --- a/packages/node/package.json +++ b/packages/node/package.json @@ -1,6 +1,6 @@ { "name": "@transloadit/node", - "version": "6.7.0", + "version": "6.8.0", "description": "Node.js SDK and `transloadit` CLI for Transloadit", "homepage": "https://github.com/transloadit/node-sdk/tree/main/packages/node", "bugs": { diff --git a/packages/node/src/Transloadit.ts b/packages/node/src/Transloadit.ts index f1d97749..8b9ae86f 100644 --- a/packages/node/src/Transloadit.ts +++ b/packages/node/src/Transloadit.ts @@ -605,6 +605,7 @@ export class Transloadit { const result = await this._remoteJson({ urlSuffix: `/dam/assets/${id}`, method: 'get', + signEndpoint: true, params, signal: options.signal, }) @@ -629,6 +630,7 @@ export class Transloadit { const result = await this._remoteJson({ urlSuffix: '/dam/assets', method: 'get', + signEndpoint: true, params, signal: options.signal, }) @@ -653,6 +655,7 @@ export class Transloadit { const result = await this._remoteJson({ urlSuffix: `/dam/assets/${id}`, method: 'patch', + signEndpoint: true, params, signal: options.signal, }) @@ -676,6 +679,7 @@ export class Transloadit { const result = await this._remoteJson({ urlSuffix: `/dam/assets/${id}`, method: 'delete', + signEndpoint: true, signal: options.signal, }) checkResult(result) @@ -1712,6 +1716,8 @@ export class Transloadit { timeout?: Delays method?: 'delete' | 'get' | 'patch' | 'post' | 'put' params?: TParams + /** Sign the request's method and path into `params.auth`, as Storage requests require. */ + signEndpoint?: boolean fields?: Fields headers?: Headers signal?: AbortSignal @@ -1722,7 +1728,8 @@ export class Transloadit { isTrustedUrl = false, timeout = { request: this._defaultTimeout }, method = 'get', - params = {}, + params: paramsInput = {}, + signEndpoint = false, fields, headers, signal, @@ -1739,6 +1746,14 @@ export class Transloadit { } } + const requested: OptionalAuthParams = paramsInput + const params: OptionalAuthParams = signEndpoint + ? { + ...requested, + auth: { ...requested.auth, method: method.toUpperCase(), path: new URL(url).pathname }, + } + : requested + if (method === 'get') { url = this._appendParamsToUrl(url, params) } diff --git a/packages/node/src/apiTypes.ts b/packages/node/src/apiTypes.ts index c57fbc6e..2fdf9653 100644 --- a/packages/node/src/apiTypes.ts +++ b/packages/node/src/apiTypes.ts @@ -10,7 +10,14 @@ export { export { assemblyInstructionsSchema } from './alphalib/types/template.ts' export interface OptionalAuthParams { - auth?: { key?: string; expires?: string } + auth?: { + key?: string + expires?: string + /** HTTP method of the request these params are signed for, in upper case. */ + method?: string + /** URL path of the request these params are signed for, without the query string. */ + path?: string + } } // todo make zod schemas for these types in the backend for these too (in alphalib?) diff --git a/packages/node/test/unit/storageAssets.test.ts b/packages/node/test/unit/storageAssets.test.ts index c407a262..a392b0c0 100644 --- a/packages/node/test/unit/storageAssets.test.ts +++ b/packages/node/test/unit/storageAssets.test.ts @@ -176,6 +176,85 @@ test('soft-deletes by identity and returns the deletion receipt', async () => { }) }) +/** The signed `auth` of a multipart Storage mutation. */ +function formAuth(body: unknown): unknown { + const params = /name="params"\r\n\r\n([^\r\n]+)/.exec(String(body))?.[1] + return params === undefined ? undefined : JSON.parse(params).auth +} + +test('signs every Storage request for its own method and path', async () => { + const signedFor: unknown[] = [] + const queryAuth = (query: Record) => { + signedFor.push(JSON.parse(String(query.params)).auth) + return true + } + const found = { ok: 'DAM_ASSET_FOUND', message: 'Found', asset } + nock(origin).get(`/dam/assets/${asset.asset_id}`).query(queryAuth).reply(200, found) + nock(origin).get('/dam/assets').query(queryAuth).reply(200, { + ok: 'DAM_ASSETS_LISTED', + message: 'Listed', + workspace: asset.workspace, + assets: [], + next_cursor: null, + }) + nock(origin) + .patch(`/dam/assets/${asset.asset_id}`, (body) => signedFor.push(formAuth(body)) > 0) + .reply(200, { ok: 'DAM_ASSET_MOVED', message: 'Moved', asset }) + nock(origin) + .delete(`/dam/assets/${asset.asset_id}`, (body) => signedFor.push(formAuth(body)) > 0) + .reply(200, { + ok: 'DAM_ASSET_DELETED', + message: 'Deleted', + asset_id: asset.asset_id, + deleted_at: '2026-09-17T12:00:00.000Z', + }) + + await client.getStoredAsset(asset.asset_id) + await client.listStoredAssets({ prefix: 'photos/' }) + await client.moveStoredAsset(asset.asset_id, { filename: 'renamed.jpg' }) + await client.deleteStoredAsset(asset.asset_id) + + const assetPath = `/dam/assets/${asset.asset_id}` + expect(signedFor).toEqual([ + expect.objectContaining({ key: 'key', method: 'GET', path: assetPath }), + expect.objectContaining({ key: 'key', method: 'GET', path: '/dam/assets' }), + expect.objectContaining({ key: 'key', method: 'PATCH', path: assetPath }), + expect.objectContaining({ key: 'key', method: 'DELETE', path: assetPath }), + ]) +}) + +test('signs the path the API receives when the endpoint has a base path', async () => { + const proxied = new Transloadit({ + authKey: 'key', + authSecret: 'secret', + endpoint: `${origin}/transloadit`, + }) + const request = nock(origin) + .get('/transloadit/dam/assets') + .query((query) => JSON.parse(String(query.params)).auth.path === '/transloadit/dam/assets') + .reply(200, { + ok: 'DAM_ASSETS_LISTED', + message: 'Listed', + workspace: asset.workspace, + assets: [], + next_cursor: null, + }) + await proxied.listStoredAssets() + expect(request.isDone()).toBe(true) +}) + +test('leaves the method and path out of other signed requests', async () => { + const request = nock(origin) + .get('/bill/2026-11') + .query((query) => { + const { auth } = JSON.parse(String(query.params)) + return auth.key === 'key' && !('method' in auth) && !('path' in auth) + }) + .reply(200, { ok: 'BILL_FOUND' }) + await client.getBill('2026-11') + expect(request.isDone()).toBe(true) +}) + test('rejects an empty move and identifiers before sending a mutation', async () => { await expect(client.moveStoredAsset(asset.asset_id, {})).rejects.toThrow() await expect(client.moveStoredAsset('../escape', { filename: 'photo.jpg' })).rejects.toThrow() diff --git a/packages/transloadit/package.json b/packages/transloadit/package.json index 89b4b5b3..bf5b0236 100644 --- a/packages/transloadit/package.json +++ b/packages/transloadit/package.json @@ -1,6 +1,6 @@ { "name": "transloadit", - "version": "6.7.0", + "version": "6.8.0", "description": "Node.js SDK and `transloadit` CLI for Transloadit", "homepage": "https://github.com/transloadit/node-sdk/tree/main/packages/node", "bugs": { diff --git a/yarn.lock b/yarn.lock index 592ba909..4fc8dd8e 100644 --- a/yarn.lock +++ b/yarn.lock @@ -1619,7 +1619,7 @@ __metadata: dependencies: "@modelcontextprotocol/sdk": "npm:^1.29.0" "@playwright/test": "npm:1.64.0" - "@transloadit/node": "npm:^6.7.0" + "@transloadit/node": "npm:^6.8.0" "@transloadit/sev-logger": "npm:^0.1.9" "@transloadit/zod": "npm:^7.5.0" "@types/express": "npm:^5.0.6" @@ -1636,7 +1636,7 @@ __metadata: languageName: unknown linkType: soft -"@transloadit/node@npm:^6.7.0, @transloadit/node@workspace:packages/node": +"@transloadit/node@npm:^6.8.0, @transloadit/node@workspace:packages/node": version: 0.0.0-use.local resolution: "@transloadit/node@workspace:packages/node" dependencies: