Skip to content

Do not fetch remote JSON-LD contexts - #580

Open
Vitorinox wants to merge 1 commit into
php-embed:masterfrom
Vitorinox:fix/jsonld-remote-context
Open

Vitorinox wants to merge 1 commit into
php-embed:masterfrom
Vitorinox:fix/jsonld-remote-context

Conversation

@Vitorinox

Copy link
Copy Markdown
Collaborator

Summary

JSON-LD expansion loaded a page-supplied @context with file_get_contents (FileGetContentsLoader). That request never went through the crawler or UrlPolicy, and it followed redirects, so a context URL was an SSRF path left open by #573 / #579. Extraction also depended on the network for ordinary schema.org contexts.

LinkedData now passes an explicit documentLoader into each ML\JsonLD call. The global JsonLD::setDefaultDocumentLoader() is not changed, so host applications keep their own loader.

The loader performs no network or filesystem I/O:

  • Inline JSON (the document already extracted from the page) is parsed in memory.
  • http/https contexts on schema.org, www.schema.org, and schema.googleapis.com — with or without a trailing slash, .jsonld, or docs/jsonldcontext.json — are answered with a bundled {"@context":{"@vocab":"http://schema.org/"}}.
  • Every other remote URL or path raises JsonLdException. LinkedData already catches that, so extraction continues with the rest of the page.

Existing fixtures are unchanged, including Tumblr publishedTime of 2017-09-14 10:20:35. The test-only http/https stream wrapper added in #579 (tests/SchemaOrgContext.php) is removed; the suite passes without it.

Tests

  • A page whose @context points at http://127.0.0.1/... on a local php -S server that records hits makes no request. The same URL is followed (including the redirect) by FileGetContentsLoader.
  • schema.org, www.schema.org, and schema.googleapis.com context URL variants still resolve datePublished.
  • An unknown host, a non-context schema.org path, a malformed script, and an invalid @context leave the HTML title intact and do not fatal.

PHPUnit passes on PHP 7.4, 8.3, 8.4, and 8.5. PHPStan (level max, strict rules) is clean.

Page-supplied @context URLs were loaded with file_get_contents, which
bypasses the crawler and the URL policy and follows redirects. Pass an
offline document loader that serves a bundled schema.org vocabulary
and refuses every other remote document.

@Vitorinox Vitorinox left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markfullmer Since you reviewed #579, would you mind taking a look at this one? It closes the remaining SSRF path from #573: JSON-LD @context URLs were fetched with file_get_contents, bypassing the crawler and UrlPolicy.

cc @karthi-the-hacker, as the reporter of #573. Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant