Skip to content

Represent %raw, %ffi and %%raw with parsetree nodes - #8879

Draft
cknitt wants to merge 3 commits into
masterfrom
raw-extension-nodes
Draft

cknitt wants to merge 3 commits into
masterfrom
raw-extension-nodes

Conversation

@cknitt

@cknitt cknitt commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Part of #8624 ("extension-keyed language features → nodes"). Builds on #8875 (merged).

Summary

%raw, %ffi and %%raw get dedicated parsetree nodes instead of being extensions with a string payload, and Pconst_raw_source is removed.

| Pexp_raw of {kind: raw_kind; name_loc: Location.t; source: raw_source}
| Pstr_raw of {name_loc: Location.t; source: raw_source; attrs: attributes}

and raw_kind = Raw_expression (* %raw *) | Raw_function (* %ffi *)

and raw_source =
  | Raw_string of string loc (* %raw("..."): the JavaScript source *)
  | Raw_template of string loc (* %raw(`...`): the template source as written *)
  • %raw and %ffi share one node: both lower to the same raw JavaScript, and %ffi only adds the function check and an arity-based type. The kind keeps them apart for printing, typing and the v0 bridge.
  • The source is kept as written, so the formatter output is unchanged. Ast_raw.js_source gives the JavaScript.
  • When a node is turned back into an extension (printer, comments table, v0 bridge), a string is printed as a template if its source contains an unescaped ", as the printer did before. Unlike before, the template escapes backquotes, ${ and backslashes where needed, so it parses back to the same JavaScript.
  • name_loc is the location of %raw / %ffi / %%raw, which cannot be derived from the node's location when attributes precede it.
  • Payloads the node can't represent (braced or attributed payloads, non-strings) stay extensions and are processed as before.
  • PPXs still see %raw/%ffi/%%raw extensions with the same v0 encoding, and Ast_mapper_from0 reads them back into the nodes.
  • The .cmi, .cmt and AST magic numbers are bumped.

Fix

Through a PPX, a template payload such as %raw(`"a\${b}"`) lost its template marker in the v0 bridge. Its escapes were then not decoded, and locations in error messages shifted by one character. The node keeps the template, so compiling with and without a PPX now gives the same result.

Braced string payloads such as %raw({"\"a\""}) were taken as JavaScript source through a PPX, and came back with a @res.braces attribute instead of braces. They are now mapped like any other expression, so they are decoded as ordinary strings with and without a PPX.

Tests

  • ast-mapping/RawExtensions.res covers strings, templates with escapes, braced strings, %ffi, attributes, doc comments and %%raw.
  • raw_output_test.res compiles braced %raw and %%raw strings.
  • ounit_ast_mapper0_tests: raw sources printed as templates parse back to the same JavaScript, and attributed templates survive the v0 bridge.
  • ounit_ast_mapper0_tests: raw nodes reach v0 as the extensions they are written as and come back unchanged, including a %%raw with attributes.

Compared with master (#8875), the formatter output and the v0 tree a PPX receives are identical for every .res/.resi file in the repository. Compiled output and errors are identical for about 200 %raw/%ffi/%%raw forms in many positions, including types, patterns and modules where they stay extensions. Through a PPX they differ only in the fixes above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NP47qMfjMbjUTKZDsLthDo

@cknitt
cknitt force-pushed the raw-extension-nodes branch from da8d79b to efc319d Compare October 16, 2026 14:45
Base automatically changed from fixed-external-args-parsetree to master October 16, 2026 14:49
`Pexp_raw` (with a `Raw_expression` / `Raw_function` kind) and `Pstr_raw` hold
the JavaScript source as written, so `Pconst_raw_source` is gone. PPXs still see
the extensions, and v0 templates now keep their decoding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NP47qMfjMbjUTKZDsLthDo
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt
cknitt force-pushed the raw-extension-nodes branch from efc319d to 88b4db6 Compare October 16, 2026 14:56
@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.83513% with 73 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.98%. Comparing base (735a485) to head (36f11f8).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
compiler/ml/printast.ml 0.00% 9 Missing ⚠️
compiler/syntax/src/res_parens.ml 0.00% 8 Missing ⚠️
tests/ounit_tests/ounit_ast_mapper0_tests.ml 87.50% 8 Missing ⚠️
compiler/frontend/ast_exp_handle_external.ml 86.11% 5 Missing ⚠️
compiler/ml/ast_iterator.ml 0.00% 5 Missing ⚠️
compiler/ml/typecore.ml 0.00% 5 Missing ⚠️
compiler/syntax/src/res_comments_table.ml 0.00% 5 Missing ⚠️
compiler/ml/ast_mapper_from0.ml 82.60% 4 Missing ⚠️
compiler/ml/ast_payload.ml 55.55% 4 Missing ⚠️
compiler/ml/ast_raw.ml 91.66% 3 Missing ⚠️
... and 11 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8879      +/-   ##
==========================================
- Coverage   81.01%   80.98%   -0.03%     
==========================================
  Files         454      455       +1     
  Lines       63187    63390     +203     
==========================================
+ Hits        51193    51339     +146     
- Misses      11994    12051      +57     
Files with missing lines Coverage Δ
analysis/src/dump_ast.ml 44.01% <ø> (+0.18%) ⬆️
compiler/ext/config.ml 100.00% <ø> (ø)
compiler/frontend/bs_builtin_ppx.ml 90.61% <100.00%> (+0.12%) ⬆️
compiler/ml/ast_mapper.ml 98.45% <100.00%> (+0.03%) ⬆️
compiler/ml/ast_mapper_to0.ml 77.75% <100.00%> (+0.46%) ⬆️
compiler/syntax/src/res_core.ml 91.99% <100.00%> (+<0.01%) ⬆️
compiler/syntax/src/res_parsetree_viewer.ml 94.42% <ø> (-0.05%) ⬇️
compiler/syntax/src/res_printer.ml 94.03% <100.00%> (+0.10%) ⬆️
analysis/reanalyze/src/annotation.ml 52.63% <0.00%> (ø)
analysis/src/document_symbol.ml 56.09% <0.00%> (ø)
... and 19 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cknitt
cknitt added this pull request to stack #8880 October 16, 2026 15:02
@cknitt

cknitt commented Oct 16, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T19:19:18.581415Z 36f11f8 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 88b4db6f91

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread compiler/ml/ast_payload.ml
@pkg-pr-new

pkg-pr-new Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@8879

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@8879

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@8879

@rescript/darwin-x64

npm i https://pkg.pr.new/@rescript/darwin-x64@8879

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@8879

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@8879

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@8879

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@8879

commit: 36f11f8

@github-actions

Copy link
Copy Markdown

A braced string payload is an ordinary string, with or without a PPX.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NP47qMfjMbjUTKZDsLthDo
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt

cknitt commented Oct 16, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: caa10c51f3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread compiler/ml/ast_mapper_from0.ml
A source that needs a template now escapes backquotes, ${ and backslashes,
and attributed templates stay templates through the v0 bridge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NP47qMfjMbjUTKZDsLthDo
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt

cknitt commented Oct 16, 2026

Copy link
Copy Markdown
Member Author

@codex review

@github-actions

Copy link
Copy Markdown

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 36f11f847a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant