Skip to content

Compose canonical decompositions (NFC) in the default shaper - #380

Closed
lehni wants to merge 1 commit into
foliojs:masterfrom
lineto:feature/unicode-normalization
Closed

lehni wants to merge 1 commit into
foliojs:masterfrom
lineto:feature/unicode-normalization

Conversation

@lehni

@lehni lehni commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Problem

HarfBuzz normalizes each base + combining-mark cluster before GSUB/GPOS: the marks are reordered by combining class and composed onto the base when the font has a precomposed glyph. The default shaper does no such step, so decomposed input shapes as separate glyphs instead of the precomposed glyph HarfBuzz and browsers produce — e.g. "i" + U+0300 → [i, gravecomb] instead of [igrave]; Arabic alef + fathatan + hamza-above → three glyphs instead of [alef-with-hamza, fathatan] (the hamza composes onto the alef across the lower-class fathatan).

Fix

Apply font-aware NFC per cluster at the start of the default shaper's feature assignment, before GSUB:

  • Reorder + compose via normalize('NFC'), but only when the font has a glyph for the precomposed codepoint; otherwise leave the marks decomposed for GPOS mark positioning.
  • Only apply when the result changes the glyph count (composition / decompose fallback) — pure canonical reordering is left alone so it can't disturb the order downstream GSUB expects (e.g. Arabic shadda + vowel calt).

Scoped to the default shaper and the Arabic/Hebrew/Thai shapers that inherit its assignFeatures; the Indic/Hangul/Universal shapers keep their own composition.

Tests

Adds 6 shaping tests (FiraSans Latin + Amiri Arabic), including a regression test that a non-composing mark cluster is left unreordered.

- Compose each base + combining-mark cluster into the font's precomposed glyph via font-aware NFC before GSUB, matching HarfBuzz: decomposed input (i + U+0300, or Arabic alef + fathatan + hamza-above) shapes to the precomposed glyph instead of separate marks
- Only apply the result when it changes the glyph count (composition or its decompose fallback); leave pure canonical reordering alone so it can't disturb downstream GSUB (e.g. Arabic shadda + vowel calt)
- Decompose back any composed codepoint the font has no glyph for, so its marks stay available for GPOS mark positioning
- Scope to the default shaper and the Arabic/Hebrew/Thai shapers that inherit it; Indic/Hangul/Universal keep their own composition
- Add 6 canonical-composition shaping tests (FiraSans Latin, Amiri Arabic incl. the reorder regression)
@blikblum

Copy link
Copy Markdown
Member

I reviewed with Astra and suggested some things. Please look at #403 (seems i cannot update this PR because it was open when i was not a maintainer)

@lehni

lehni commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, looks good to me, and thanks for catching the missing glyph sync without GSUB. Closing this in favor of #403.

One thing for later: clusters that don't compose keep their typed mark order (e.g. q + U+0307 + U+0323), while HarfBuzz always reorders canonically and gets Amiri's shadda case right by giving Arabic marks adjusted classes. Happy to follow up with that separately.

@blikblum

Copy link
Copy Markdown
Member

Thanks, looks good to me, and thanks for catching the missing glyph sync without GSUB. Closing this in favor of #403.

One thing for later: clusters that don't compose keep their typed mark order (e.g. q + U+0307 + U+0323), while HarfBuzz always reorders canonically and gets Amiri's shadda case right by giving Arabic marks adjusted classes. Happy to follow up with that separately.

This behavior is observed on the wild or is just hypothetical?

Having use case with actual fonts increases the chance of accepting the changes

@lehni

lehni commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

It's observed in real text: vocalized Arabic is often typed with the vowel before shadda. With Amiri, Allah typed as alef, lam, lam, fatha, shadda, heh: HarfBuzz reorders shadda first and gets the Allah ligature and the small fatha (uni0647.fina_Lellah uni064E.small uni0651 …), while fontkit keeps the typed order and loses both (uni0647.fina_LamHeh uni0651 uni064E …). The same happens for beh, meem and reh with fatha, damma or kasra before shadda. #404 fixes these; I've rebased it on #403 and added the Allah case as a test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants