[ybai08]

This fixes #192.

With the header-ids extra, a markdown header's id is slugified from the raw markdown source. Link URLs, image paths and emphasis markers therefore end up in the id:

Header	Before	After
# [Inline link](https://github.com)	inline-linkhttpsgithubcom	inline-link
# [Reference link][ref]	reference-linkref	reference-link
#  Image	logologopng-image	image
# <span>Inline</span> HTML	spaninlinespan-html	inline-html
# __Strong__ and _em_	__strong__-and-_em_	strong-and-em

_h_sub now runs the span gamut first and builds the id from the rendered header, and header_id_from_text strips tags before slugifying. Its docstring already says it receives the header's HTML content, and the mixed-mode HTML headers (_h_tag_sub) already pass HTML, so both paths now treat tags the same way. The safe mode handling from #739 is kept: HTML spans are unhashed (to their sanitized form) before the id is built, so the existing header_ids_safe_mode* cases are unchanged. In escape mode the escaped tags are visible text, so they still count towards the id there (hello-bworldb).

A footnote reference in a header is still a placeholder at this point (numbers are assigned after the block gamut), so the placeholder is dropped from the id text. Without that, the placeholder ended up in the id attribute and _do_footnote_marker later failed with a ValueError. Footnoted headers keep the id they had before (# Header with a footnote[^1] -> header-with-a-footnote1).

Behaviour changes worth knowing about:

• ids change for headers that contain links, images, inline HTML or _/__ emphasis, so existing anchors to such headers move. Headers without inline markup keep their ids. Over the 289 .text inputs under test/ (rendered with header-ids, footnotes, tables, fenced-code-blocks), only the new cases and the safe mode inputs changed id, and the safe mode inputs only change when rendered without safe mode.
• outside safe mode, a header whose only content is an HTML tag (# <sometext>) has no visible text, so it now gets the same fallback id as other headers with no word characters (-1), instead of sometext.
• subclasses that override header_id_from_text now receive the rendered header HTML for markdown headers rather than the markdown source.

Tests: two new tm-cases, header_ids_issue192 (the headers above, plus a code span that keeps its underscores) and header_ids_footnote_ref (footnote refs in headers, which covers the ValueError above). With lib/markdown2.py reverted to master both new cases fail on the ids; the rest of the suite passes either way.

The suite was run: make testone equivalent (cd test && python test.py -- -knownfailure, 298 tests) and python test/test_redos.py (10 tests) on Python 3.12 and 3.9 with pygments, wavedrom, latex2mathml and emoji installed, all passing. pycodestyle --diff on the change reports only the two existing E128 lines of the header_id_from_text signature. Not run: the other Python versions and the Windows and Linux jobs in the CI matrix.

[Crozzers]

Have you measured perf on running span gamut on each header?

Plus this would break any existing hrefs if they were recompiled with this code

[ybai08]

Perf: the span gamut doesn't run an extra time. _h_sub already ran _run_span_gamut(header_group) once for the header's HTML; this moves that call above the id computation and builds the id from its result, so the only new work per header is one tag-stripping regex. Timing markdown() on master vs this branch (best of 7 × 5 runs, CPython 3.9.6, macOS arm64):

document	extras	master	branch
1000 headers with links/emphasis/code + paragraphs	header-ids	101.1 ms	100.9 ms
same	header-ids, toc	113.9 ms	114.3 ms
1000 plain headers	header-ids	22.2 ms	22.4 ms
same	header-ids, toc	23.7 ms	23.8 ms

Existing hrefs: ids stay the same for plain headers and for emphasis, code spans and entities (## *Emphasis* here → emphasis-here, ## Use `code` → use-code, ## Foo & Bar → foo-bar on both). They change only where the markdown source leaked into the id, i.e. headers with links or images: ## [Docs](http://x.io/a) goes from docshttpxioa to docs, and ## ![img](a.png) Logo from imgapng-logo to logo. So existing anchors to those headers would break when re-rendered.

If that's a concern, the new behaviour could sit behind a header-ids option (e.g. {"header-ids": {"from-rendered": True}}) with the current ids as the default. Happy to change it that way, or to close this if you'd rather keep ids as they are.

[nicholasserra]

@Crozzers Tell me more on your backwards compat concern. You're worried that someone who previously rendered header IDs and then put them online and shared links would have invalid links if they went and re-rendered this and published the page again?