Skip to content

perf: exact rectangle clip for valid polygons (planet 25% faster, 31% less CPU) - #949

Merged
systemed merged 2 commits into
systemed:masterfrom
dktue:perf/rect-clip
Oct 31, 2026
Merged

systemed merged 2 commits into
systemed:masterfrom
dktue:perf/rect-clip

Conversation

@dktue

@dktue dktue commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Tile writing spent most of its CPU in the slow polygon clipping fallback. fast_clip (SutherlandÔÇôHodgman) leaves zero-width bridges along the tile edge for concave polygons, Boost then reports the result as invalid, and tilemaker redoes the clip with geom::intersection plus a repair. One landcover relation (1,353 holes) took 3ÔÇô9 s in every tile it touched, causing a ~10 s single-threaded tail on Germany.

  • Exact rectangle clipper (rect_clip, src/geom.cpp): WeilerÔÇôAtherton specialised to an axis-aligned box. Used when the input polygon is known to be valid; its output is still validity-checked, and anything it cannot handle falls back to the existing path.
  • Validity flag for stored multipolygons: relations that are valid (or were corrected) at import are not re-validated in every tile.
  • Repaired clips are cached as valid when the repair proves them valid, so child tiles take the fast path.
  • New unit test test/geom.test.cpp, added to make test.

Results (openmaptiles profile, 32 threads)

master this PR
Germany wall (3 runs each) 62.062.2 s 52.152.2 s 16%
Germany CPU ~1,404 s ~1,155 s 18%
Planet wall (--store) 65:43 49:07 25%
Planet CPU 102,198 s 70,962 s 31%
Planet peak RSS 170.4 GB 169.6 GB

Output

  • Planet: identical tile counts per zoom. In 200k random tiles, 90.7% are byte-identical, 7.5% contain the same features reordered, and 1.8% differ in content, with equal feature counts per layer except +2 water.
  • Germany semantic comparison (area per layer/tags): about 760 polygon groups gain area. These are lakes, residential areas and forests that master drops because Boost's intersection fails when a vertex lies exactly on the clip edge. There are no real losses beyond tiny z11/z12 simplification differences.
  • 473k real clips were checked against the SutherlandÔÇôHodgman area (an exact reference): 0 mismatches.

Depends on #933 or #941 (scaleRing drops rings that start with a duplicated vertex). This PR sends more polygons through scaleRing unchanged, which exposes that bug more often: without the fix, about 64 polygon groups are lost in Germany. With #933 applied, those losses disappear.

Update: rings touching the box edge

A ring that touches the box edge at one vertex, with the polygon on both
sides of it, came out of rect_clip touching itself. Boost rejected that,
and the Boost fallback then lost most or all of the polygon. rect_clip now
splits the chain there, so the result is two polygons meeting at the vertex.

Germany, simplification off, compared with this PR before the fix (both with
#941's scaleRing fix): 72 tile/layer groups gain area, none lose any. They are
features that were missing or mostly missing at z14, e.g. a whole park in
14/8552/11040 and a lake in 14/8846/11110. Speed is unchanged.

In a randomised test (95k valid polygons on an integer grid, so vertices often
sit exactly on the box edges), invalid rect_clip results fall from 1,949 to
1,045; the clipped area is exact in every case, before and after.

daniel.korn and others added 2 commits October 29, 2026 09:17
A ring can touch the box edge at a single vertex, with the polygon on
both sides of that vertex along the edge. rect_clip kept it as one
chain, so the walk along the box edge ran through the vertex and the
result was a ring touching itself. Boost rejects that as invalid, and
the Boost intersection fallback then lost most or all of the polygon.

Split the chain at such a vertex, so it is both an exit and an entry
and the result is two polygons meeting there. Touches where the
polygon lies between the two arms (a wedge) are left as they were.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
geoneutrino added a commit to geoneutrino/tilemaker that referenced this pull request Oct 30, 2026
Large clipped polygons such as coastline/ocean shapefiles are written to tiles invalid: scaling them onto the integer tile grid introduces self-intersections. Until now only SIMPLIFIED geometry was repaired, so those layers stayed invalid and can be mis-rendered by strict renderers - as missing water, or as land hidden behind an ocean polygon.

New layer option invalid_polygon_repair_scope: "simplified_only" (default, unchanged behaviour) repairs only invalid polygons that simplification may have broken; "all" is a superset and additionally repairs polygons that were merely clipped and quantised. For those, repair_multi_polygon runs with a strict area guard that also rejects any repair GROWING a polygon - growing means a hole was filled, which would make an island disappear. That bound must not apply to simplified geometry, where legitimate repairs do change area; applying it globally measurably made things worse (5 -> 20 invalid features in one tile).

The bound is passed as a scoped enum, AreaBound, rather than a bool: other changes also append parameters to repair_multi_polygon (systemed#939 a bool, systemed#949 a bool*), and with a bool here a merge that puts the parameters in the wrong order would still compile, silently.

Verified: with the default the output is unchanged (Bangladesh z7 96/55: 166 features, 5 invalid, 319 holes). With "all" on the ocean layers, that tile goes to 4 invalid / 318 holes and the ocean feature behind a reported rendering artefact becomes valid; Denmark z5 16/10 improves 5 -> 1 invalid while Funen correctly stays land; Philippines z5 26/14 improves 7 -> 1 invalid with no land wrongly covered by ocean.
@systemed
systemed merged commit 2bd12b6 into systemed:master Oct 31, 2026
11 checks passed
@dktue

dktue commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Thanks a lot for merging this, @systemed!

One thing to flag: as mentioned in the description, this PR depends on the scaleRing fix from @TimSC's #941. #949 sends more polygons through scaleRing unchanged, and without the fix some of them lose rings. On Germany (OpenMapTiles profile, master before vs after this PR), whole landcover and park polygons now drop out of some z14 tiles; one park loses 80% of its area in a tile. With #941's scaleRing part applied as well, those z14 losses are gone, and the lost area is about 30× smaller (only the usual tiny z10–12 simplification differences remain).

The scaleRing part of #941 (src/coordinates_geom.cpp) applies cleanly to current master. It also brings back about 13,500 z14 polygon groups on Germany that master has been dropping all along (lakes, parks and landuse whose ring starts with a duplicated vertex). If merging #941 as a whole is difficult, I'm happy to open a small PR with just that part, credited to @TimSC.

cldellow pushed a commit to cldellow/tilemaker that referenced this pull request Nov 1, 2026
…gons twice

systemed#949 made profiles with many huge polygons 52% slower: rect_clip ran the
Liang-Barsky computation for every segment of the whole polygon on every
tile, and the polygon was loaded from the store a second time because
fast_clip had modified the first copy.

- clipRing rejects a segment whose end points both lie beyond the same
  box edge with four comparisons, before the clip computation
- a valid polygon over 1,000 points is kept for rect_clip; fast_clip works
  on a copy instead of the moved original

parks profile, Alaska/BC/Yukon, 16 threads: 275.6 -> 210.2 s (-24%),
CPU -26%. Output identical (parks extract, Germany OpenMapTiles).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c2a0fcf)
geoneutrino added a commit to geoneutrino/tilemaker that referenced this pull request Nov 2, 2026
Large clipped polygons such as coastline/ocean shapefiles are written to tiles invalid: scaling them onto the integer tile grid introduces self-intersections. Until now only SIMPLIFIED geometry was repaired, so those layers stayed invalid and can be mis-rendered by strict renderers - as missing water, or as land hidden behind an ocean polygon.

New layer option invalid_polygon_repair_scope: "simplified_only" (default, unchanged behaviour) repairs only invalid polygons that simplification may have broken; "all" is a superset and additionally repairs polygons that were merely clipped and quantised. For those, repair_multi_polygon runs with a strict area guard that also rejects any repair GROWING a polygon - growing means a hole was filled, which would make an island disappear. That bound must not apply to simplified geometry, where legitimate repairs do change area; applying it globally measurably made things worse (5 -> 20 invalid features in one tile).

The bound is passed as a scoped enum, AreaBound, rather than a bool: other changes also append parameters to repair_multi_polygon (systemed#939 a bool, systemed#949 a bool*), and with a bool here a merge that puts the parameters in the wrong order would still compile, silently.

Verified: with the default the output is unchanged (Bangladesh z7 96/55: 166 features, 5 invalid, 319 holes). With "all" on the ocean layers, that tile goes to 4 invalid / 318 holes and the ocean feature behind a reported rendering artefact becomes valid; Denmark z5 16/10 improves 5 -> 1 invalid while Funen correctly stays land; Philippines z5 26/14 improves 7 -> 1 invalid with no land wrongly covered by ocean.
geoneutrino added a commit to geoneutrino/tilemaker that referenced this pull request Nov 2, 2026
Large clipped polygons such as coastline/ocean shapefiles are written to tiles invalid: scaling them onto the integer tile grid introduces self-intersections. Until now only SIMPLIFIED geometry was repaired, so those layers stayed invalid and can be mis-rendered by strict renderers - as missing water, or as land hidden behind an ocean polygon.

New layer option invalid_polygon_repair_scope: "simplified_only" (default, unchanged behaviour) repairs only invalid polygons that simplification may have broken; "all" is a superset and additionally repairs polygons that were merely clipped and quantised. For those, repair_multi_polygon runs with a strict area guard that also rejects any repair GROWING a polygon - growing means a hole was filled, which would make an island disappear. That bound must not apply to simplified geometry, where legitimate repairs do change area; applying it globally measurably made things worse (5 -> 20 invalid features in one tile).

The bound is passed as a scoped enum, AreaBound, rather than a bool: other changes also append parameters to repair_multi_polygon (systemed#939 a bool, systemed#949 a bool*), and with a bool here a merge that puts the parameters in the wrong order would still compile, silently.

Verified: with the default the output is unchanged (Bangladesh z7 96/55: 166 features, 5 invalid, 319 holes). With "all" on the ocean layers, that tile goes to 4 invalid / 318 holes and the ocean feature behind a reported rendering artefact becomes valid; Denmark z5 16/10 improves 5 -> 1 invalid while Funen correctly stays land; Philippines z5 26/14 improves 7 -> 1 invalid with no land wrongly covered by ocean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants