Skip to content

perf: parallelise z6 finalizing and tile collection - #972

Open
dktue wants to merge 1 commit into
systemed:masterfrom
dktue:perf/serial-steps
Open

dktue wants to merge 1 commit into
systemed:masterfrom
dktue:perf/serial-steps

Conversation

@dktue

@dktue dktue commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Gain (planet, PMTiles, 32 threads): the two serial steps between reading and tile writing drop from 70.5 s to 24.2 s (−66%; −46 s, about 2.7% of the run): collecting tiles 25.4 → 1.1 s (−96%), z6 finalizing 26.3 → 7.3 s (−72%). Total wall 1703 → 1686 s (−1.0%, one run each; the untouched tile phase varies by ±25 s between runs). Output unchanged.

Between reading the .pbf and writing tiles, two steps run on few threads. On the planet (32 threads) they take about 50 s, with most cores idle:

  • finalizing z6 tiles (~23 s at 5–9 busy threads): finalizeObjects sorts the objects of the 4096 z6 tiles one after another. Each sort uses all threads, which suits small extracts (one z6 tile may hold most objects) but leaves the cores idle for the many small tiles of a large extract. A comment in sortOutputObjects already names this as a future improvement.
  • collecting tiles (~28 s on 1 thread): populateTilesAtZoom walks every output object to mark the tiles that hold objects, then marks every z14 tile in the bounding box of each large object (e.g. ocean polygons) together with its 14 parents, one tile at a time.

This PR changes three things.

  • finalizeObjects: z6 tiles holding more than 1/threads of all objects are still sorted one at a time with all threads. The others are sorted in parallel, one thread each, largest first. Each sort copies its z6 tile into a buffer, so the number of objects being sorted at once is capped at 1/8 of all objects (a single tile may always run). The counting sort is stable whatever its chunk count, so the order of objects, and the output, are unchanged.
  • populateTilesAtZoom takes threadNum. Workers take whole columns of z6 tiles, mark tiles in bitmaps of their own (allocated lazily, so each touches only its part), and the bitmaps are merged at the end with the new PreciseTileCoordinatesSet::merge.
  • Large objects: the parents of the tiles in a box are the tiles in the box scaled down, so addCoveredTilesToOutput now calls the new TileCoordinatesSet::setBox once per zoom. PreciseTileCoordinatesSet::setBox fills each row word by word. It ignores the part of a box outside the grid; set() wrapped a y beyond the last row into the next row.

New unit tests check setBox against set() for boxes that start and end mid-word, span several words or touch the edge, and merge against setting both sets' tiles.

Europe, PMTiles, 32 threads, --store --fast

before after
collecting tiles 3.4 s 0.6 s (−82%)
finalizing z6 tiles 7.3 s 4.8 s (−34%)
wall 425.9 s 417.4 s (−2.0%)

Planet, PMTiles, 32 threads, --store --fast

before after
collecting tiles 25.4 s 1.1 s (−96%)
finalizing z6 tiles 26.3 s 7.3 s (−72%)
end of reading → first tile written 70.5 s 24.2 s (−66%, −46 s)
wall 1703 s 1686 s (−17 s, −1.0%)
anon memory peak 13.76 GB 13.73 GB

The two steps save 46 s. The total gains less because the tile phase of this run took 24 s longer (1046 s vs 1022 s). This PR doesn't touch the tile phase, and identical planet runs have varied by tens of seconds before, so I take that as run-to-run noise (one run each).

Output

Germany, compared with compare_area (per-tile area/length per layer and tag group): all layers identical. Only poi differs, which already differs between identical runs.

Baseline for the measurements: master + #961 + #966 + #968 + the precheck and malloc follow-ups.

🤖 Generated with Claude Code

- finalizeObjects: sort small z6 tiles in parallel (one thread each,
  largest first, at most 1/8 of all objects in flight); z6 tiles with
  more than 1/threads of the objects keep the all-threads sort
- populateTilesAtZoom(threadNum): workers mark tiles per z6 column in
  bitmaps of their own, merged at the end (PreciseTileCoordinatesSet::merge)
- addCoveredTilesToOutput: one setBox per zoom instead of setting each
  z14 tile and its parents; setBox fills rows word by word

Planet: collecting tiles 25.4 s -> 1.1 s (-96%), z6 finalizing 26.3 s ->
7.3 s (-72%), wall -1.0%. Europe: collecting tiles 3.4 s -> 0.6 s (-82%),
z6 finalizing 7.3 s -> 4.8 s (-34%), wall -2.0%.
Output unchanged (Germany compare_area).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant