You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Incorrect render for a large multipolygon lake #191
As you can see, it is very complex area. The picture here is based on Geofabrik's latest Finnish file and is drawn here without any simplification. Are there any tricks that I could try?
This is made with the latest build from master branch. I'm wondering if this is related somehow to #181 although I understood that there should be a workaround for that in the latest commits.
Ugh, that's insanely complex. Unfortunately something that big is going to be near-impossible to reduce to a testcase.
One possible reason is that Tilemaker can't cope with non-closed inners, and there are several in here (e.g. https://www.openstreetmap.org/way/868890259). A quick hack to ignore them might be adding a check around line 247 of osm_store.h, viz:
Ring inner;
if (!ways.isClosed(*it)) { continue; } /* <--- new */
if (ways.count(*it)==1) { fillPoints(inner, ways.at(*it)); }
I've just pushed 12cb742 which will piece together multiple ways to form an inner. It seems to fix both of the above examples on a quick test, though I've not looked in great detail.
There still appears to be an issue at z7/8 where Sääminginsalo disappears - unsure what's going on there but it's possibly a simplification issue.
I've used Osmium to pick only water areas osmium tags-filter finland-latest.osm.pbf a/natural=water --overwrite --verbose -o fin_water.osm.pbf but I guess it shouldn't otherwise affect the result.
The big area is 7.38/62.059/28.879 and the origin of spikes is 9.2/61.0754/28.0492. I use tileserver-gl-light to visualize the raw data of the mbtiles file. I processed these with the original Lua file in resources folder but removed the simplification for water totally and set min zoom level to be four for all.
This can also be a problem with MapBox, I'll try to re-enable the simplification and setting a proper min zoom level. Probably the tile contains too much information.
I don't see any issues at z10, but at z9 various islands start getting dropped. I certainly don't see any renderings like the above, but different clients may of course treat invalid geometries in different ways.
I think this is going to need a bit of time to track down, creating a minimal test case (i.e. one that doesn't take hours to process!) and that's not time I have at the moment. But I'll keep this open for future reference.
Places to look (for my reference):
Lake Saimaa northwest of Imatra - islands disappear at z8
south of Puumala - islands disappear at z9
around Savonlinna - islands disappear and massive misrendering at z8
Yes, this is unfortunately very time consuming. My tiles are still being processed.
I noticed yesterday that some lakes disappeared too quickly and realized that there might be a problem with setting the minimum zoom. The values (meters per pixel) apply for the equator but not for north: https://docs.mapbox.com/help/glossary/zoom-level/ And I'm not sure if they are one level off or not. That was the reason I disabled that part of the Lua code but didn't have more time yet to continue that research.
A similar issue occurs for another insanely complex lake Inari (https://www.openstreetmap.org/relation/402543) at zoom 6. In this case even without any extra simplification parameters (no "simplify_level", no "simplify_ratio") for the water layer in config.json.
I've checked this MP in JOSM using the validation options, and it comes out fine. So the original MP appears technically sound.
Can you guys run tilemaker with --verbose? if tilemaker thinks the input relation is invalid, it will give an output message. If it gives no output message on this relation, in that case, the input geometry is valid, but tilemaker makes the geometry invalid during processing.
For the whole Finland extract tilemaker warns about 27 relations that either have intersections or intersecting interiors or interior rings outside. It includes cases of Saimaa (7379046) and Inari (402543), both have self-intersections in the tilemaker's opinion, but it doesn't mention the smaller lake Suvasvesi (900367), which I think also has this incorrect simplification issue.
It is quite possible these issues are caused by the fact coordinates get scaled when writing the pbf output. The floating point coordinates are scaled back to integers and they may cause issues with the polygons.
I haven't checked the situation now after my last comment as using Tippecanoe for the process, but would be interesting to try again and see if the issue is still visible.
On balance it’s much better than in 2020 when the issue was opened. There have been a couple of fixes which have greatly improved multipolygon handling. I would say though that it’s not 100% bulletproof - if you try and simplify complex geometries a lot for lower zoom levels, it’ll still occasionally trip up on edge cases. I use Natural Earth data for the very lowest zoom levels instead of OSM, and that works really well.
After a very long time, I've been working with creating tiles again. I can confirm that the situation is much much better nowadays. When there is no simplification used at all, there are not any incorrectly drawn water areas in Finland.
If simplification is used, a very small number of some drawing issues like this will appear:
With value "simplify_level": 0.0003 the issue is minimal but increasing the number seems to also increase the amount of problems. But even with 0.0013 there is not much them. However, it seems to affect only certain zoom levels for some areas, not all zoom levels below a certain value. In most cases the affected area was very small and it took some time even to spot them.
I retested with current master (4e0f03d), today's Finland extract and the water layer config from April (simplify_below 11, simplify_level 0.0005, filter_below 10), counting lake polygons that are invalid in tile coordinates:
Activity
ttsirkia commentedon Dec 7, 2020
This is made with the latest build from master branch. I'm wondering if this is related somehow to #181 although I understood that there should be a workaround for that in the latest commits.
systemed commentedon Dec 8, 2020
Ugh, that's insanely complex. Unfortunately something that big is going to be near-impossible to reduce to a testcase.
One possible reason is that Tilemaker can't cope with non-closed inners, and there are several in here (e.g. https://www.openstreetmap.org/way/868890259). A quick hack to ignore them might be adding a check around line 247 of osm_store.h, viz:
ttsirkia commentedon Dec 8, 2020
Thanks! I'll try that later today.
ttsirkia commentedon Dec 8, 2020
I found another position in which the lake polygon is not that complex:
https://www.openstreetmap.org/relation/409844#map=14/62.2353/28.8916
systemed commentedon Dec 8, 2020
Yes, that does look like an issue with non-closed inners (the break at the top island is exactly where that would happen).
systemed commentedon Dec 8, 2020
I've just pushed 12cb742 which will piece together multiple ways to form an inner. It seems to fix both of the above examples on a quick test, though I've not looked in great detail.
There still appears to be an issue at z7/8 where Sääminginsalo disappears - unsure what's going on there but it's possibly a simplification issue.
ttsirkia commentedon Dec 8, 2020
Oh, that's cool! I'll compile the new version, recreate the tiles and see how it behaves. Thank you so much!
ttsirkia commentedon Dec 8, 2020
Hmm, here it looks like this:
and
I've used Osmium to pick only water areas
osmium tags-filter finland-latest.osm.pbf a/natural=water --overwrite --verbose -o fin_water.osm.pbfbut I guess it shouldn't otherwise affect the result.systemed commentedon Dec 8, 2020
Can you tell me where those are? And what zoom levels?
ttsirkia commentedon Dec 8, 2020
The big area is 7.38/62.059/28.879 and the origin of spikes is 9.2/61.0754/28.0492. I use
tileserver-gl-lightto visualize the raw data of the mbtiles file. I processed these with the original Lua file inresourcesfolder but removed the simplification forwatertotally and set min zoom level to be four for all.ttsirkia commentedon Dec 8, 2020
Strange, the spikes will disappear in zoom level 9 after MapBox has redrawn the map. They come back when zooming to 8.
ttsirkia commentedon Dec 8, 2020
This can also be a problem with MapBox, I'll try to re-enable the simplification and setting a proper min zoom level. Probably the tile contains too much information.
systemed commentedon Dec 8, 2020
I don't see any issues at z10, but at z9 various islands start getting dropped. I certainly don't see any renderings like the above, but different clients may of course treat invalid geometries in different ways.
I think this is going to need a bit of time to track down, creating a minimal test case (i.e. one that doesn't take hours to process!) and that's not time I have at the moment. But I'll keep this open for future reference.
Places to look (for my reference):
ttsirkia commentedon Dec 8, 2020
Yes, this is unfortunately very time consuming. My tiles are still being processed.
I noticed yesterday that some lakes disappeared too quickly and realized that there might be a problem with setting the minimum zoom. The values (meters per pixel) apply for the equator but not for north: https://docs.mapbox.com/help/glossary/zoom-level/ And I'm not sure if they are one level off or not. That was the reason I disabled that part of the Lua code but didn't have more time yet to continue that research.
systemed commentedon Dec 8, 2020
Out of interest, do you know what Boost version you have?
32 remaining items
arturbil commentedon Nov 26, 2021
Found another problematic lake (https://www.openstreetmap.org/relation/900367), still complex, but 40x smaller in size than Saimaa. Attaching geojson if that can be of any use:
suvasvesi.geojson.zip
mboeringa commentedon Nov 26, 2021
The Saimaa MP as both OSM XML and GeoJSON format exports from JOSM:
Saimaa_GeoJSON.zip
Saimaa_OSM_XML.zip
mboeringa commentedon Nov 26, 2021
@arturbil
I've checked this MP in JOSM using the validation options, and it comes out fine. So the original MP appears technically sound.
mboeringa commentedon Nov 26, 2021
That one also checks out fine in JOSM
kleunen commentedon Nov 26, 2021
Can you guys run tilemaker with --verbose? if tilemaker thinks the input relation is invalid, it will give an output message. If it gives no output message on this relation, in that case, the input geometry is valid, but tilemaker makes the geometry invalid during processing.
arturbil commentedon Nov 26, 2021
For the whole Finland extract tilemaker warns about 27 relations that either have intersections or intersecting interiors or interior rings outside. It includes cases of Saimaa (7379046) and Inari (402543), both have self-intersections in the tilemaker's opinion, but it doesn't mention the smaller lake Suvasvesi (900367), which I think also has this incorrect simplification issue.
arturbil commentedon Dec 1, 2021
Found a bit smaller relation (https://www.openstreetmap.org/relation/1478385) - this time it's a forest. Tilemaker says "Relation 1478385 has intersecting interiors".
wood-1478385.geojson.zip
.
kleunen commentedon Dec 10, 2021
It is quite possible these issues are caused by the fact coordinates get scaled when writing the pbf output. The floating point coordinates are scaled back to integers and they may cause issues with the polygons.
brawer commentedon May 18, 2025
Has this been resolved, or is it still an issue?
ttsirkia commentedon May 18, 2025
I haven't checked the situation now after my last comment as using Tippecanoe for the process, but would be interesting to try again and see if the issue is still visible.
systemed commentedon May 18, 2025
On balance it’s much better than in 2020 when the issue was opened. There have been a couple of fixes which have greatly improved multipolygon handling. I would say though that it’s not 100% bulletproof - if you try and simplify complex geometries a lot for lower zoom levels, it’ll still occasionally trip up on edge cases. I use Natural Earth data for the very lowest zoom levels instead of OSM, and that works really well.
ttsirkia commentedon May 18, 2025
Sounds great! I'll try the situation with the Finnish lakes in the near future.
ttsirkia commentedon Apr 18, 2026
After a very long time, I've been working with creating tiles again. I can confirm that the situation is much much better nowadays. When there is no simplification used at all, there are not any incorrectly drawn water areas in Finland.
If simplification is used, a very small number of some drawing issues like this will appear:
With value
"simplify_level": 0.0003the issue is minimal but increasing the number seems to also increase the amount of problems. But even with0.0013there is not much them. However, it seems to affect only certain zoom levels for some areas, not all zoom levels below a certain value. In most cases the affected area was very small and it took some time even to spot them.ttsirkia commentedon Apr 18, 2026
Here is one place just as an example:
https://www.openstreetmap.org/#map=14/60.60991/26.76896
10.71/60.6224/26.7425Visible only in zoom level 10.
And another:
https://www.openstreetmap.org/#map=14/62.49118/29.36345
9.81/62.5092/29.2962The issue is visible only in zoom levels 8 and 9.
Both constructed with:
"water": { "minzoom": 4, "maxzoom": 12, "simplify_below": 11, "simplify_level": 0.0005, "filter_below": 10, "filter_area:": 50 },dktue commentedon Oct 29, 2026
I retested with current master (
4e0f03d), today's Finland extract and the water layer config from April (simplify_below11,simplify_level0.0005,filter_below10), counting lake polygons that are invalid in tile coordinates:"invalid_polygon_repair_scope": "all""all"fixes that.