Opened 8 years ago

Last modified 11 months ago

#745 new defect/bug

variable blue areas

Reported by: foxxi59 Owned by: cp15
Priority: major Milestone: version 0.5.2
Component: mapdrivers/OSM Version: git master
Severity: Keywords: tile, water, binfile
Cc: jff@…

Description

By android and Linux (ubuntu 10.10) the map was displayed with variable blue areas.

The maps are osm-based.

Attachments (26)

Foto3.png (76.5 KB) - added by foxxi59 8 years ago.
Full blue
Foto4.png (89.1 KB) - added by foxxi59 8 years ago.
Full blue with item Window
Foto5.png (5.0 KB) - added by foxxi59 8 years ago.
Item Part1
Foto6.png (720 bytes) - added by foxxi59 8 years ago.
Item Part 2
Foto7.png (4.5 KB) - added by foxxi59 8 years ago.
Item Part 3
Bildschirmfoto-Navit.png (152.3 KB) - added by martin bruns 7 years ago.
Navit map with blue area
Bildschirmfoto-Navit-1.png (166.0 KB) - added by martin bruns 7 years ago.
Navit map without blue area
Lake-Constance.png (118.2 KB) - added by martin bruns 7 years ago.
france_in_ocean.jpg (27.5 KB) - added by jhc 7 years ago.
France in the ocean from Paris to Marseille
paris_in_ocean.jpg (176.2 KB) - added by jhc 7 years ago.
Paris in the ocean
blonville_jff-navitmap-FR_10_04_2011.bin.jpg (35.5 KB) - added by jhc 7 years ago.
Blonville : clean coastline, road, and area on 10/04/2011
blonville_jff-navitmap-FR_20_11_2011.bin.jpg (54.9 KB) - added by jhc 7 years ago.
Blonville : garbled coastline, road, and area on 20/11/2011
argenton_jff-navitmap-FR_10_04_2011.bin.jpg (43.0 KB) - added by jhc 7 years ago.
Argenton: clean coastline on 10/04/2011
argenton_jff-navitmap-FR_20_11_2011.bin.jpg (60.1 KB) - added by jhc 7 years ago.
Argenton: damaged coastline on 20/11/2011
olmetto_jff-navitmap-FR_20_11_2011.bin.jpg (54.3 KB) - added by jhc 7 years ago.
Olmeto, from a France map generated by jff on 20/11/2011
olmeto_corse_geofabrik_17_11_2011.pbf.bin.jpg (56.1 KB) - added by jhc 7 years ago.
Olmeto, from a Corsica map generated using maptool on 17/11/2011
small.portopollo.working.bin (13.4 KB) - added by martin bruns 7 years ago.
middle.portopollo.notworking.bin (22.7 KB) - added by martin bruns 7 years ago.
verysmall.portopollo.notworking.bin (709 bytes) - added by martin bruns 7 years ago.
small.portopollo.working.osm.gz (53.0 KB) - added by martin bruns 7 years ago.
middle.portopollo.notworking.osm.gz (72.2 KB) - added by martin bruns 7 years ago.
verysmall.portopollo.notworking.osm.gz (2.7 KB) - added by martin bruns 7 years ago.
coastline_problem_01.png (15.7 KB) - added by tegzed 7 years ago.
the problem with unclosed coastlines
201304_maps9.navit-project.org_LakeConstance__poly_water_tiled-visible_and_invisible.gif (161.5 KB) - added by google 5 years ago.
201308_maps9.navit-project.org_LakeConstance__poly_water_tiled-invisible.gif (153.7 KB) - added by google 5 years ago.
201308_maps9.navit-project.org_LakeConstance__poly_water_tiled-visible.gif (106.2 KB) - added by google 5 years ago.

Download all attachments as: .zip

Change History (54)

comment:1 Changed 8 years ago by korrosa

Unfortunately, "variable blue areas" doesn't really help us. Before submitting a ticket, please check: http://wiki.navit-project.org/index.php/Reporting_Bugs

At least provide the location of those "variable blue areas" within the map, or an OSM based link, and we can check what you're seeing. It also might be a good idea to provide the layout which you're using. Something else which might be a good is a screenshot...

Changed 8 years ago by foxxi59

Full blue

Changed 8 years ago by foxxi59

Full blue with item Window

Changed 8 years ago by foxxi59

Item Part1

Changed 8 years ago by foxxi59

Item Part 2

Changed 8 years ago by foxxi59

Item Part 3

comment:2 Changed 8 years ago by foxxi59

Via IRC I ask 2 or 3 month before and I got the answer:

This is a known problem with an integer - overflow.

comment:3 Changed 8 years ago by tegzed

Hi Foxxy59,

Which version of Navit are you using? Where did you get your maps? Can you try a recent map from http://maps.navit-project.org/ with a recent svn build? It can be that the map item type ids are changed since an old version.

comment:4 Changed 8 years ago by foxxi59

Version navit: 0.5.0 svn 3931

The maps are from http://jff-webhosting.net/osm/navit/world/europe/

With a map from http://maps.navit-project.org/ the same blue sea

Jörg

comment:5 Changed 8 years ago by tegzed

That version should be fine... Can you specify the exact location where this occurs on the map? I'll try to reproduce it on my system.

comment:6 Changed 8 years ago by foxxi59

The location is Landscheid near by Trier and the motorway A60

comment:7 Changed 8 years ago by tegzed

I have tried that area using map extractor and seft created map(from osm xml) also, both worked correctly on my navit. Can you try creating your map from osm xml data as a test for that area? If this self made map fixes the problem then there could be something wrong with item type ids in your navit. You can also consult Number6 on irc he is experienced with map creation and as I remember he faced this problem before.

comment:8 Changed 8 years ago by foxxi59

Hi,

in android svn 4061 und linux svn 4056 the maps allways displayed without the blue areas.

comment:9 Changed 7 years ago by thuryn1

Hi,

on Linux Fedora 15, gtk, latest SVN, maps from osm - maps allways displayed WITH the blue areas. It looks, that blue areas start displayed, when I add bigger area. I built czech_republic.osm - works well. I built germany.osm and blue area start to show. But this could be rendom behavior.

Changed 7 years ago by martin bruns

Navit map with blue area

Changed 7 years ago by martin bruns

Navit map without blue area

comment:10 Changed 7 years ago by martin bruns

Hi,

I found that when I disable poly_water_tiled in my layout then the blue areas are gone. So as a workaround I did this to all the layout way it was in. See the attached screenshots, with and without the blue area.

http://trac.navit-project.org/attachment/ticket/745/Bildschirmfoto-Navit.png

http://trac.navit-project.org/attachment/ticket/745/Bildschirmfoto-Navit.png

But it also depends of the map. I tested different maps from http://maps.navit-project.org/ on some it is on some not. I have no idea what this triggers in maptool.

M

comment:11 follow-up: Changed 7 years ago by thuryn1

I've noticed one more bug (maybe), which could be related to this bug. If try to find water area called Fláje (Czech republic), I can see water area as is in real world. If I see maps in navit and try to find Fláje, the water area is not rendered. There is more than one example of this defect, but not all water areas is not rendered.

Changed 7 years ago by martin bruns

comment:12 in reply to: ↑ 11 Changed 7 years ago by martin bruns

Yes you are right I also noticed this on the lake constance see attached picture. So I think the maptool needs to be checked for that.

http://trac.navit-project.org/attachment/ticket/745/Lake-Constance.png

M

comment:13 Changed 7 years ago by jpduval2008

With Navit 0.50-4281 on Android for Motorola Defy, I have the same problem :

When I put the following item as the first one of the first layer :

<itemgra item_types="poly_water_tiled"><polygon color="#82c8ea"/></itemgra>

1) It is painted around Lyon, France, althrough there is no ocean around. In fact, the ocean square is drawn from Paris to Milano where there is nothing else to paint, such as town, wood, etc. The default color of the layout is ignored.

2) It is correctly painted in the south of Marseille for the Méditerranée sea.

The bug disppear when I disable the poly_water_tiled item. But I have no more ocean ou sea...

I can't add a screenshot, since I only use Navit on Android.

It looks like an overflow of an integer value that creates a huge polygon an instance of the poly_water_tiled item instead of a small one.

Changed 7 years ago by jhc

France in the ocean from Paris to Marseille

Changed 7 years ago by jhc

Paris in the ocean

comment:14 Changed 7 years ago by jhc

I have been trying for a few days setting up an environment for debugging maptool on this annoying bug, but up to now I did not succeed in generating a map for France from http://download.geofabrik.de/osm/europe/france.osm.pbf : I had to hack buffer.c for files > 2Gb, still it takes too long on Cygwin on my old PC, and possibly loops forever in Phase 6 (navit-svn-4838).

I support what martin bruns and jpduval2008 write. It is indeed related to poly_water_tiled, same problem both on Android and on Windows : see http://trac.navit-project.org/attachment/ticket/745/france_in_ocean.jpg and http://trac.navit-project.org/attachment/ticket/745/paris_in_ocean.jpg

Has anyone investigated what <freinhard> wrote in http://irclogs.navit.ie/%23navit-2011-08-14.log : "osm.c:1167 in osm_add_tag, there are some cast's from long to int, which look like they could cause the overflow?" ?

Cheers,

comment:15 Changed 7 years ago by korrosa

The interesting thing about this problem is that it may be caused by an incorrectly tagged lake, perhaps with wrong inner and outer boundaries, or an unclosed way. Of course, maptool should really try and deal with these errors instead of drowning half the French...

comment:16 Changed 7 years ago by jhc

korrosa : I would rather suspect maptool : apparently this happens only when a large (>200Mb) OSM file is converted by maptool to a .bin file : example : http://jff-webhosting.net/osm/navit/world/europe/navitmap-FR.bin or Germany, but not from czech_republic.osm .

If some water area was inside out, we should be able to locate it by generating a map around it.

So my opinion is that when water areas accross tiles are split into tiled items, under some conditions (overflow ? rare cases ?) some poly_water_tiled items are generated inside out or rather with garbled points.

comment:17 Changed 7 years ago by korrosa

Just to point out: I wasn't suggesting it wasn't maptool, just that if people are experiencing this problem, then it could be a data issue.

Changed 7 years ago by jhc

Blonville : clean coastline, road, and area on 10/04/2011

Changed 7 years ago by jhc

Blonville : garbled coastline, road, and area on 20/11/2011

Changed 7 years ago by jhc

Argenton: clean coastline on 10/04/2011

Changed 7 years ago by jhc

Argenton: damaged coastline on 20/11/2011

comment:18 Changed 7 years ago by jhc

korrosa : Indeed there are data issues on coastline. See blonville_jff-navitmap-FR_20_11_2011.bin.jpg and argenton_jff-navitmap-FR_20_11_2011.bin.jpg.

But it looks like to me it is due to a corrruption of points or ways, within maptool, and that this corruption is getting worse with time (as size of the map increases).

For reference, blonville_jff-navitmap-FR_10_04_2011.bin.jpg and argenton_jff-navitmap-FR_10_04_2011.bin.jpg show the status on the same map, http://jff-webhosting.net/osm/navit/world/europe/navitmap-FR.bin version of 10/4/2011, taken 7 months before, 754MB instead of 994MB on 20/11/2011...

Notice in the Blonville case, how the coastline, the road, and the constructed area are all damaged, as they probably use the same corrupted points and ways...

Changed 7 years ago by jhc

Olmeto, from a France map generated by jff on 20/11/2011

Changed 7 years ago by jhc

Olmeto, from a Corsica map generated using maptool on 17/11/2011

comment:19 Changed 7 years ago by jhc

More stunning example : Olmeto, in Corsica. OSM source is nearly the same (3 days difference, around 17/11/2011).

When the source file is small (corse.osm.pbf, 10MB, from geofabrik), OK : olmeto_corse_geofabrik_17_11_2011.pbf.bin.jpg.

When the file is large (France, 1700MB), wooops ! : olmeto_jff-navitmap-FR_20_11_2011.bin.jpg.

Last edited 7 years ago by jhc (previous) (diff)

comment:20 Changed 7 years ago by martin bruns

I think we are talking here about two (at least) problems.

  • the type of problem I reported in the picture http://trac.navit-project.org/attachment/ticket/745/Lake-Constance.png is basically missing water in lakes and rivers. I found that this problem is fixed with maptool svn rev. 4890, thanks! Unfortunately the bin maps on http://maps.navit-project.org/ are still affected from this. Does someone know when the tools behind that page will be updated. Nevertheless I think this problem is fixed.
  • The other issues are the blue areas over several maps. I think there is currently only a workaround available for this, namely the one I suggested earlier by disabling the poly_water_tiled items. Does anyone have an idea about that? Im pretty sure it is a maptool bug so far.

comment:21 Changed 7 years ago by jhc

To martin bruns : Yes, there are distinct problems. Lake-Constance.png is probably better addressed in the multipolygon ticket #754.

This ticket here #745 is definitely due to ocean tiles added by maptool at the wrong place, possibly as a consequence of maptool damaging map data, when the size of the map goes over some limit (overflow ?), but also maybe because coastlines on one country or area don't define where the sea is, unless the country or area is an island.

A dirty workaround in maptool could be to avoid creating an ocean tile if the tile contains a ... city, e.g. ?

I have found and tested another workaround, but it involves using maps generated without ocean tiles, and using one ocean map (coastline.bin) that contain all ocean tiles for the whole world, and nothing else. As a secondary benefit, using a map of an island will not drown anybody else...

Changed 7 years ago by martin bruns

Changed 7 years ago by martin bruns

Changed 7 years ago by martin bruns

Changed 7 years ago by martin bruns

Changed 7 years ago by martin bruns

Changed 7 years ago by martin bruns

comment:22 Changed 7 years ago by martin bruns

I played a bit with the size of the map.osm.

The http://trac.navit-project.org/attachment/ticket/745/small.portopollo.working.osm.gz map.osm file works fine but http://trac.navit-project.org/attachment/ticket/745/middle.portopollo.notworking.osm.gz not, I compared the difference in terms of coastlines and found one way which is in the middle one but not the small one. I only created a map using that way, http://trac.navit-project.org/attachment/ticket/745/verysmall.portopollo.notworking.osm.gz And still it is not working. That coastline is around one island.

I cannot judge whether that will help to fix the problem in maptool somehow, maybe it is fine for someone else to workon, seems to be similar info jhc found/mention.

To jhc: What do you exactly mean by your workaround using coastline.bin. I would like to give it a try. Would you mind to explain it in more detail?

Thanks for linking to the multipolygon ticket #754 . funny we are dealing with #745 and #754.

M

Changed 7 years ago by tegzed

the problem with unclosed coastlines

comment:23 Changed 7 years ago by tegzed

I think the problem is related to the lack of enough info in map _extracts and possible errors in map data. In extracts only partial coastlines are available. These can be unclosed and for unclosed coastlines it can be impossible to decide what is water and what's not. I created a map from the above attached middle.portopollo.notworking.osm.gz that contains only the water tiles generated from coastlines in the osm file (see the attached screenshot: coastline_problem_01.png). (I used a rule file with a single coastline entry for maptool to create only the coastline based items). I think based on the screenshot that maptool has no info to decide if the area near the unclosed part of coastline is water or not. And maybe the tile generator uses the water/land info from neighbor tiles to determine the type of the actual tile, so one bad decision can flood a lot of "dry land"s. I think the best solution is jhc's workaround above to generate a global coastline map from full coastline data and coastline-less binfiles from map extracts.

comment:24 Changed 5 years ago by usul

  • Component changed from core to mapdrivers/OSM
  • Keywords tile water binfile added; blue maps removed
  • Milestone changed from version 0.5.0 to version 0.5.1
  • Owner changed from KaZeR to cp15

Does this issue appear till now? At least at the offical map servers, I don't see such behaviour for months?

comment:25 Changed 5 years ago by martin bruns

Yes I also do not see a problem with the latest SVN 5543 and a fresh Germany map (version 130706). So that one should be closed.

Thanks for highlighting this.

M

comment:26 Changed 5 years ago by tryagain

Actually, this bug reintroduces sometimes for some regions and is related to osm data problems (lake\river isles tagged with natural=coastline, self-intersecting coastlines, not closed coastlines).

Some of the recent examples are:

http://www.openstreetmap.org/browse/way/222972775/history

http://www.openstreetmap.org/browse/way/209998416/history

http://www.openstreetmap.org/browse/way/157217675/history

I've never seen any blue area problem which I could not solve by fixing a bug in osm coastline data.

We could plan implementing more robust logic in maptool to workaround these bugs automatically or at least report them. Unsure if this can be done without affecting maptool performance too much.

comment:27 Changed 5 years ago by google

I use a dirty workaround for this bug by eliminating the existence of open waters:

File: navit.xml

Section: <layout name="Car" ...>

Position: about Line 800

<itemgra item_types="poly_water_tiled" order="0-">
   <polygon color="#82c8ea"/>
</itemgra>

you can either comment the the whole block to

<!--<itemgra item_types="poly_water_tiled" order="0-">
   <polygon color="#82c8ea"/>
</itemgra>-->

or just change the order of the poly_water_tiled item to a value larger than 0 (1 for example)

<itemgra item_types="poly_water_tiled" order="1">
   <polygon color="#82c8ea"/>
</itemgra>

Please compare Files:

In April 2013 high sea areas were invisible on the the map taken from http://maps9.navit-project.org anyway and the bug did not occur:

In August 2013 high sea areas are existent on the the map taken from http://maps9.navit-project.org and they are overflowing:

Removing the high sea areas by making the item poly_water_tiled ineffective (as shown above) results in a usable map but it also shows a probable failure that the Rhine and over rivers weren't drawn through anymore, as done in April 2013:

comment:28 Changed 11 months ago by http://wiki.navit-project.org/index.php/user:jkoan

  • Milestone changed from version 0.5.1 to version 0.5.2

This ticket was pushed back in order to bring 0.5.1 out soon.

Note: See TracTickets for help on using tickets.