You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jul 30, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Jul 30, 2026. It is now read-only.
Currently the viever doesn't work on local gateways as it tires to access global gateway, which is blocked by no-CORS. I reported CORS issue ipfs/infra#166 and created patch to enable those options by default ipfs/kubo#2778.
The best way of accessing data on IPFS using XHR is rooted relative link: /ipfs/Qm...AAA. Then if site is opened on global gateway, the global gateway will be used, and vice versa.
I would like to suggest a few features for preventing tracking and maximizing user privacy:
Provide an easy way to pin map data for large geographic areas (like cities), so that queries can be made privately on the user's own machine
Compress the map data as much as possible so that it's practical to store on e.g. mobile devices (not sure how large the OSM data is currently, if it's small this may not be an issue)
Structure the data so that efficient differential updates are possible (again OSM may take care of this for you, or not)
Have an optional warning before trackable remote requests are made (e.g. if the map isn't stored locally, don't just blindly fetch it, since that could leak information)
Thanks @davidar for letting me know about this project!
Provide an easy way to pin map data for large geographic areas (like cities), so that queries can be made privately on the user's own machine
@btrask Good idea. I'd have to think about the best way to do this.
Compress the map data as much as possible so that it's practical to store on e.g. mobile devices (not sure how large the OSM data is currently, if it's small this may not be an issue)
It's about 50GB in total, but substantially less for lower levels of detail. Having said that, I don't think it's possible (or at least not recommended) to run an ipfs node on mobile devices yet.
Structure the data so that efficient differential updates are possible (again OSM may take care of this for you, or not)
IPFS deduplicates tiles, so update efficiency would depend on how many tiles are changed.
Have an optional warning before trackable remote requests are made (e.g. if the map isn't stored locally, don't just blindly fetch it, since that could leak information)
Yeah, the issue of tracking/privacy in the context of ipfs has been brought up before. In this case you could just use ipfs offline, but that would cause remote requests to silently fail rather than issue a visible warning, so probably isn't the best solution. @jbenet Thoughts?
IPFS deduplicates tiles, so update efficiency would depend on how many tiles are changed.
Deduplication helps, but it's still up to the application to make sure that its data dedups effectively. That means making sure that small changes to the map result in localized changes within the database. (You probably already know this, but it doesn't hurt to document.)
@btrask localised changes to the map should only change a small number of tiles. @lukasmartinelli would probably be able to give a more concrete answer than me though :)
@btrask localised changes to the map should only change a small number of tiles. @lukasmartinelli would probably be able to give a more concrete answer than me though :)
There are two things that go on.
A small number of tiles change very frequently. I can't say percent numbers but it feels like 20% of the tiles change every week and the others don't have any changes at all.
There are butterfly effects when changing boundaries.
Thanks to de-duplication properties of IFPS it shouldn't really affect us, I might write a tool to calculate de-duplication ratio of two different hashes if there is need for that.
@Kubuxu telling the deduplication factor between two hashes is pretty straightforward. The simplest way is to run ipfs refs -r on both, and use comm to take the differences of the set.
Alternatively, using ipfs object diff if the data is formatted in a nice enough way would provide a nice view
Just had the time to play properly with this for the first time and my reaction was WOW! :D This looks and feels really good!
@lukasmartinelli what about http://osm2vectortiles.org/downloads/ are served by IPFS also, it would really only requires to add an hash to be there on the webpage since we are already hosting it, making the download of those 50GB way faster :)
@lukasmartinelli what about http://osm2vectortiles.org/downloads/ are served by IPFS also, it would really only requires to add an hash to be there on the webpage since we are already hosting it, making the download of those 50GB way faster :)
Not entirely sure whether I get it. Could IPFS then serve the downloads - instead of S3?
But we still need to upload the downloads to S3? IPFS would mirror them.
So we would need to append a shortened sha-1 of the file to the filename and then IPFS could serve the downloads?
We can have IPFS serving all the downloads or we can have the hybrid (the first option would reduce storage and bandwidth costs).
It would be a sha2-256 and you can get it by doing an ipfs add <path to the blob>. Once you get that hash, you will be able to get it through ipfs.io or a user can also 'reprovide' it by booting up a node and loading it through their IPFS node. I'm pulling the data set right now to add it to my local node and so that I can share with you the IPFS Hash. However, if you have the data set already, you can do it too :)
Activity
davidar commentedon Jun 1, 2016
https://ipfs.io/ipfs/QmNUFNQXWVzbhePhj6bscW6TPu5azraCpbejbNY9zyfFjo/
A higher-resolution map is ready to go, but blocked on ipfs/kubo#1924
CC: @jbenet @whyrusleeping
davidar commentedon Jun 1, 2016
Thanks again to @osm2vectortiles @lukasmartinelli for making this possible :)
lukasmartinelli commentedon Jun 1, 2016
Really cool. Do you generate the vector tiles yourself or use osm2vectortiles downloads v1.0 or v2.0?
We could even use that as CDN in our showcases.
This is also only data for upper zoom levels? As I zoom in in Europe > zoom level 11 I miss data.
davidar commentedon Jun 1, 2016
@lukasmartinelli I extracted them from the osm2vectortiles v1.0 download. What changes have been made in v2.0?
That would be cool :)
Currently it only goes up to level 9, but once ipfs/kubo#1924 is fixed I'll add the remaining zoom levels (10-14)
Kubuxu commentedon Jun 1, 2016
Currently the viever doesn't work on local gateways as it tires to access global gateway, which is blocked by no-CORS. I reported CORS issue ipfs/infra#166 and created patch to enable those options by default ipfs/kubo#2778.
The best way of accessing data on IPFS using XHR is rooted relative link:
/ipfs/Qm...AAA. Then if site is opened on global gateway, the global gateway will be used, and vice versa.lukasmartinelli commentedon Jun 1, 2016
Okay perfect. We are also just finished with v2.0, we will release a changelog.
http://osm2vectortiles.org/downloads/
In short: has much better quality, smaller filesize and biggest of all can now be updated in a weekly interval.
Really happy to have IPFS host this 😁 Thanks @davidar for your work.
davidar commentedon Jun 1, 2016
@Kubuxu Fixed! https://ipfs.io/ipfs/QmNfwFPQqJARCQz87i1V3uAEF2t6WxAffTfA678Ad1JZiv/?/ipfs/QmUefFZttPf9xq4KTkk94rBbZEVrBsTreDi4JA8KYhQFX6/
btrask commentedon Jun 4, 2016
This sounds very promising!
I would like to suggest a few features for preventing tracking and maximizing user privacy:
Thanks @davidar for letting me know about this project!
davidar commentedon Jun 5, 2016
@lukasmartinelli Updated to osm2vectortiles v2.0 (z0-z8): https://ipfs.io/ipfs/QmTAEbxXKoBWDzNduLS9WmAoNkcuHgQbc3fm9oGBe3FfR4/?/ipfs/Qmf3KBYSv9H4xADgKqpvwWrXkiMh5BfZohFXCaaFh7u4nb/
@btrask Good idea. I'd have to think about the best way to do this.
It's about 50GB in total, but substantially less for lower levels of detail. Having said that, I don't think it's possible (or at least not recommended) to run an ipfs node on mobile devices yet.
IPFS deduplicates tiles, so update efficiency would depend on how many tiles are changed.
Yeah, the issue of tracking/privacy in the context of ipfs has been brought up before. In this case you could just use ipfs offline, but that would cause remote requests to silently fail rather than issue a visible warning, so probably isn't the best solution. @jbenet Thoughts?
davidar commentedon Jun 5, 2016
@lgierth Does it? The readme must be outdated
btrask commentedon Jun 5, 2016
@davidar sounds good!
Just one comment:
Deduplication helps, but it's still up to the application to make sure that its data dedups effectively. That means making sure that small changes to the map result in localized changes within the database. (You probably already know this, but it doesn't hurt to document.)
davidar commentedon Jun 6, 2016
@btrask localised changes to the map should only change a small number of tiles. @lukasmartinelli would probably be able to give a more concrete answer than me though :)
lukasmartinelli commentedon Jun 6, 2016
There are two things that go on.
A small number of tiles change very frequently. I can't say percent numbers but it feels like 20% of the tiles change every week and the others don't have any changes at all.
There are butterfly effects when changing boundaries.
Quote from @ImreSamu osm2vectortiles/osm2vectortiles#251
OSM butterfly effect : https://en.wikipedia.org/wiki/Butterfly_effect
The butterfly effect refers to a concept that small causes can have large effects.for example modifying small part of France - Schweiz border : http://www.openstreetmap.org/changeset/39252252
Has an effect of other part of the French-administered territories ...

Kubuxu commentedon Jun 6, 2016
Thanks to de-duplication properties of IFPS it shouldn't really affect us, I might write a tool to calculate de-duplication ratio of two different hashes if there is need for that.
davidar commentedon Jun 6, 2016
@lukasmartinelli why does that happen, out of interest?
lukasmartinelli commentedon Jun 10, 2016
Because all these lines are in the same superior relation "France".
whyrusleeping commentedon Jun 16, 2016
@Kubuxu telling the deduplication factor between two hashes is pretty straightforward. The simplest way is to run
ipfs refs -ron both, and usecommto take the differences of the set.Alternatively, using
ipfs object diffif the data is formatted in a nice enough way would provide a nice viewdaviddias commentedon Jun 24, 2016
Just had the time to play properly with this for the first time and my reaction was WOW! :D This looks and feels really good!
@lukasmartinelli what about http://osm2vectortiles.org/downloads/ are served by IPFS also, it would really only requires to add an hash to be there on the webpage since we are already hosting it, making the download of those 50GB way faster :)
lukasmartinelli commentedon Jun 24, 2016
Not entirely sure whether I get it. Could IPFS then serve the downloads - instead of S3?
But we still need to upload the downloads to S3? IPFS would mirror them.
So we would need to append a shortened sha-1 of the file to the filename and then IPFS could serve the downloads?
daviddias commentedon Jun 24, 2016
We can have IPFS serving all the downloads or we can have the hybrid (the first option would reduce storage and bandwidth costs).
It would be a
sha2-256and you can get it by doing anipfs add <path to the blob>. Once you get that hash, you will be able to get it throughipfs.ioor a user can also 'reprovide' it by booting up a node and loading it through their IPFS node. I'm pulling the data set right now to add it to my local node and so that I can share with you the IPFS Hash. However, if you have the data set already, you can do it too :)daviddias commentedon Jun 24, 2016
Here you go :D
planet_z0-z8.mbtiles
planet_z0-z5.mbtiles
planet.mbtiles
If you run an local ipfs node, replace
ipfs.iobylocalhost:8080on the links :)The
Qm...are base58 encoded sha2-256 hashesjbenet commentedon Jul 5, 2016
cc @mikolalysenko
davidar commentedon Jul 30, 2016
Graduating to its own repo: https://github.com/davidar/ipfs-maps