Setting up an OpenStreetMap Virtualbox server for research purposes

It's the start of the new year again, time to download OpenStreetMap global data. As a result, I was surprised to find that the main station could no longer be opened. After carefully understanding the whole story, I expressed my understanding. Fortunately, PBF data can still be obtained, but the tile service has been shut down.

1. Main problems of OpenStreetMap

The main reasons why OpenStreetMap is Blocked are as follows. The first is to violate the laws and regulations related to surveying and mapping, causing infringement on important geographic coordinates. Anyone can go up and mark it, which facilitates illegal surveying and mapping with ulterior motives. Secondly, the disputed geographical delineation and terminology do not comply with our regulations. Especially the islands and borders are in a mess and have been changed beyond recognition.

Through the Block master station, the cost of illegal surveying and mapping has indeed been raised, and law-abiding citizens should fully support it. Considering the global data contained in OpenStreetMap, there is still a high learning value. In the case that the main site is no longer accessible, we can download PBF data from other offline mirrors for research purposes. As long as it is not published as a tile service, there will be no problem.

2. PBF data acquisition

The master station cannot download the PBF file. It is also impossible to download the paid VPN through the traffic of a friend's company. PBF costs tens of GB at every turn, and the traffic overhead is too high, which will exhaust the cost. Don’t download images or even plot images by building a ladder by yourself. You may face traceability and cause trouble, so remind programmers to pay attention . Currently, the mirror address can be selected for regular access. Only a few European mirrors and fullplanet exports are accessible. But if your network speed is super slow, the download speed of 69GB file is only tens of KB, and at the update rate of weekly, the file name will be changed before the download is finished.

1. Data for some regions can be obtained from https://download.geofabrik.de/, such as

$ wget https://download.geofabrik.de/asia-latest.osm.pbf

Data for Asia can be downloaded.

2. Download cities or global mirrors from the European mirror BBBike

https://download.bbbike.org/osm/

It is best to use a certain mine to download, the speed is very fast.

3.ArchLinux tile service changes

OpenStreetMap's tile service is implemented by the mod_tile project. In 2023, there is a mod_tile.git package in the AUR library of the ArchLinux series, but it is outdated. The easiest way is to compile the source code after checking out the source code from the git repository after installing the dependencies.

$ git clone https://github.com/openstreetmap/mod_tile.git
$ cd mod_tile*
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install
$ sudo cp src/.libs/mod_tile.so /usr/lib/httpd/modules/mod_tile.so
$ ls /usr/local/bin/renderd
只要存在src/.libs/mod_tile.so和/usr/local/bin/renderd，那大概就木有问题。

For specific configuration, please refer to other articles in the column. There are two important configuration files, one is the configuration of mod_tile in apache2, and the other is the configuration of renderd.

1. httpd.conf

Include conf/extra/mod_tile.conf

2. mod_tile.conf

LoadModule tile_module modules/mod_tile.so
<VirtualHost *:80>
    ServerName archosm
    ServerAlias a.archosm b.archosm c.archosm d.archosm
	DocumentRoot /srv/http
    ModTileTileDir /var/lib/mod_tile
    LoadTileConfigFile /etc/renderd.conf
    ModTileEnableStats On
    ModTileBulkMode Off
    ModTileRequestTimeout 3
   #...
</VirtualHost>

3. renderd.conf

[renderd]
socketname=/var/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/lib/mod_tile
stats_file=/var/run/renderd/renderd.stats

[mapnik]
plugins_dir=/usr/lib/mapnik/input
font_dir=/usr/share/fonts/
font_dir_recurse=1

[default]
URI=/osm_tiles/
TILEDIR=/var/lib/mod_tile
XML=/home/archosm/osmstyle/openstreetmap-carto/mapnik.xml
HOST=tile.archosm.org
TILESIZE=256
MINZOOM=0
MAXZOOM=20

[contour]
URI=/contour/
TILEDIR=/var/lib/mod_tile
TILESIZE=256
XML=/home/archosm/osmstyle/contour-carto/contour.xml
HOST=dem.archosm.org

The relevant parameter configuration should be modified according to the specific situation. The rest of the steps and other articles in this column have changed little. Reminder! Since the main site has been blocked, the provision of OpenStreetMap-related tile hosting services cannot avoid legal and regulatory issues. If you need OpenStreetMap local tiles, you can continue to visit the virtual machine cloud disk link in my column to obtain virtual machines for research use , but never publish OSM data publicly yourself.

4. Why does the distribution still choose Manjaro

Using manjaro as the OpenStreetMap server, of course, still considers that the software is very new. Manjaro software update is super fast, and it is more convenient than ArchLinux, simply, super awesome!
ArchOSM does not need to be updated in the virtual machine, so there is no need to consider the problem of postgreSQL and other key infrastructure upgrades. Of course, in case PostgreSQL is updated to a major version, I'm sorry, but I have to re-import the data backup again. With this feature, it is recommended that users who need to consider stability still choose the Debain system (UbuntuLTS).

5. Evaluation of OpenStreetMap

1. A truly open map. Really anyone can edit. Of course, there must be a large number of inaccuracies, such as "grandma's fish pond" and other place names often appear. There are also marks with ulterior motives, which are very annoying and make it impossible to access.
2. Data models, tool chains, servers, data, and applications are all open source. This is very powerful, even Google and BING cannot do it. I don't know how much manpower and resources are needed to develop such a standard. How can we start making profits by disclosing data structures, algorithms and data?
3. Whether it is pgRoutings or layers, it is indeed not as beautiful as BING, Google, and even domestic maps, and the color matching is also lacklustre. Especially the algorithm, without the support of dynamic traffic data, it is impossible to do well. Therefore, OSM is mainly used as a free tile base map for many industry software. In Matlab or other software, the background is not pure gray.