Tile-Generator

C++ toolset for generating optimized vector map tiles from OpenStreetMap PBF files, targeting ESP32-based GPS navigators.

Features

• Pure Hilbert Indexing (New): Uses a space-filling curve for both data ordering and indexing, ensuring maximum spatial locality and optimized SD card seek patterns.
• Binary Tile Deduplication (New): Identifies identical tiles (e.g., land/ocean background) and reuses data blocks, significantly reducing final file size.
• High-Performance C++ Engine: OSM PBF parsing and tile generation using GEOS, GDAL, and Libosmium.
• Efficient Binary Format: Packed NPK2 containers with Delta+ZigZag+VarInt coordinate encoding.
• Memory-Mapped Storage: Uses mmap for feature storage, allowing processing of large PBF files with minimal RAM.
• Multi-threaded Processing: Parallel tile generation leveraging all available CPU cores.
• Z-Order Management: 4-pass rendering pipeline supported in binary format.

Generation Process

The generator operates in multiple passes to ensure topological consistency and optimal packaging:

1. Pass 1 (Relations): Scans PBF for administrative boundaries and water multipolygons.
2. Pass 2 (Features): Extracts nodes and ways, applying semantic filtering and layer assignment.
3. Pass 3 (Water): Integrates global water polygons from external Shapefiles (using GDAL/OGR).
4. Pass 4 (Tiles): Parallel clipping, simplification (GEOS), and NPK2-Hilbert packaging.

Dependencies & Installation

Required Libraries

• Libosmium: PBF parsing.
• GEOS: Geometry operations (clipping, simplification, unions).
• GDAL/OGR: Shapefile support for ocean water polygons.
• Protozero: Low-level OSM data handling.
• C++17 Compiler: GCC 9+ or Clang.

Installation (Ubuntu/Debian)

sudo apt-get update
sudo apt-get install -y build-essential cmake \
    libosmium2-dev libgeos-dev libgdal-dev \
    libbz2-dev zlib1g-dev libexpat1-dev

Compilation

mkdir build && cd build
cmake ..
make -j$(nproc)

Usage

./nav_generator <osm_pbf_file> <output_dir> <features_json> [--zoom min-max]

Example

./nav_generator andorra-latest.osm.pbf ./NAVMAP features.json --zoom 10-15

Ocean Water Polygons (optional)

Download the pre-computed water polygons shapefile (once, ~540 MB):

wget https://osmdata.openstreetmap.de/download/water-polygons-split-4326.zip
unzip water-polygons-split-4326.zip

Generate with oceans:

./nav_generator input.pbf output features.json --zoom 6-17 \
    --water-shp water-polygons-split-4326/water_polygons.shp

The generator reads the PBF bounding box and only loads water polygons intersecting the extract area. See WATER.md for details.

Visualization & Debugging

The project includes a Python-based simulator to validate maps before flashing them to the ESP32.

Prerequisites

• Python 3.x
• Pygame: pip install pygame

Usage

python3 tile_viewer.py <output_dir> --lat <latitude> --lon <longitude> --config features.json

Map controls

Key / Action	Effect
Arrow keys / Mouse drag	Pan map
[ / ]	Zoom out / in
Scroll wheel	Zoom out / in
B	Toggle background color (white/black)
F	Toggle polygon fill
G	Toggle tile grid
H	Toggle Hilbert path (spatial locality check)
S / L	Toggle stats / legend panels
Q / Escape	Quit

A* routing overlay

If a ROUTE/ directory exists inside <output_dir> (generated by the graph builder), the viewer supports interactive routing:

Action	Effect
Left-click on map	Set route origin (green marker)
Right-click on map	Set route destination (red marker) + compute A* route (blue track)
R	Clear route and reload tiles

The routing log panel (bottom-right) shows:

• Origin and destination coordinates
• Graph size (nodes / edges)
• Nearest graph nodes for src and dst
• Route length (km) and node count
• Nodes visited by A* and computation time

Route Generator

route_generator builds A* routing graph files from the same OSM PBF. It runs independently from tile generation — use it with vector tiles, PNG tiles, or any map source.

Usage

route_generator <input.pbf> <output_dir>

The output directory receives a ROUTE/ROUTE.bin file containing the full routing graph. Internally the graph is partitioned into 0.05°×0.05° subcells with an index — readers load only the cells needed for the route on-demand.

Example

# Build tile generator and route generator
cd build && cmake .. && make -j$(nproc)

# Generate routing graph for Andorra
./route_generator andorra-251227.osm.pbf .
# → ./ROUTE/ROUTE.bin

# Copy to SD card alongside tiles
rsync -av NAVMAP/ /media/sdcard/NAVMAP/
rsync -av ROUTE/  /media/sdcard/ROUTE/

For the full binary format specification see docs/route_generator.md.

Internal Format Details

• Tile container: NPK2-Hilbert (Flat Hilbert Index) — docs/bin_tile_format.md
• Tile internal format: NAV1 (Geometry + Text labels)
• Coordinates: Web Mercator, 12-bit tile-relative space (0-4096)
• Routing graph: ROUTE.bin (header 32B + index 20B/cell + nodes 12B + edges 12B, interleaved per cell) — docs/route_generator.md

---

Author

Jordi Gauchía (jgauchia@jgauchia.com)