Skip to content

feat: Add Map Generator tab for offline map tile downloading#2599

Open
MartinovEm wants to merge 37 commits into
iNavFlight:maintenance-9.xfrom
MartinovEm:feature/map-generator
Open

feat: Add Map Generator tab for offline map tile downloading#2599
MartinovEm wants to merge 37 commits into
iNavFlight:maintenance-9.xfrom
MartinovEm:feature/map-generator

Conversation

@MartinovEm
Copy link
Copy Markdown

@MartinovEm MartinovEm commented Mar 26, 2026

Add a new Map Generator tab that allows downloading offline map tiles for OSD map widgets on radios. Works without a flight controller connection.

Hero

Supported output targets: b14ckyy ETHOS Mapping Widget, Yaapu Telemetry (ETHOS and EdgeTX sub-targets). Map providers: OpenStreetMap, ESRI, Google (Street/Satellite/Hybrid/Terrain). Features: draw rectangle to select download area with live measurements, configurable zoom level range (1-20), sync tiles directly to mounted SD card, export tiles as ZIP archive, eject SD card from the app (Windows/macOS/Linux), local tile cache for faster repeated downloads, save/restore settings persistence, geocoder search, measure tool, right-click context menu.

Drawing Sync

@github-actions
Copy link
Copy Markdown

Branch Targeting Suggestion

You've targeted the master branch with this PR. Please consider if a version branch might be more appropriate:

  • maintenance-9.x - If your change is backward-compatible and won't create compatibility issues between INAV firmware and Configurator 9.x versions. This will allow your PR to be included in the next 9.x release.

  • maintenance-10.x - If your change introduces compatibility requirements between firmware and configurator that would break 9.x compatibility. This is for PRs which will be included in INAV 10.x

If master is the correct target for this change, no action is needed.


This is an automated suggestion to help route contributions to the appropriate branch.

@MartinovEm MartinovEm force-pushed the feature/map-generator branch from 0f18283 to e6e75da Compare March 26, 2026 09:33
@MartinovEm MartinovEm changed the base branch from master to maintenance-9.x March 26, 2026 09:37
@MartinovEm MartinovEm force-pushed the feature/map-generator branch 5 times, most recently from 2861132 to bf1f572 Compare March 28, 2026 09:06
@MartinovEm
Copy link
Copy Markdown
Author

Update: Added INAV Terrain (SRTM) target

The Map Generator now supports generating INAV-compatible .DAT terrain files directly from the configurator — no external tools needed.

What it does

Select "INAV Terrain (SRTM)" as the target, draw a rectangle over your flying area, and the configurator downloads NASA SRTM 30m elevation data, converts it to .DAT files, and writes them directly to the flight controller's SD card (or exports as ZIP).

How it works

  1. Downloads SRTM1 HGT tiles from AWS (public domain NASA data)
  2. Decompresses gzip, performs bilinear interpolation at each grid point
  3. Packs into 2048-byte blocks with CRC16-XMODEM checksums
  4. Outputs one .DAT file per 1°×1° degree square (INAV-compatible format)

Features

  • 1°×1° terrain grid overlay with .DAT filename labels on the map
  • IndexedDB HGT cache — downloaded SRTM tiles persist across sessions, no re-download on repeat generation
  • Live altitude readout — hover the mouse to see SRTM elevation after generation (from cached data, no server requests)
  • SD card sync with FREESPAC.E auto-deletion, write progress with speed/ETA
  • ZIP export when no SD card is linked
  • SRTM coverage warning for selections beyond 60°N / 56°S
  • Completion screen shows the CLI command to enable terrain (set terrain_enabled = ON)
  • Ocean depths clamped to 0m (matches INAV firmware behavior)

Validation

The terrain math has been validated against 133M+ data points across 13 tiles (flat terrain to 2900m mountains). Flight-tested on real hardware (SpeedyBee F405 Wing, INAV 9.0.1) — sub-meter mean error vs source data.

Screenshots

Terrain mode — grid overlay and area selection:

selecting_area

Generating terrain files — sync modal with progress:

generating_tiles

Sync complete — files written to SD card:

terrain_synced

@MartinovEm
Copy link
Copy Markdown
Author

Fix: OSM tiles returning 403 in packaged builds (commit 97e9739)

While working on the Map Generator tab, I noticed that the packaged/release builds of the configurator get 403 errors from OSM tile servers (affects GPS tab, Mission Control — any tab that loads OSM tiles). Dev mode works fine.

Root cause: The packaged build loads the app using the local file protocol (mainWindow.loadFile(...)), so tile requests carry an invalid or empty Referer header and Chromium's generic User-Agent. OSM rejects these per their Tile Usage Policy (sections 3.1 and 3.4), which requires:

  • A unique User-Agent identifying the application (not a library/browser default)
  • A valid Referer header from web contexts

In dev mode, Vite serves via http://localhost, giving a valid Referer — so the issue only appears in packaged builds.

Fix: Added session.webRequest.onBeforeSendHeaders in js/main/main.js to set a proper Referer and User-Agent for OSM tile requests. 6 lines, no other code affected. Tested in a packaged win64 build — zero 403s.

This brings the configurator into compliance with OSM's tile usage requirements.

MartinovEm and others added 15 commits April 20, 2026 00:41
Add a new Map Generator tab that allows downloading offline map tiles for OSD map widgets on radios. Works without a flight controller connection. Supported output targets: b14ckyy ETHOS Mapping Widget, Yaapu Telemetry (ETHOS and EdgeTX sub-targets). Map providers: OpenStreetMap, ESRI, Google (Street/Satellite/Hybrid/Terrain). Features: draw rectangle to select download area with live measurements, configurable zoom level range (0-19), sync tiles directly to mounted SD card, export tiles as ZIP archive, eject SD card from the app (Windows/macOS/Linux), local tile cache for faster repeated downloads, save/restore settings persistence, geocoder search, measure tool, right-click context menu.
- New target: INAV Terrain generates .DAT files from NASA SRTM1 30m data
- SRTM conversion engine: HGT download, gzip decompression, bilinear
  interpolation, CRC16-XMODEM checksums, INAV-compatible DAT format
- IndexedDB HGT cache for resume/reuse of downloaded SRTM tiles
- 1 degree x 1 degree terrain grid overlay with .DAT filename labels
- Live altitude readout from cached SRTM data on mouse hover
- Terrain status: selected area, grid coverage, estimated size, file list
- SRTM coverage warning for selections beyond 60N/56S
- SD card sync with FREESPAC.E auto-deletion and write progress
- ZIP export alternative when no SD card linked
- Completion screen with CLI enable command (set terrain_enabled = ON)
- Ocean bathymetry clamped to 0m (matches INAV firmware behavior)
SonarCloud flags 'syncAborted is not modified in this loop' when the
variable is checked in the for-condition. Moving the check to an
explicit if/break inside the loop body resolves the false positive
(syncAborted is set by the async cancel button handler between await
iterations).
- Fix EPERM when writing .DAT files to drive root (mkdir on G:\ fails)
- Auto-detect SD card removal on tab load and window focus (pathExists IPC)
- Separate tile cache and HGT cache UI per target mode
- Style HGT cache clear button to match tile cache button
- Rename Clear Cache to Clear Tile Cache (only clears map tiles now)
@MartinovEm MartinovEm force-pushed the feature/map-generator branch from ead53a0 to 18be333 Compare April 19, 2026 21:43
- package.json: replace file-saver with fflate (upstream), keep
  jszip/leaflet/leaflet-control-geocoder/leaflet-draw (Map Generator)
- locale/en/messages.json: keep mapgen* keys, add layer* keys
  from upstream geo layers feature (mission control)
…n.lock

- package.json: fix fflate indentation conflict (keep all deps from both sides)
- yarn.lock: regenerated after merge to include all packages
@sonarqubecloud
Copy link
Copy Markdown

@sensei-hacker
Copy link
Copy Markdown
Member

Fix: OSM tiles returning 403 in packaged builds (commit 97e9739)

While working on the Map Generator tab, I noticed that the packaged/release builds of the configurator get 403 errors from OSM tile servers (affects GPS tab, Mission Control — any tab that loads OSM tiles). Dev mode works fine.

Root cause: The packaged build loads the app using the local file protocol (mainWindow.loadFile(...)), so tile requests carry an invalid or empty Referer header and Chromium's generic User-Agent. OSM rejects these per their Tile Usage Policy (sections 3.1 and 3.4), which requires:

  • A unique User-Agent identifying the application (not a library/browser default)
  • A valid Referer header from web contexts

In dev mode, Vite serves via http://localhost, giving a valid Referer — so the issue only appears in packaged builds.

Fix: Added session.webRequest.onBeforeSendHeaders in js/main/main.js to set a proper Referer and User-Agent for OSM tile requests. 6 lines, no other code affected. Tested in a packaged win64 build — zero 403s.

This brings the configurator into compliance with OSM's tile usage requirements.

That's really interesting. I have confirmed it does set the User-Agent as required -- but I checked in dev mode.
They say it doesn't have to set a referer since it's not a web site, but it doesn't hurt to send one.

Speaking of their TOS, I see also:

You must not:

Bulk download (“scrape”) tiles or offer prefetch features.

Do you have thoughts on that? I think the spirit of it is they don't want people downloading tiles they don't end up using, to reduce wasteful load on their servers. Would this feature, if merged, potentially cause that?

@MartinovEm
Copy link
Copy Markdown
Author

MartinovEm commented May 25, 2026

@sensei-hacker Thanks for pointing this out!

I agree with your interpretation of the OSM Standard tile policy regarding offline/bulk download. Before I change behavior in this PR, I want to confirm the intended direction with you:

  1. Keep OSM available for normal interactive map viewing, so the Map Generator tab follows the rest of the Configurator tabs, where OSM is used for previews and quick orientation.
  2. Restrict offline export flows (Sync to SD / ZIP) when OSM is selected.
  3. Show a clear in-app message explaining that OSM Standard does not permit offline bulk export, and ask users to choose another provider for export.

If you agree with this direction, I will implement it in this PR.

@sensei-hacker
Copy link
Copy Markdown
Member

sensei-hacker commented May 31, 2026

@sensei-hacker Thanks for pointing this out!

I agree with your interpretation of the OSM Standard tile policy regarding offline/bulk download. Before I change behavior in this PR, I want to confirm the intended direction with you:

  1. Keep OSM available for normal interactive map viewing, so the Map Generator tab follows the rest of the Configurator tabs, where OSM is used for previews and quick orientation.
  2. Restrict offline export flows (Sync to SD / ZIP) when OSM is selected.
  3. Show a clear in-app message explaining that OSM Standard does not permit offline bulk export, and ask users to choose another provider for export.

If you agree with this direction, I will implement it in this PR.

Seems reasonable to me. I see that for example MapTiler offers up to 5,000 sessions per month with their free tier, using the same API. Would it make sense to have a link to a documentation page that has a couple such providers?
perhaps even add a commercial provider or two (such as Maptiler) directly int he drop-down? These would be providers that are free for any reasonable number of requests.

@MartinovEm
Copy link
Copy Markdown
Author

@sensei-hacker Thank you for the suggestions! I've implemented the first part:

  • OSM is now preview-only -- the Sync to SD and Export as ZIP buttons are disabled when OSM Standard is selected, with an inline notice directing users to choose ESRI or Google for offline export. OSM still works for interactive map viewing, consistent with how it's used in other configurator tabs.

Regarding MapTiler: I'm starting work on adding it as a provider directly in the dropdown. Since it requires a personal API key, the UI will include a key input field (shown only when MapTiler is selected) with a short inline message guiding users to get a free key at maptiler.com -- no separate documentation page needed.

@sensei-hacker
Copy link
Copy Markdown
Member

Since it requires a personal API key, the UI will include a key input field (shown only when MapTiler is selected) with a short inline message guiding users to get a free key at maptiler.com -- no separate documentation page needed.

That will probably be more convenient than the existing API key field that is under the gear icon at top right of Configurator, won't it.

I guess it can be the same setting under the hood, but just put the input field right where people need it?

@MartinovEm
Copy link
Copy Markdown
Author

MartinovEm commented May 31, 2026

@sensei-hacker Thanks for the suggestion! Just to clarify — there isn't an existing map API key field under the gear icon. The gear settings only have the map provider for the Mission/GPS tabs (OSM / ESRI / MapProxy with URL + layer), which is separate from the Map Generator. So the key is new and lives right in the Map Generator tab where it's most useful, stored persistently in electron-store (mapgen_maptiler_key) so users only need to enter it once.

Here's a summary of what's been added:

New provider: MapTiler with 6 map types:

Type Notes
Satellite Aerial imagery (JPEG)
Hybrid Satellite + labels (JPEG)
Street MapTiler streets style (PNG)
OSM Style OpenStreetMap-rendered tiles — a compliant alternative for bulk export
Outdoor Topographic + outdoor features (PNG)
Topo Clean topo lines (PNG)

The key input shows only when MapTiler is selected, with an inline "Get a free key at maptiler.com" link. A help icon next to the Provider dropdown links directly to the README setup section. MapTiler's free tier covers 5,000 sessions/month which is plenty for personal use.

MapTiler_Key

Full setup guide with registration steps: README → MapTiler Setup

@sonarqubecloud
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants