Skip to content

Add MapLibre Tile (MLT) output - #938

Open
dktue wants to merge 1 commit into
systemed:masterfrom
dktue:mlt-output
Open

dktue wants to merge 1 commit into
systemed:masterfrom
dktue:mlt-output

Conversation

@dktue

@dktue dktue commented Oct 12, 2026

Copy link
Copy Markdown

Adds optional MapLibre Tile (MLT) output, as discussed in #856. It's off by default at build time and in the config; MVT output is unchanged.

Results (Switzerland, bundled OpenMapTiles profile, .pmtiles; details below):

  • 12% smaller tilesets: 274.6 MB as MLT vs 313.5 MB as MVT, after gzip.
  • Memory: +0.9% peak RSS (7.65 vs 7.58 GB). MLT output adds almost no memory.
  • Time: +8% wall clock (85.2 vs 79.1 s). The encoder is configured to run about 7× faster than with its default settings, for a compressed size within 2.5% of what the defaults produce.
  • No change for MVT users: with this PR, MVT output and speed are the same as master within measurement noise.

Usage

Build with -DTILEMAKER_WITH_MLT=ON, then set "tile_format": "mlt" in settings. Lua profiles and layer configs don't change. In MapLibre styles, the source needs "encoding": "mlt".

  • .pmtiles: header tile type 0x06 (MapLibre Tile)
  • .mbtiles: metadata format is mlt
  • directory output: z/x/y.mlt
  • --merge is rejected with MLT (merging needs to decode existing tiles; could be added later with the MLT decoder)

How it works

  • Encoder: the C++ encoder from maplibre-tile-spec (merged there in June), fetched with CMake FetchContent at a pinned commit. Only the encoder target and its two needed submodules (fsst, earcut) are used. FETCHCONTENT_SOURCE_DIR_MLT can point at a local checkout for offline builds.
  • One code path for both formats: tile_worker.cpp's write functions are templated over a small backend. The MVT backend is plain vtzero, so the MVT code path compiles to the same thing as before. The MLT backend (mlt_writer.h) has builders with the same method names that collect a tile's features, then encodes once per tile with a thread-local encoder.
  • Feature order is kept (encoder sortFeatures off), so ZOrder still controls draw order.
  • Column types: MLT stores each attribute as a typed column per layer, and the encoder turns int + float mixes into strings. tilemaker promotes the integers of mixed int/float columns to double instead, so AttributeInteger and AttributeNumeric on the same key in one layer stay numeric.
  • Encoder settings: for compressed output (gzip/deflate .mbtiles/directories, and .pmtiles, which are always gzipped), FSST string compression and Morton/Hilbert vertex dictionaries are off and geometry uses plain varints. These choices make up most of the encoder's time, but after gzip they save next to nothing (see below). Uncompressed output keeps the encoder's defaults.
  • Build: the CMake build moves to C++20, which the encoder's headers need. Two small compatibility fixes come with it: BOOST_ASIO_DISABLE_CO_AWAIT (Boost 1.74's Asio coroutine header doesn't compile as C++20; tilemaker doesn't use coroutines), and std::uncaught_exceptions() instead of the removed std::uncaught_exception() in the bundled sqlite_modern_cpp (MSVC). The Makefile build is unchanged and has no MLT support. MLT itself needs CMake 3.25+ (required by maplibre-tile-spec).
  • Docker and CI: the Dockerfile builds with MLT on (adds git and ca-certificates for the fetch). The Linux CMake CI job builds with MLT and writes an MLT .pmtiles of Liechtenstein.

Testing

  • MLT matches MVT feature for feature. I used a test-only build that writes every tile as both MVT and MLT from the same data in the same run, then decoded both with vtzero and with maplibre-tile-spec's own decoder and compared them in order (ids, geometry type, every coordinate, every property). Clip cache lookups were disabled for this, because cache reuse is timing dependent and otherwise makes the two passes differ. Inputs: Monaco and Luxembourg, with the bundled OpenMapTiles profile.
    • Monaco: identical, 10,912 features.
    • Luxembourg (about 1 million features): the only geometry differences are in polygon grouping. MVT has no explicit polygon structure, so readers infer it from ring winding, and after simplification a tiny ring can have the "wrong" winding and get read as a hole. MLT stores the structure tilemaker actually wrote.
    • The one other difference wasn't from MLT. A few cycleways came out as class=track instead of path. That turned out to be an existing intermittent issue on master (in 4 of 5 runs of unmodified master on Luxembourg); I'll report it separately.
  • Mixed int/float columns: a synthetic profile writing AttributeInteger and AttributeNumeric to the same key gives a DOUBLE column with the expected values.
  • Unit tests: make test passes, including a new config validator test for tile_format.
  • Not tested locally: MSVC and macOS with MLT enabled. CI covers the C++20 switch on all platforms but only builds MLT on Linux.

Benchmarks

Switzerland (548 MB .osm.pbf), bundled OpenMapTiles profile, .pmtiles output, 22 threads, average of 3 interleaved runs:

format wall CPU peak RSS size
MVT (master) 79.1 s 1429 s 7.58 GB 313.5 MB
MVT (this PR) 78.9 s 1444 s 7.59 GB 313.4 MB
MLT 85.2 s 1558 s 7.65 GB 274.6 MB (−12%)

MVT is unchanged within noise. MLT is about 8% slower with under 1% more memory, for tiles 12% smaller.

Encoder settings, single-threaded, Luxembourg tiles, sizes after zlib:

encoder settings ms/tile size after zlib
encoder defaults 4.4 27.15 MB
defaults + FSST 10.0 27.18 MB
this PR (no FSST, no Morton, plain geometry) 0.6 27.84 MB

Open questions

  • Should MLT be on by default? I kept it opt-in because it adds a network fetch at configure time and needs CMake 3.25.
  • EncoderConfig::geometryEncodingOption is commented upstream as "primarily for testing". The commit is pinned, but it may be worth asking maplibre-tile-spec to treat it as supported.
  • I'm happy to split out the C++20 change or the templating refactor if that's easier to review.

🤖 Generated with Claude Code

Add an optional MLT writer, built with -DTILEMAKER_WITH_MLT=ON and
selected with "tile_format": "mlt". The tile writing code is templated
over an MVT (vtzero) and an MLT backend; MLT tiles are encoded with the
maplibre-tile-spec C++ encoder, fetched at a pinned commit.

- PMTiles tile type 0x06, MBTiles format=mlt, .mlt directory output
- feature order (ZOrder) preserved; mixed int/float columns kept numeric
- fast encoder settings for compressed output
- CMake build moves to C++20, with fixes for Boost 1.74 Asio and the
  bundled sqlite_modern_cpp; Dockerfile and Linux CI build with MLT

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@systemed

Copy link
Copy Markdown
Owner

Thank you - this is brilliant. Fully supportive of the opt-in approach because of the increased system requirements but we can maybe move to opt-out some time in the future. I'll take a proper look in the next couple of days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants