path: root/README.adoc

= hittekaart(1)
:source-highlighter: rouge

== NAME

hittekaart - A GPX track heatmap generator.

== SYNOPSIS

----
hittekaart [--output=...] [--min-zoom=...] [--max-zoom=...] [--threads=...] [--sqlite] FILES...
----

== INSTALLATION

You can build the binary `./target/release/hittekaart` with `cargo`:

----
cargo build --release
----

== DESCRIPTION

`hittekaart` is a tool to generate heatmaps from GPX tracks. It reads a number
of GPX files and produces OSM-compatible overlay tiles. Note that `hittekaart`
itself does not display any maps, instead you can use the generated heatmap
tiles as overlay layers in other applications such as
https://leafletjs.com/[Leaflet] or
https://sourceforge.net/projects/viking/[Viking].

=== OUTPUT FORMAT

The generated tiles are saved according to the
https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames[slippy map tilenames],
so they can be served by any normal HTTP server.

By default, the directory `tiles/` will be used as the root directory, so a
tile would be saved as `tiles/{zoom}/{x}/{y}.png`. You can change this by using
the `--output/-o` option.

=== SQLITE OUTPUT

In order to overcome storage overhead when saving many small files (see the tip
and table further below), `hittekaart` can instead output a SQLite database
with the heatmap tile data. To do so, use the `--sqlite` command line option,
and control where the SQLite file should be placed with `--output`/`-o`.

While this does not allow you to immediately serve the tiles with a HTTP
server, it does cut down on the wasted space on non-optimal file systems.

The generated SQLite file will have one table with the following schema:

[source,sql]
----
CREATE TABLE tiles (
    zoom INTEGER,
    x INTEGER,
    y INTEGER,
    data BLOB,
    PRIMARY KEY (zoom, x, y)
);
----

=== INPUT FILES

`hittekaart` expects GPX track files with the `.gpx` extension. It will parse
them and read the track points. Optionally, you can feed it _compressed_ GPX
files:

* Files ending with `.gz` will be considered `gzip` compressed and unpacked on
  reading.
* Files ending with `.br` will be considered `brotli` compressed and unpacked
  on reading.

=== MULTITHREADING

Heatmap generation involves some CPU intensive parts, namely the parsing of the
GPX files and the PNG compression. Especially the latter takes up a big part of
the heatmap generation. By default, `hittekaart` will parallelize those
processes, by default using as many threads as you have CPU cores.

If you want to limit the number of threads that will be used, for example
because you want to run the heatmap generation in the background, use the
`--threads` option. Setting a thread count of 0 will use the default behaviour
of using all cores, while setting it to any other number will use that many
threads. The setting `--threads=1` effectively disables multithreading.

=== ZOOM LEVELS

By default, all zoom levels from 0 (a one-tile overview of the whole world) to
19 (the maximum that the openstreetmap.org tile server offers) will be
generated. However, you can control that setting with `--min-zoom` and
`--max-zoom`, for example if you don't want the heat maps to be as detailed.

Keep in mind that every zoom level has four times as many tiles as the previous
zoom level, which quickly increases the number of tiles. This also means that
the more detailed levels will take up considerably more space than previous
levels. For example, a sample of 64 tracks leads to the following sizes:

[%header,cols="1,1,1,1"]
|===
|Level
|# Tiles
|Size
|Cum. Size

|0 |1 |8.4 KiB |8.4 KiB

|1 |1 |8.4 KiB |16.8 KiB

|2 |1 |8.4 KiB |25.2 KiB

|3 |1 |8.5 KiB |33.7 KiB

|4 |1 |8.7 KiB |42.4 KiB

|5 |2 |9.3 KiB |51.7 KiB

|6 |2 |9.9 KiB |61.6 KiB

|7 |2 |11 KiB |72.6 KiB

|8 |2 |13 KiB |85.6 KiB

|9 |3 |24 KiB |109.6 KiB

|10 |7 |40 KiB |149.6 KiB

|11 |23 |81 KiB |230.6 KiB

|12 |60 |173 KiB |403.6 KiB

|13 |160 |373 KiB |776.6 KiB

|14 |402 |813 KiB |1589.6 KiB

|15 |973 |1.8 MiB |3.4 MiB

|16 |2,294 |3.9 MiB |7.3 MiB

|17 |5,277 |8.1 MiB |15.4 MiB

|18 |11,638 |16 MiB |31.4 MiB

|19 |25,729 |34 MiB |65.4 MiB
|===

You can see that starting at level 14, each _single_ level takes as much space
as all previous levels combined.

[TIP]
.A Note on Small Files
====
The table shows the logical file sizes. Usually, the files are a bit bigger on
disk, as the file size is not an exact multiple of the block size. For a
standard block size of 4 KiB on an ext4 system for example, you would end up
with a total of 195 MiB, 107 MiB just for zoom level 19. This is a massive
increase in storage requirement, simply from the fact that the files do not
fill up all allocated blocks.

If you intend to store a lot of heatmaps, it might be worth setting up a file
system that is optimized for a large amount of small files, for example by
setting a smaller block size. Many of the PNG images are smaller than 2 KiB
(half a standard block); for those 50% of storage is wasted already.

If you don't need the tiles in separate files, you can use the SQLite output
mode. For the same data as above, the SQLite database would be 73 MiB in size.
====

== OPTIONS

The following options are supported:

`--min-zoom=ZOOMLEVEL`::
    Set the minimum zoom level to generate. Defaults to 0, which is a one-tile
    overview of the world.

`--max-zoom=ZOOMLEVEL`::
    Set the maximum zoom level to generate (inclusive). Defaults to 19, the
    maximum zoom that the openstreetmap.org tile server offers.

`-t THREADS`, `--threads=THREADS`::
    Sets the number of threads. Defaults to 0, which means that `hittekaart`
    will automatically pick a default.

`-o DIRECTORY`, `--output=DIRECTORY`::
    Generate the output tiles into the given directory. Defaults to `tiles/`
    when generating single files, and `tiles.sqlite` when storing the tiles in
    a SQLite database.

`--sqlite`::
    Output a single SQLite file with all tiles instead of saving each tile as a
    separate PNG file. In this case, `-o` can be used to set the location of
    the SQLite database. The schema is described above.

== EXAMPLE

You can generate a heatmap and serve it locally with the following commands:

----
hittekaart ~/Documents/GPX/*.gpx
cd tiles
python -m http.server
----

With the tile server running, you can then open a HTML file like the following:

[source,html]
----
<!DOCTYPE html>
<html>
  <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.3/dist/leaflet.css"
   integrity="sha256-kLaT2GOSpHechhsozzB+flnD+zUyjE2LlfWPgU04xyI="
   crossorigin=""/>
  <script src="https://unpkg.com/leaflet@1.9.3/dist/leaflet.js"
   integrity="sha256-WBkoXOwTeyKclOHuWtc+i2uENFpDZ9YPdf5Hf+D7ewM="
   crossorigin=""></script>
  <style>
#map { height: 800px; }
  </style>
  <body>
    <div id="map"></div>
    <script>
      var map = L.map('map').setView([52.520008, 13.404954], 13);
      var osm = L.tileLayer('https://tile.openstreetmap.org/{z}/{x}/{y}.png', {
        maxZoom: 19,
        attribution: '&copy; <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>'
      });
      var heatmap = L.tileLayer('http://localhost:8000/{z}/{x}/{y}.png', {
        maxZoom: 19,
      });

      L.control.layers([osm], [heatmap]).addTo(map);
    </script>
  </body>
</html>
----

== BUGS

Feel free you report bugs on the issue tracker at
https://gitlab.com/dunj3/hittekaart/-/issues or via email (see below).

== AUTHOR

Daniel Schadt - contact me electronically with this project's name at
kingdread.de

== COPYRIGHT

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program. If not, see <https://www.gnu.org/licenses/>.