diff options
Diffstat (limited to 'doc/developer')
| -rw-r--r-- | doc/developer/updater.rst | 151 |
1 files changed, 151 insertions, 0 deletions
diff --git a/doc/developer/updater.rst b/doc/developer/updater.rst new file mode 100644 index 0000000..f8fa59f --- /dev/null +++ b/doc/developer/updater.rst @@ -0,0 +1,151 @@ +The Updater +=========== + +The process of updating for administrators is described in +:doc:`../administration/upgrading`. Here, we will describe the developer side +of the update process. + +Motivation +---------- + +Some updates do not only change the code of Fietsboek, but adapt the data that +is stored. New features for example might require new columns in the database, +or other adaptions might move data around (commit 7a60619d_ for example moves +the GPX data out of the database and into the data directory). + +.. _7a60619d: https://gitlab.com/dunj3/fietsboek/-/commit/7a60619d3f6fd523d42f50753436f3b7e7d72ca4 + +This process is commonly called *data migration*. In Fietsboek, we use Alembic_ +to do the heavy lifting of SQL migrations. Alembic can automatically create +migration scripts and provides an abstraction over the different database +backends. + +.. _Alembic: https://alembic.sqlalchemy.org/en/latest/ + +With the image uploads, and more concretely with commit 5a205756_, Fietsboek +has been extended to store data on disk. This has resulted in `issue 20`_, +which raises the question about how to deal with update scripts now: Alembic +handles the SQL updates well, but now we have a second place to run migrations +for. + +.. _5a205756: https://gitlab.com/dunj3/fietsboek/-/commit/5a2057560a5703a59408009e40b51de5a0c18600 +.. _issue 20: https://gitlab.com/dunj3/fietsboek/-/issues/20 + +This has resulted in the ``fietsupdate`` tool, which implements a similar logic +as Alembic, but for general purpose Python code instead of SQL. It also acts as +the user interface for administrators, providing some abstractions over the +Alembic CLI. + +Update scripts +-------------- + +An update script contains the logic that ``fietsupdate`` applies when going +from one version to the next. Such a script looks like the this: + +.. code:: python + + """Revision upgrade script v0.8.0 + + Date created: 2023-06-05 21:00:37.464583 + """ + from fietsboek.updater.script import UpdateScript + + update_id = 'v0.8.0' + previous = [ + 'v0.7.0', + ] + alembic_revision = '3149aa2d0114' + + + class Up(UpdateScript): + def pre_alembic(self, config): + pass + + def post_alembic(self, config): + pass + + + class Down(UpdateScript): + def pre_alembic(self, config): + pass + + def post_alembic(self, config): + pass + +* Each script imports the :class:`~fietsboek.updater.script.UpdateScript` class + as a base. +* Each update has a unique ID, which is used on the command line to identify + the correct script. For most scripts, this is a randomly generated string, + but some specific revisions have a more readable ID. +* Each update (except for the initial one) has references to its previous + versions. As such, the commits form a DAG. This allows ``fietsupdate`` to + check which updates have been applied, and which still need to be applied. +* Each update has a reference to the underlying alembic version. When + ``fietsupdate`` applies the update, it will make sure that the database will + have the given alembic version after. +* Finally, each update has the actual update logic, in four sections: + Migrations that are run *before* the alembic update and *after* the alembic + update, each for the upgrade and the downgrade direction. + +Creating Alembic versions +------------------------- + +There is nothing special about how we use Alembic to generate SQL revisions. As +such, we refer to the `Alembic documentation`__. + +.. __: https://alembic.sqlalchemy.org/en/latest/cookbook.html#create-revision-migrations + +In general, a command like this should work:: + + alembic -c development.ini revision --autogenerate + +Creating Fietsupdate versions +----------------------------- + +The ``fietsupdate`` has a hidden command to generate a revision from the +template, and it will populate it with the current alembic revision:: + + fietsupdate revision -c development.ini + +When to create revisions +------------------------ + +If you change Fietsboek code or documentation that does not change the way that +data is stored, you do *not* need to bother with Alembic or Fietsupdate. + +If you change the database schema, or the way that the data is stored in the +existing schema, you need to create an Alembic revision. + +If you change the data that is stored in the data directory, you need to create +a Fietsupdate script. + +If a new Fietsboek version is released, a new Fietsupdate migration is created +with the version as revision ID, to ensure that ``fietsupdate update v0.1.2`` +will work. This script is usually empty, and only contains a reference to the +relevant previous migration ID and the correct alembic revision. + +Further notes +------------- + +When running Fietsboek from ``master``, it might happen that the repository is +in a state in which Alembic revisions exist, but no Fietsupdate revision that +refers to them. In this case, you must run + +:: + + alembic -c development.ini upgrade head + +Before creating revisions, make sure that your repository and database are in +the correct state. Otherwise your revision might refer to an outdated revision. +In particular, you should ensure that you have applied all previous updates. + +While both Alembic and Fietsupdate have support for "branches" and "merges", it +is still preferable to use this feature as little as possible. If possible, +rebase the changes to give them a linear order. + +We cannot guarantee that migration scripts will always continue to work (e.g. +if they depend on functionality which is later removed from Fietsboek, +dependencies that are removed, ...). For this reason, big update jumps *might* +be hard to support. However, all scripts should at least stay loadable (that +means their metadata is readable, there are no syntax errors), and for empty +databases they should work (to allow bootstrapping of fresh instances). |