Skip to main content

Upgrade or remove monitoring

Upgrade​

Each Putki release publishes a new monitoring bundle. Upgrading keeps all recorded runs.

  1. Unzip the new bundle into a new folder and copy your .env into it.
  2. Compare your .env with the new .env.example and add any settings you need.
  3. From the old folder, stop the stack: docker compose -f docker-compose-monitoring.yml down. Without -v, so the data volumes stay.
  4. From the new folder, start it: docker compose -f docker-compose-monitoring.yml up -d. It finds the same data volumes, and logging-db-init brings the schema up to date.
  5. Re-run install-logging.sh against each project with the same --project-name. It lists the files it replaces and leaves your environment variables alone.

If you imported the dashboards into your own Grafana, import the new JSON files over the old ones. If you manage your own database, apply the new sql/init-logging.sql.

From the original 2026.09 bundle​

The 2026.09 bundle was replaced on the release page with a corrected build of the same name. You have the original, or an earlier bundle, if the VERSION file in your folder has one line only. The corrected build adds a second line starting with built.

  • Keep your data: set the project name once. Older bundles named their data volumes after the folder they ran in. Before step 4, add that name to the new .env: the folder name in lower case, without dots, so putki-monitoring-2026.09 becomes

    COMPOSE_PROJECT_NAME=putki-monitoring-202609

    docker volume ls | grep monitoring shows the name if you are unsure. Newer bundles use a fixed name, so this is needed only once.

  • The bundled database is now the default. The new .env.example contains COMPOSE_PROFILES=logging-db. If you used your own PostgreSQL, leave that line out of your .env. If you started the stack with --profile logging-db, the line replaces the flag.

  • Run times are now stored with their time zone. The first start migrates log_object. Runs already recorded are read as UTC. If your Hop ran on local time, runs recorded before the upgrade keep their old offset; runs after it are exact.

  • The Marquez (OpenLineage) datasource is removed. Grafana deletes it on the first start.

Remove from a project​

Delete the five files the installer added:

rm -f  <project>/logging/pipeline-log.hpl <project>/logging/workflow-log.hpl
rm -rf <project>/metadata/pipeline-log <project>/metadata/workflow-log
rm -f  <project>/metadata/rdbms/logging.json

Runs stop being recorded at once; nothing else in the project changes. The LOGGING_* variables can stay in the environment file: nothing uses them without the pipelines.

Remove the stack​

docker compose -f docker-compose-monitoring.yml down       # keeps the data
docker compose -f docker-compose-monitoring.yml down -v    # also deletes all recorded runs