Skip to main content

Troubleshoot

Start with the first question: did anything reach the database? On the logging database:

SELECT type, count(*) AS runs, max(start_date) AS latest FROM log_object GROUP BY type;

With the bundled database, run it with docker compose -f docker-compose-monitoring.yml exec logging-db psql -U hop_logging -d logging. An empty result means the Hop side is not writing; rows mean the problem is on the Grafana side.

The dashboards show no data​

  • The query above returns nothing. Hop is not logging. Check that the run used the project you installed into and the environment file you passed to the installer: a run under an environment without the LOGGING_* variables fails to connect, and a run in another project has no logging pipelines. The Hop log of the run shows either as an error from a transform named Insert / update pipeline or Insert / update workflow.
  • The query returns rows. Widen the time range, and set Project, Name and Outcome to All. The dashboard counts top-level runs: workflows and pipelines started directly, not the ones another workflow started. Set Runs to Every execution to count those too.

A run shows up late, or in the wrong place on the time axis​

Your logging database was created by the original 2026.09 bundle or an earlier one (a VERSION file with one line), and Hop runs on a machine whose time zone is not UTC: run times were stored without their time zone. Upgrade the bundle, or apply the current sql/init-logging.sql to your database: it migrates the table once. See Upgrade.

A run stays "Running"​

A run is recorded when it starts and updated when it ends. A run that stays Running never logged its end:

  • Its Hop process stopped: killed, out of memory, the container removed. The run keeps the status it last had. The Run still running long after its usual time alert reports it.
  • The logging at the end of the run failed. The Hop log of the run shows an error from a transform named Insert / update pipeline or Insert / update workflow. In the Putki stack this also happens when Hop's execution information location (Neo4j) cannot be reached: fix that connection, and later runs end normally.

Runs left this way do not change on their own; they stop counting once they fall outside the time range.

A port is already allocated​

docker compose up stops with Bind for 127.0.0.1:3000 failed: port is already allocated (or 5432). Something on the machine already uses that port. In .env, pick another:

GRAFANA_PORT=3300               # then open http://localhost:3300
LOGGING_DB_PUBLISHED_PORT=5433  # then install with --logging-port 5433

Hop cannot connect to the logging database​

Connection refused or a timeout in the Hop log. The host in LOGGING_HOSTNAME is resolved from wherever Hop runs:

  • Hop on the same machine as the bundled database: localhost, with the port from LOGGING_DB_PUBLISHED_PORT (default 5432).
  • Hop in a container: not localhost, which is the container itself. Use the database's service name on a shared Docker network, or the host's address.
  • Hop on another machine: the bundled database listens on 127.0.0.1 only. Set LOGGING_DB_BIND_ADDR=0.0.0.0 in .env, restrict access with a firewall, and use the host's address.

Grafana: password authentication failed​

Grafana reads with GRAFANA_DB_USER and GRAFANA_DB_PASSWORD from .env. logging-db-init creates that user, and sets its password, on every start. Check its output: docker compose -f docker-compose-monitoring.yml logs logging-db-init should say Grafana read-only login is present. If it warns instead, the user in LOGGING_DB_USER may not create users: see Use your own PostgreSQL.

If Hop fails to write instead, LOGGING_DB_* in .env and LOGGING_* in the Hop environment file must name the same database, user and password.

If you changed LOGGING_DB_PASSWORD after the bundled database was first created: PostgreSQL only reads it on first start. Change the password in the database as well:

docker compose -f docker-compose-monitoring.yml exec logging-db \
  psql -U hop_logging -d logging -c "ALTER USER hop_logging PASSWORD 'the-new-password'"

Hop fails with "violates foreign key constraint log_detail_channel_id_fkey"​

The Insert / update pipeline (or workflow) transform in the logging pipeline no longer commits every row. log_detail rows refer to log_object rows written over a separate connection, so each parent row must be committed before its detail row is written. Set the transform's commit size back to 1, or re-run the installer to restore the shipped pipelines.

The Project filter shows __PUTKI_PROJECT_NAME__​

The project was installed without --project-name. Re-run the installer with a name. Runs recorded before that keep the placeholder; rename them with:

UPDATE log_object SET project = 'sales' WHERE project = '__PUTKI_PROJECT_NAME__';

A run's log says "(no log detail for this execution)"​

Normal for rows that produced no log text, such as a Start action. If it happens for every workflow and pipeline, check that the project's logging/pipeline-log.hpl is the shipped one (re-run the installer) and that the log_detail table exists.

Compose: required variable is missing a value​

There is no .env, or one of the passwords in it is empty. Run ./init-env.sh: it fills in whichever of GRAFANA_ADMIN_PASSWORD, LOGGING_DB_PASSWORD and GRAFANA_DB_PASSWORD is missing.

Alerts do not arrive​

  • Is a chat set? docker compose -f docker-compose-monitoring.yml logs grafana | grep "Putki alerting" names the chats alerts go to. "nowhere" means no PUTKI_ALERT_*_WEBHOOK_URL reached Grafana: check .env and run up -d again.
  • Does the chat accept them? In Grafana, Alerting → Contact points → Putki alerts → Test sends a test message. If it fails, the error names the reason, usually a webhook that was deleted or a firewall that blocks Grafana from reaching hooks.slack.com or chat.googleapis.com.
  • Does the rule fire? Alerting → Alert rules → Putki shows each rule's state. A rule in Error cannot read the logs: see the datasource problems above. Run much slower than usual and the stuck rule need 5 successful runs of the same name first.

Single sign-on​

  • No button for a provider. Grafana's log says why: docker compose -f docker-compose-monitoring.yml logs grafana | grep "Putki SSO". Google stays off without GRAFANA_SSO_ALLOWED_DOMAINS, Entra ID without GRAFANA_ENTRA_TENANT_ID, OIDC without GRAFANA_OIDC_ISSUER.
  • The provider says the redirect URI does not match. Register exactly <GRAFANA_ROOT_URL>login/google, login/azuread or login/generic_oauth, with the scheme, host and path of GRAFANA_ROOT_URL.
  • Signed in, but with the wrong role. The lists in GRAFANA_SSO_ADMINS and GRAFANA_SSO_EDITORS match the address the provider sends; the user's profile in Grafana (Administration → Users) shows it. The role is set again at every sign-in.
  • "Login failed". The address is outside GRAFANA_SSO_ALLOWED_DOMAINS.
  • Signed in, but Grafana is empty. GRAFANA_SSO_DEFAULT_ROLE=None and the address is in neither list.

Logging in to Grafana does not stick​

GRAFANA_COOKIE_SECURE=true over plain HTTP. Remove it, or serve Grafana over HTTPS. See Expose Grafana safely.

Asking for help​

Send these to Putki support. Remove passwords first: .env and the Hop environment file both contain them in recoverable form.

cat VERSION; docker compose version
docker compose -f docker-compose-monitoring.yml ps -a
docker compose -f docker-compose-monitoring.yml logs grafana logging-db-init --tail 100
./install-logging.sh --project-home <project> --dry-run

Add the result of the query at the top of this page, and the Hop log of a run that should have been recorded.