Watching a Server for Drift

A WatchOp over a sql environment runs chant lifecycle diff <env> --live on a schedule and records the run’s Drift outcome. It works the same way for ClickHouse and Postgres. The example the lexicon ships, examples/drift-watch, is a ClickHouse project with one.

Declare the watch

ops/schema-watch.op.ts

import { WatchOp } from "@intentius/chant/op";

const { op } = WatchOp({ name: "schema-watch", env: "prod", schedule: "17 * * * *" });

export default op;

env selects sql.profiles.prod. The watch only reads, so give that profile a user that can only read:

chant.config.ts

export default {
  lexicons: ["sql"],
  ownership: { stack: "shop", env: "prod" },
  sql: {
    dialect: "clickhouse",
    profiles: {
      prod: {
        url: "https://clickhouse.example.com:8443",
        user: { env: "CLICKHOUSE_READER_USER" },
        password: { env: "CLICKHOUSE_READER_PASSWORD" },
        databases: ["shop"],
      },
    },
  },
} satisfies ChantConfig;

On ClickHouse, a user with SELECT on the project’s databases and readonly = 1 is enough. On Postgres, a role that can log in, with USAGE on the project’s schemas and default_transaction_read_only = on; the catalogs it reads are readable by every role.

What it reports

chant run schema-watch prints the diff and the outcome:

On the server	The diff	Drift
what the build declares, as applied	No drift detected	false
an object the project did not create, beside the owned ones	nothing; the diff reads only declared objects	false
an owned object’s property changed out of band (ALTER TABLE ... MODIFY TTL, a column default)	PROPERTY DRIFT, the property’s declared and live values	true
an owned object dropped	MISSING, the object and the address that was read	true
a Postgres table’s columns in another order than the declaration’s, such as a column a PostgresMigrationOp rename added last	nothing; columns are compared by name, since Postgres cannot move a column without rewriting the table	false

The diff compares the server with what the checkout declares, not with what was last applied, so a change merged but not yet applied reads as drift too.

On a ClickHouse cluster

With topology: "cluster:<name>" on the profile, the live read covers every replica of every shard, through the profile’s server (clusterAllReplicas('<name>', system.tables)), as well as the profile’s server itself. A change made on one server without ON CLUSTER (for example ALTER TABLE events MODIFY TTL ... run on shard 2 only) is drift even though the profile’s server still holds the declaration. When the servers’ copies of a table differ, the report shows the copy that departs most from the declaration and names its server:

  - events (ClickHouse::Table)
      ttl: ts + INTERVAL 30 DAY → ts + toIntervalDay(7) [seen on: ch-s2r1]

The read needs every server to answer. If one cannot be read, every object is reported unobserved with the reason; none is reported as matching.

A Distributed table’s cluster, database and table arguments compare the same whether they are declared as identifiers (Distributed(main, shop, events, id)) or as strings. The server prints them quoted.

Render it to CI

generateOpsPipeline renders the watch as a scheduled job on the Op’s own cron. Pass the reader’s credentials as the job’s variables and nothing else:

import { generateOpsPipeline } from "@intentius/chant/op";

await generateOpsPipeline(
  [{
    name: "schema-watch",
    variables: {
      CLICKHOUSE_READER_USER: "chant_reader",
      CLICKHOUSE_READER_PASSWORD: "${{ secrets.CLICKHOUSE_READER_PASSWORD }}",
    },
  }],
  "github",
  { beforeScript: ["npm ci", 'echo "$PWD/node_modules/.bin" >> "$GITHUB_PATH"'] },
);

Forge	Schedule	Repository token
GitHub	on.schedule	permissions: contents: read
Forgejo	on.schedule	the runner’s own; Forgejo reads no permissions: block, so none is written
GitLab	a Pipeline Schedule with CHANT_SCHEDULED_OP=schema-watch, noted at the top of the file	the job’s own CI_JOB_TOKEN; no other token is added

On GitLab, name the secret as a CI/CD variable ($CLICKHOUSE_READER_PASSWORD). chant run calls chant lifecycle itself, so chant has to be on the job’s path, which the beforeScript above sees to; on GitLab, export PATH="$PWD/node_modules/.bin:$PATH" does the same.